AI is moving beyond chatbots and copilots into the physical world. Across laboratories, factories and hospitals, a new generation of AI agents is beginning to work alongside people, helping them understand their environment, access knowledge and take action in real time.
However, building agentic systems that combine models, skills, harnesses, tools and an agentic runtime to help people perform hands-on work is challenging. To operate effectively in dynamic, real-world environments, these agents must do more than generate responses.
Like human workers, they need knowledge, tools and specialized skills to perceive and understand the world through video, audio and sensor data, interpret fast-changing conditions and spatial context, retrieve information from enterprise systems, reason about the next best action and use software tools to complete tasks. All of this must happen with low latency and in a way that supports the user without creating distraction.
NVIDIA XR AI is a developer library that helps developers build these agentic applications. By connecting inputs from AR glasses and XR devices with AI models, enterprise data, tools and accelerated computing, NVIDIA XR AI enables agents that can perceive, reason and act in the flow of work.
It provides a foundation for developers to build or connect skills and tools for enterprise XR applications, and simplifies the integration of multimodal perception, enterprise retrieval, reasoning models and agent orchestration. Together, these capabilities make it easier to build spatially aware, multimodal AI agents that deliver low-latency, context-aware assistance in AR and XR experiences.
The platform brings together four core capabilities:
- Ingests real-world signals from AR and XR devices, including video, audio, depth, pose and sensor data.
- Connects agents to specialized tools and services, including NVIDIA Metropolis and the NVIDIA Metropolis for video search and summarization (VSS) for visual AI and video understanding, and NVIDIA NeMo Retriever for enterprise knowledge retrieval and retrieval-augmented generation.
- Supports a broad ecosystem of AI models, including NVIDIA Nemotron reasoning models, NVIDIA Cosmos Reason and other compatible foundation models.
- Integrates agent orchestration and accelerated runtime services to help developers move from prototype to production.
NVIDIA NeMo Agent Toolkit enables tool use, reasoning workflows and multi-agent coordination, while NVIDIA accelerated computing platforms — including NVIDIA DGX Spark, NVIDIA DGX Station and NVIDIA RTX PRO systems — provide the infrastructure to run inference across cloud, data center and edge environments.
Together, these capabilities enable AI agents that can understand their surroundings, access enterprise knowledge, reason about complex tasks and deliver contextual assistance in real time.
Source link
#Hands #Free #AIs #NVIDIA #Brings #Agents #Glasses
