NVIDIA's cutting-edge software is combining digital twins with real-time artificial intelligence (AI) to create radical advancements in the field of industrial automation. By utilising a sophisticated suite of software - Omniverse, Metropolis, Isaac and cuOpt - NVIDIA provide a simulated environment, often termed an 'AI gym', in which robots and AI agents can train and be evaluated within complex industrial landscapes, proving crucial in sectors such as manufacturing, factory logistics, and robotics.

The simulation-first approach adopted by NVIDIA is ideal for industries involved with large, heavy items, expensive machinery or complex robotics and logistics. The concept is to utilise digital twins to design, test, and fine-tune large-scale real-time AI systems entirely within a simulation environment before implementing them in the physical infrastructure of the industrial sites. This innovative approach significantly cuts down the time and cost involved in implementing complex industrial automation strategies.

NVIDIA's suite of software interacts within the AI gym to enable developers to train AI agents efficiently. Such agents can then help navigate robots and humans through unpredictable or complex situations. NVIDIA's digital twin simulation is impressively comprehensive, encompassing a 100,000-square-foot warehouse, dozens of digital workers, several autonomous mobile robots (AMRs), vision AI agents, and sensors.

The software interacts efficiently, with each AMR processing visual information gathered from six sensors, all simulated within the digital twin environment. Meanwhile, the NVIDIA Metropolis platform for vision AI generates a comprehensive map of worker activity within the warehouse, aggregating data from 100 simulated ceiling-mounted camera streams via multi-camera tracking. This complete overview serves to inform optimal AMR routes, calculated by NVIDIA’s cuOpt software.

NVIDIA's cuOpt software is a ground-breaking optimisation AI microservice that solves complex routing problems using dynamic evolutionary algorithms, powered by GPU-acceleration. This all takes place in real time, under the guidance of Isaac Mission Control, which coordinates the entire fleet of AMRs using the data and routes generated by cuOpt to efficiently dispatch and execute commands.

Digital twins - the AI gyms where complex AI agents are developed - can significantly streamline industrial processes, and manage robot fleets in factories or enhance human-robot interactions in supply chain distribution centres. By providing physically accurate environments for AI evaluation, simulation and training, the software allows AI agents and AMRs to adapt to unpredictable real-world situations.

For instance, should an AMR's path be blocked, preventing it from picking up a pallet, the NVIDIA Metropolis software can instantly update an occupancy grid, mapping all humans, robots and objects in a single view. The cuOpt software then quickly formulates an optimal route avoiding the blockage, with the AMR responding accordingly to minimise downtime.

The Metropolis software also offers extensive visual insights by powering the NVIDIA Visual Insight Agent (VIA) framework. This enables AI agents to assist operations teams in answering queries about incidents, like a blockage in an aisle in the factory. These AI agents can rapidly offer insights, and even predict potential issues in the future.

The future looks bright for industrial digitalisation with NVIDIA's pioneering simulation-based training methodologies and their continuous deployment of modular NVIDIA NIM Inference microservices. Industries across the board are likely to benefit from the ability to summarise, search and extract actionable insights from videos using natural language.