Using AI Agents to Teach Robots Like Microduck to Walk in Simulation

extract

Teaching a robot to navigate an obstacle course requires more than a walking policy. It must perceive its surroundings, maintain balance, and adjust its actions when obstacles block the path ahead. Developing these capabilities connects several stages: creating assets and an environment, using physics simulation to model how the robot interacts with its environment, and training and evaluating policies in simulation. NVIDIA RTX Spark serves as a local development hub, supporting AI-assisted planning and policy validation. NVIDIA Isaac Lab provides the robot-learning framework for reinforcement learning in the cloud, while Newton models physical behavior, and NVIDIA Omniverse libraries generate simulated visual observations through RTX rendering. Trained policies return to RTX Spark for local evaluation in NVIDIA Isaac Sim. This post walks through the workflow using Microduck as a reference robot, with Hermes, NVIDIA Nemotron, and Astra assisting with planning and preparation. You’ll learn how to create a simulated obstacle course, train a navigation policy, and evaluate its behavior in simulation—a process you can adapt to your own robots and tasks. This establishes a foundation for physical deployment, where differences between simulation and real-world behavior require further refinement. 1. Create a plan of the obstacle course The workflow begins with a prompt describing the obstacle course and what Microduck must learn. On RTX Spark, Hermes and NVIDIA Nemotron 3.5 Bolt translate that prompt into a step-by-step project plan. The instructions establish the intended task, available resources, expected outputs, and the stages the agents should follow. Use the following example to structure your prompt: Build and test a simulated Microduck obstacle course with navigation and locomotion challenges. Make a photorealistic concept image and have Astra build and test it. ... Workflow Use Hermes and Nemotron 3.5 Bolt for preparation and Astra for asset creation and simulation. ... 1. Research and plan Review official Microduck documentation and summarize capabilities, constraints, and unknowns. Define the course layout, obstacle dimensions, clearances, and intended route. ... 2. Generate the concept Use a verified Microduck reference image to generate a photorealistic course concept. Check robot appearance, relative scale, and movement space; save prompts and provenance. ... 3. Identify assets Extract unique reusable assets, group repeated instances, and select representative crops. Record dimensions, locations, and reference limitations; preserve original images. ... 4. Prepare the handoff Package the plan, robot brief, concept, asset inventory, crops, and execution instructions. Verify required files, delegate to Astra, and wait for confirmed results. ... 5. Create and verify assets Have Astra enhance references, model geometry in FreeCAD, and apply materials in Blender. Before simulation, verify every asset is completed or demonstrably blocked, with at least one usable model. ... 6. Assemble and test Verify robot capabilities, assemble the course, and test navigation, challenges, and resets. Provide reproducible instructions and evidence; do not operate physical hardware. ... Local planning includes a brief review of Microduck’s repository and documentation. This review identifies the robot’s capabilities, resources for subsequent work, and details that still need verification. The resulting plan extends beyond asset creation: it outlines environment construction, robot preparation, simulation, training, validation, and deployment. The course needs to test both movement and perception. Microduck must learn to move around obstacles and through constrained spaces while maintaining balance. Use these requirements to guide the proposed layout and the objects needed to construct it. A local image-generation model produces a concept image of the course. Hermes and Nemotron then analyze the image and create an inventory of its visible, reusable assets. Each asset receives an identifier, a description of its visible features, and a cropped reference image. The inventory records where objects are cut off or obscured so that later stages can distinguish visible details from assumptions. Hermes combines these outputs into a handoff package: the overall project plan, Microduck research and source links, the concept image, the asset inventory, cropped references, and instructions for the next stages. It then automatically delegates the work to Astra. The package gives Astra the broader project context and asset specifications needed to continue. 2. Create simulation assets Using the project plan and references from the previous stage, Astra creates the 3D assets needed for the course. Cropping objects from a larger concept image can result in small, low-resolution references. Before modeling, Astra uses cloud image editing to isolate the objects and produce clearer, higher-resolution images. Astra compares the enhanced references with the original crops, checking that silhouettes, proportions, colors, and distinctive features remain consistent. The enhanced images provide clearer modeling guidance, while the originals document what was visible in the concept image. Dimensions and hidden surfaces that cannot be established from the images are recorded as modeling assumptions. Astra directs FreeCAD to construct the base geometry and editable CAD structure, then uses Blender to add fine geometric details and materials. Both applications run locally through programmatic execution. The generated assets are saved as editable models and exported USD assets, with previews and output checks for review. You can also get assets from SimReady Central Library, a free and ungated library of more than 2,000 simulation-ready OpenUSD assets. Reusing them alongside newly generated objects reduces the asset preparation needed to assemble the obstacle course in Isaac Sim. The resulting assets become the building blocks of Microduck’s training environment in simulation. 3. Assemble the simulation environment Astra composes the obstacle course using the prepared assets in Isaac Sim. Isaac Sim provides access to Omniverse libraries’ rendering and sensor-simulation capabilities, while Newton models Microduck’s physical behavior, including movement and balance. A robot’s articulated structure and movement are defined in simulation through links, joints, and actuators. These elements establish the relationship between the policy’s actions and the simulated robot. Links represent connected bodies, joints constrain their relative motion, and actuators drive motion. Together, they model a robot interacting with its environment. AI agents can help configure these elements. Simulated sensor observations provide information about the obstacles ahead. As the robot attempts to walk, the simulation captures the consequences of its actions: whether it maintains its posture, makes progress through the course, or stumbles and falls. The result is a complete learning environment: a robot that can act, an obstacle course it can interact with, and a simulated environment that can generate feedback from those interactions. Reinforcement learning (RL) uses this feedback to train robot policies. 4. Train the perception and locomotion policy With Microduck and its environment ready, NVIDIA Isaac Lab provides the tools to train and evaluate robot policies at scale. This open-source framework runs many Microduck simulations at once, generating experience that an RL algorithm uses to improve its shared policy. For each task, you define what the robot observes, the actions it can take, and the rewards used to guide learning. You then implement the task using Isaac Lab APIs. Newton, an open-source, GPU-accelerated physics engine, simulates how Microduck moves and interacts with the ground and its environment in response to those actions. Early attempts may include unstable steps, collisions, and falls. The training algorithm uses these outcomes and the task’s rewards to update the policy toward more successful behavior. Each attempt follows the same simulation loop: Simulated observations → policy actions → simulated motion → updated observations Parallel simulation provides experience from many attempts at once to train the policy. In this task, ramps and uneven surfaces are the obstacles: Microduck must traverse a curved ramp while maintaining balance and avoiding falls. Varying the terrain gives the policy experience adjusting its movements to different ground conditions. The ramp and obstacle training experiment uses Astra-generated assets from the previous sections and the Isaac Lab Microduck example environment as its starting point. A locomotion policy pretrained in that environment receives additional training in Isaac Lab for traversing ramps and obstacles, using head-camera depth and robot-state observations. This adaptation run used 4,096 parallel environments on an NVIDIA RTX PRO 6000 Blackwell Workstation Edition GPU. 5. Validate the trained policy on RTX Spark After training, validate the trained policy in Isaac Sim on a local RTX Spark. Microduck receives simulated observations, predicts actions, moves through the environment, and responds to the updated scene based on the policy. Check whether Microduck detects obstacles, adjusts its movement, and completes the course. This stage separates policy learning from policy evaluation. During training, the policy changes as it receives new experience. During evaluation, you examine the behavior of a selected policy and identify where it succeeds or needs further work. Record course completion, falls, collisions, and the situations in which progress stops. These measurements, paired with rollout videos, can guide iterations to the environment or training setup. 6. Prepare for policy deployment on the physical robot Once the policy performs reliably in simulation, the next step is to run it on the robot’s onboard computer. This post covers the work leading up to that point: creating assets, assembling the environment, and training and evaluating the policy. Detailed hardware deployment is outside the scope. Preparing for the simulation-to-real transition starts with a simulation that reflects how the real robot behaves. Isaac Lab supports Better Actuator Models (BAM) to model servo response, including command delays, mechanical backlash (the small amount of play in the drivetrain), and drops in voltage as motors draw power from a shared battery. Matching these settings to measurements from Microduck can help narrow the sim-to-real gap. Even with a carefully calibrated model, sensors, ground contact, and control timing can differ on the real robot. Isaac Lab’s domain randomization varies physical parameters and observation noise within realistic ranges, giving the policy practice handling those differences. Hardware tests then show where further adjustments are needed, guiding the next round of simulation, training, and validation. Figure 10 shows a locomotion policy trained in Isaac Lab running on the physical Microduck. Get started This workflow connects AI-assisted planning, simulation, asset creation, and robot policy training and evaluation. It combines local asset and environment generation, cloud-scale policy learning, and local policy validation on the RTX Spark. AI agents turn prompts into plans, with Isaac Lab supporting policy training and Isaac Sim supporting evaluation before deployment. Learn more about the platforms used in this workflow, and try the examples in their documentation: - Access ungated assets from SimReady Central Library - Omniverse libraries for integrating simulation capabilities into applications - RTX Spark - Installing Isaac Sim on RTX Spark—download the Windows x86 binaries (runnable out of the box via Windows x86 emulation on Arm) - Isaac Sim for robot and environment simulation - Newton for physics simulation - Isaac Lab for reinforcement learning and imitation learning policy training - Isaac Lab Learning Path - Microduck example Isaac Lab environment - Microduck sim-to-real tutorial on Hugging Face - Microduck walking demo using native BAM servos Acknowledgements This workflow was developed through collaboration across teams at NVIDIA, including: Aaron Young, Antoine Richard, Chase Telegin, Gavriel State, Hammad Mazhar, Kelly Guo, Milad Rakhsha, Oyindamola Omotuyi, Pierre Fleau, Rishabh Chadha, Rogelio Olguin, Ruben D’Sa, and Yenal Kal.