<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Simulation | Alexander Nettekoven</title><link>https://www.alexkoven.com/tag/simulation/</link><atom:link href="https://www.alexkoven.com/tag/simulation/index.xml" rel="self" type="application/rss+xml"/><description>Simulation</description><generator>Wowchemy (https://wowchemy.com)</generator><language>en-us</language><lastBuildDate>Fri, 13 Nov 2026 00:00:00 +0000</lastBuildDate><image><url>https://www.alexkoven.com/media/sharing_2026.jpg</url><title>Simulation</title><link>https://www.alexkoven.com/tag/simulation/</link></image><item><title>Letting a Humanoid Run Its Packing Loop: Stage Prompts, Stage Ends and a Twin</title><link>https://www.alexkoven.com/project/humanoid_stage_policies/</link><pubDate>Fri, 13 Nov 2026 00:00:00 +0000</pubDate><guid>https://www.alexkoven.com/project/humanoid_stage_policies/</guid><description>&lt;figure class="post-clip" style="max-width: 100%; margin: 1.5rem auto;">
&lt;video controls muted playsinline loop preload="metadata" autoplay poster="lead.jpg" style="width: 100%; height: auto; border-radius: 6px;">
&lt;source src="lead.mp4" type="video/mp4">
&lt;/video>&lt;figcaption>Figure 1: The Astribot S1 on 2026-11-13 runs a π0.7 policy trained on stage spans of real demonstrations. It sets out a box, packs red bags into it and pushes the full box onto the conveyor, while Gemini picks each stage. Shown at four times real speed.&lt;/figcaption>
&lt;/figure>
&lt;p>In October I taught an Astribot S1 humanoid at Formic to pack boxes in four days, as described in the &lt;a href="https://www.alexkoven.com/project/humanoid_box_packing/">previous post&lt;/a>. In that demo the policy learned each stage from its own episodes, and a person or Gemini chose the next stage. Over the following month I worked on letting the robot run the loop on its own. Three things changed, namely the demonstrations, the way the policy learns its stages, and what decides that a stage is over. On 2026-11-13 the robot ran the loop with Gemini choosing the stages (Figure 1).&lt;/p>
&lt;details class="toc-collapsible">
&lt;summary class="toc-collapsible__summary">Table of Contents&lt;/summary>
&lt;div class="toc-collapsible__content">
&lt;nav id="TableOfContents">
&lt;ul>
&lt;li>&lt;a href="#whole-task-demonstrations">Whole-task demonstrations&lt;/a>&lt;/li>
&lt;li>&lt;a href="#a-policy-that-ignored-its-stage">A policy that ignored its stage&lt;/a>&lt;/li>
&lt;li>&lt;a href="#stage-prompts-the-policy-follows">Stage prompts the policy follows&lt;/a>&lt;/li>
&lt;li>&lt;a href="#stage-ends-from-software-stage-choice-from-gemini">Stage ends from software, stage choice from Gemini&lt;/a>&lt;/li>
&lt;li>&lt;a href="#the-mujoco-twin">The MuJoCo twin&lt;/a>&lt;/li>
&lt;li>&lt;a href="#what-still-needs-work">What still needs work&lt;/a>&lt;/li>
&lt;li>&lt;a href="#lessons">Lessons&lt;/a>&lt;/li>
&lt;li>&lt;a href="#links">Links&lt;/a>&lt;/li>
&lt;/ul>
&lt;/nav>
&lt;/div>
&lt;/details>
&lt;h2 id="whole-task-demonstrations">Whole-task demonstrations&lt;/h2>
&lt;p>From mid-October to early November I recorded new sets on the S1 that run the whole task in one go. Further sets aimed at the weak spots of the four-day policy: boxes placed at different spots on the table, bags grabbed from the middle of the table, and pushes near the conveyor edge. Together with the red-bag episodes from October, the newest policy learned from 213 real episodes.&lt;/p>
&lt;h2 id="a-policy-that-ignored-its-stage">A policy that ignored its stage&lt;/h2>
&lt;p>My first policy on the new data learned from 140 whole-task episodes under a single prompt. On 2026-11-10 it missed the box and then went on to pack an imaginary one. A policy trained on the whole task under one prompt cannot be told where it is, so when one step fails it rolls on to the next.&lt;/p>
&lt;h2 id="stage-prompts-the-policy-follows">Stage prompts the policy follows&lt;/h2>
&lt;p>The next policy, trained on 2026-11-11, learned from real demonstrations cut into stage spans. Each span carries the prompt of its stage, which is grabbing the box, packing the bags or pushing the box. Physical Intelligence&amp;rsquo;s own retrain of my October data, with the return-home motion masked out, had run better than any of my later policies, so I also cut the return home at the end of each episode.&lt;/p>
&lt;p>The prompts now steer the policy. When the prompt changes, its actions move by 0.16 to 0.19 rad, against 0.01 to 0.02 rad for the policy trained under a single prompt.&lt;/p>
&lt;h2 id="stage-ends-from-software-stage-choice-from-gemini">Stage ends from software, stage choice from Gemini&lt;/h2>
&lt;p>In the four-day demo, Gemini watched the head camera and switched prompts on its own judgment. Now the inference software ends each stage from what the arms do. While the robot packs, it tracks gripper releases and treats arms that stay slower than 0.155 rad/s for 5 s as a finished pack, then moves the robot to its working pose. Gemini, a multimodal model from Google, only chooses among the three stage prompts or holds. This split mirrors the supervisor on the &lt;a href="https://www.alexkoven.com/project/bimanual_yam_cycles/">two-arm YAM rig&lt;/a>: plain rules handle what the joints can tell, and the vision model handles what only the camera can tell.&lt;/p>
&lt;h2 id="the-mujoco-twin">The MuJoCo twin&lt;/h2>
&lt;p>Alongside the robot work I built a MuJoCo twin of the station, MuJoCo being a physics simulator, to make more data and to test policies before they reach the robot. Policies trained on scripted demonstrations in the twin learned to pack there. On the robot they packed too, but they struggled to grasp red bags, and four bags took 50 to 160 s against about 20 s in the twin.&lt;/p>
&lt;p>The twin also ranked policies differently from the robot. The four-day policy scored 0 of 100 in the twin, and the real-data policy in Figure 1 scored 18 of 266, although both ran the task on the robot. For now the robot decides which policy is better.&lt;/p>
&lt;h2 id="what-still-needs-work">What still needs work&lt;/h2>
&lt;p>The runs on 2026-11-13 showed the next things to fix:&lt;/p>
&lt;ul>
&lt;li>The policy regrasps about twice per bag. Counting gripper releases therefore ended packing after about two bags, until I switched to the rule for slow arms.&lt;/li>
&lt;li>In one run the box held four bags, Gemini counted three, and the arms crept for 39 s.&lt;/li>
&lt;li>Pushes stall when the right arm rests against the box at the conveyor edge.&lt;/li>
&lt;li>Each stage change holds the robot at its working pose for 6 to 11 s.&lt;/li>
&lt;li>Red bags remain the hardest grasp.&lt;/li>
&lt;/ul>
&lt;h2 id="lessons">Lessons&lt;/h2>
&lt;ul>
&lt;li>How you cut the demonstrations matters. Real episodes cut into stage spans, with the return home removed, gave a policy that follows its prompts. Whole-task episodes under one prompt gave a policy that rolls on regardless.&lt;/li>
&lt;li>Let software end stages and a vision model choose them. Arm speed and gripper releases are easy to measure, while the next stage depends on what the camera sees.&lt;/li>
&lt;li>Check a twin&amp;rsquo;s ranking against the robot before trusting its scores.&lt;/li>
&lt;/ul>
&lt;h2 id="links">Links&lt;/h2>
&lt;ul>
&lt;li>&lt;a href="https://www.alexkoven.com/project/humanoid_box_packing/">Teaching a Humanoid to Pack Boxes with π0.7 in Four Days&lt;/a>, the previous post&lt;/li>
&lt;li>&lt;a href="https://www.alexkoven.com/project/bimanual_yam_cycles/">Hands-Off Cycles on Two Robot Arms&lt;/a>, the supervisor this one borrows from&lt;/li>
&lt;li>&lt;a href="https://arxiv.org/abs/2604.15483" target="_blank" rel="noopener">π0.7 paper on arXiv&lt;/a>&lt;/li>
&lt;/ul></description></item></channel></rss>