<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Humanoids | Alexander Nettekoven</title><link>https://www.alexkoven.com/tag/humanoids/</link><atom:link href="https://www.alexkoven.com/tag/humanoids/index.xml" rel="self" type="application/rss+xml"/><description>Humanoids</description><generator>Wowchemy (https://wowchemy.com)</generator><language>en-us</language><lastBuildDate>Fri, 13 Nov 2026 00:00:00 +0000</lastBuildDate><image><url>https://www.alexkoven.com/media/sharing_2026.jpg</url><title>Humanoids</title><link>https://www.alexkoven.com/tag/humanoids/</link></image><item><title>Letting a Humanoid Run Its Packing Loop: Stage Prompts, Stage Ends and a Twin</title><link>https://www.alexkoven.com/project/humanoid_stage_policies/</link><pubDate>Fri, 13 Nov 2026 00:00:00 +0000</pubDate><guid>https://www.alexkoven.com/project/humanoid_stage_policies/</guid><description>&lt;figure class="post-clip" style="max-width: 100%; margin: 1.5rem auto;">
&lt;video controls muted playsinline loop preload="metadata" autoplay poster="lead.jpg" style="width: 100%; height: auto; border-radius: 6px;">
&lt;source src="lead.mp4" type="video/mp4">
&lt;/video>&lt;figcaption>Figure 1: The Astribot S1 on 2026-11-13 runs a π0.7 policy trained on stage spans of real demonstrations. It sets out a box, packs red bags into it and pushes the full box onto the conveyor, while Gemini picks each stage. Shown at four times real speed.&lt;/figcaption>
&lt;/figure>
&lt;p>In October I taught an Astribot S1 humanoid at Formic to pack boxes in four days, as described in the &lt;a href="https://www.alexkoven.com/project/humanoid_box_packing/">previous post&lt;/a>. In that demo the policy learned each stage from its own episodes, and a person or Gemini chose the next stage. Over the following month I worked on letting the robot run the loop on its own. Three things changed, namely the demonstrations, the way the policy learns its stages, and what decides that a stage is over. On 2026-11-13 the robot ran the loop with Gemini choosing the stages (Figure 1).&lt;/p>
&lt;details class="toc-collapsible">
&lt;summary class="toc-collapsible__summary">Table of Contents&lt;/summary>
&lt;div class="toc-collapsible__content">
&lt;nav id="TableOfContents">
&lt;ul>
&lt;li>&lt;a href="#whole-task-demonstrations">Whole-task demonstrations&lt;/a>&lt;/li>
&lt;li>&lt;a href="#a-policy-that-ignored-its-stage">A policy that ignored its stage&lt;/a>&lt;/li>
&lt;li>&lt;a href="#stage-prompts-the-policy-follows">Stage prompts the policy follows&lt;/a>&lt;/li>
&lt;li>&lt;a href="#stage-ends-from-software-stage-choice-from-gemini">Stage ends from software, stage choice from Gemini&lt;/a>&lt;/li>
&lt;li>&lt;a href="#the-mujoco-twin">The MuJoCo twin&lt;/a>&lt;/li>
&lt;li>&lt;a href="#what-still-needs-work">What still needs work&lt;/a>&lt;/li>
&lt;li>&lt;a href="#lessons">Lessons&lt;/a>&lt;/li>
&lt;li>&lt;a href="#links">Links&lt;/a>&lt;/li>
&lt;/ul>
&lt;/nav>
&lt;/div>
&lt;/details>
&lt;h2 id="whole-task-demonstrations">Whole-task demonstrations&lt;/h2>
&lt;p>From mid-October to early November I recorded new sets on the S1 that run the whole task in one go. Further sets aimed at the weak spots of the four-day policy: boxes placed at different spots on the table, bags grabbed from the middle of the table, and pushes near the conveyor edge. Together with the red-bag episodes from October, the newest policy learned from 213 real episodes.&lt;/p>
&lt;h2 id="a-policy-that-ignored-its-stage">A policy that ignored its stage&lt;/h2>
&lt;p>My first policy on the new data learned from 140 whole-task episodes under a single prompt. On 2026-11-10 it missed the box and then went on to pack an imaginary one. A policy trained on the whole task under one prompt cannot be told where it is, so when one step fails it rolls on to the next.&lt;/p>
&lt;h2 id="stage-prompts-the-policy-follows">Stage prompts the policy follows&lt;/h2>
&lt;p>The next policy, trained on 2026-11-11, learned from real demonstrations cut into stage spans. Each span carries the prompt of its stage, which is grabbing the box, packing the bags or pushing the box. Physical Intelligence&amp;rsquo;s own retrain of my October data, with the return-home motion masked out, had run better than any of my later policies, so I also cut the return home at the end of each episode.&lt;/p>
&lt;p>The prompts now steer the policy. When the prompt changes, its actions move by 0.16 to 0.19 rad, against 0.01 to 0.02 rad for the policy trained under a single prompt.&lt;/p>
&lt;h2 id="stage-ends-from-software-stage-choice-from-gemini">Stage ends from software, stage choice from Gemini&lt;/h2>
&lt;p>In the four-day demo, Gemini watched the head camera and switched prompts on its own judgment. Now the inference software ends each stage from what the arms do. While the robot packs, it tracks gripper releases and treats arms that stay slower than 0.155 rad/s for 5 s as a finished pack, then moves the robot to its working pose. Gemini, a multimodal model from Google, only chooses among the three stage prompts or holds. This split mirrors the supervisor on the &lt;a href="https://www.alexkoven.com/project/bimanual_yam_cycles/">two-arm YAM rig&lt;/a>: plain rules handle what the joints can tell, and the vision model handles what only the camera can tell.&lt;/p>
&lt;h2 id="the-mujoco-twin">The MuJoCo twin&lt;/h2>
&lt;p>Alongside the robot work I built a MuJoCo twin of the station, MuJoCo being a physics simulator, to make more data and to test policies before they reach the robot. Policies trained on scripted demonstrations in the twin learned to pack there. On the robot they packed too, but they struggled to grasp red bags, and four bags took 50 to 160 s against about 20 s in the twin.&lt;/p>
&lt;p>The twin also ranked policies differently from the robot. The four-day policy scored 0 of 100 in the twin, and the real-data policy in Figure 1 scored 18 of 266, although both ran the task on the robot. For now the robot decides which policy is better.&lt;/p>
&lt;h2 id="what-still-needs-work">What still needs work&lt;/h2>
&lt;p>The runs on 2026-11-13 showed the next things to fix:&lt;/p>
&lt;ul>
&lt;li>The policy regrasps about twice per bag. Counting gripper releases therefore ended packing after about two bags, until I switched to the rule for slow arms.&lt;/li>
&lt;li>In one run the box held four bags, Gemini counted three, and the arms crept for 39 s.&lt;/li>
&lt;li>Pushes stall when the right arm rests against the box at the conveyor edge.&lt;/li>
&lt;li>Each stage change holds the robot at its working pose for 6 to 11 s.&lt;/li>
&lt;li>Red bags remain the hardest grasp.&lt;/li>
&lt;/ul>
&lt;h2 id="lessons">Lessons&lt;/h2>
&lt;ul>
&lt;li>How you cut the demonstrations matters. Real episodes cut into stage spans, with the return home removed, gave a policy that follows its prompts. Whole-task episodes under one prompt gave a policy that rolls on regardless.&lt;/li>
&lt;li>Let software end stages and a vision model choose them. Arm speed and gripper releases are easy to measure, while the next stage depends on what the camera sees.&lt;/li>
&lt;li>Check a twin&amp;rsquo;s ranking against the robot before trusting its scores.&lt;/li>
&lt;/ul>
&lt;h2 id="links">Links&lt;/h2>
&lt;ul>
&lt;li>&lt;a href="https://www.alexkoven.com/project/humanoid_box_packing/">Teaching a Humanoid to Pack Boxes with π0.7 in Four Days&lt;/a>, the previous post&lt;/li>
&lt;li>&lt;a href="https://www.alexkoven.com/project/bimanual_yam_cycles/">Hands-Off Cycles on Two Robot Arms&lt;/a>, the supervisor this one borrows from&lt;/li>
&lt;li>&lt;a href="https://arxiv.org/abs/2604.15483" target="_blank" rel="noopener">π0.7 paper on arXiv&lt;/a>&lt;/li>
&lt;/ul></description></item><item><title>Teaching a Humanoid to Pack Boxes with π0.7 in Four Days</title><link>https://www.alexkoven.com/project/humanoid_box_packing/</link><pubDate>Wed, 14 Oct 2026 00:00:00 +0000</pubDate><guid>https://www.alexkoven.com/project/humanoid_box_packing/</guid><description>&lt;figure class="post-clip" style="max-width: 100%; margin: 1.5rem auto;">
&lt;video controls muted playsinline loop preload="metadata" autoplay poster="lead.jpg" style="width: 100%; height: auto; border-radius: 6px;">
&lt;source src="lead.mp4" type="video/mp4">
&lt;/video>&lt;figcaption>Figure 1: The Astribot S1 runs my fine-tuned π0.7 policy on 2026-10-11. It sets a box on the table, packs four bags, pushes the full box onto the conveyor and fetches the next box. Played at 2x speed.&lt;/figcaption>
&lt;/figure>
&lt;p>In early October 2026, I taught an Astribot S1 humanoid a three-stage packing task at Formic. Formic brings robot automation to manufacturers across the US, and humanoids are one way to reach lines where classic automation struggles. The robot runs π0.7 (pi0.7), the newest robot foundation model from Physical Intelligence (PI). I fine-tuned it on my own demonstrations through PI&amp;rsquo;s API. I recorded the first packing demonstrations on 2026-10-07, and on 2026-10-11 the robot ran the whole task on camera (Figure 1), after four fine-tunes in between.&lt;/p>
&lt;p>In four days, a few hundred teleoperated demonstrations and four fine-tunes took π0.7 from a first test run to the full task: setting out a box, packing four bags into it and pushing the full box onto the conveyor. Each fine-tune trained on PI&amp;rsquo;s servers in four to nine hours, so every day held one round of data, training and robot tests.&lt;/p>
&lt;details class="toc-collapsible">
&lt;summary class="toc-collapsible__summary">Table of Contents&lt;/summary>
&lt;div class="toc-collapsible__content">
&lt;nav id="TableOfContents">
&lt;ul>
&lt;li>&lt;a href="#setup">Setup&lt;/a>
&lt;ul>
&lt;li>&lt;a href="#the-robot">The robot&lt;/a>&lt;/li>
&lt;li>&lt;a href="#compute-and-software">Compute and software&lt;/a>&lt;/li>
&lt;li>&lt;a href="#what-π07-is">What π0.7 is&lt;/a>&lt;/li>
&lt;/ul>
&lt;/li>
&lt;li>&lt;a href="#data-collection">Data collection&lt;/a>&lt;/li>
&lt;li>&lt;a href="#method">Method&lt;/a>
&lt;ul>
&lt;li>&lt;a href="#a-test-run-first">A test run first&lt;/a>&lt;/li>
&lt;li>&lt;a href="#the-four-fine-tunes">The four fine-tunes&lt;/a>&lt;/li>
&lt;li>&lt;a href="#running-the-policy">Running the policy&lt;/a>&lt;/li>
&lt;/ul>
&lt;/li>
&lt;li>&lt;a href="#results">Results&lt;/a>&lt;/li>
&lt;li>&lt;a href="#what-broke">What broke&lt;/a>&lt;/li>
&lt;li>&lt;a href="#lessons">Lessons&lt;/a>&lt;/li>
&lt;li>&lt;a href="#links">Links&lt;/a>&lt;/li>
&lt;/ul>
&lt;/nav>
&lt;/div>
&lt;/details>
&lt;h2 id="setup">Setup&lt;/h2>
&lt;h3 id="the-robot">The robot&lt;/h3>
&lt;p>The Astribot S1 is a wheeled humanoid with two arms of seven joints each, a two-joint head, a four-joint torso and two grippers. Its policy interface has 25 values per step, namely 20 joint positions, 3 base velocities and 2 gripper positions. The base never moved in my demonstrations, so the base velocities are exactly zero in every training frame. On the robot, my code sets them to zero again, and the robot bridge clamps the base speed to zero as a second guard.&lt;/p>
&lt;p>The S1 carries five cameras. I drop the head&amp;rsquo;s stereo pair and feed the policy the color streams of the other four, one each in the head, the torso and both wrists, at 640x360 pixels and 30 frames per second. Figure 2 shows what the head camera sees during a run.&lt;/p>
&lt;figure class="post-clip" style="max-width: 100%; margin: 1.5rem auto;">
&lt;video controls muted playsinline loop preload="metadata" poster="headcam.jpg" style="width: 100%; height: auto; border-radius: 6px;">
&lt;source src="headcam.mp4" type="video/mp4">
&lt;/video>&lt;figcaption>Figure 2: The robot&amp;rsquo;s head camera during a run of the fourth fine-tune (v4) on 2026-10-11. The policy also sees the torso and both wrist cameras. Played at 2x speed.&lt;/figcaption>
&lt;/figure>
&lt;h3 id="compute-and-software">Compute and software&lt;/h3>
&lt;p>No policy runs on the robot. π0.7 runs on PI&amp;rsquo;s servers, and the robot reaches it over a WebSocket, a standard two-way web connection. The S1 has two onboard computers, an x86 PC and an NVIDIA Jetson Orin, which run its drivers, cameras and web interface. My laptop sits on the robot&amp;rsquo;s network and runs three Docker containers, packaged programs that run side by side:&lt;/p>
&lt;ul>
&lt;li>the bridge to Astribot&amp;rsquo;s software development kit (SDK), which takes control of the robot and executes joint commands at 100 Hz;&lt;/li>
&lt;li>a bridge from the robot&amp;rsquo;s ROS 1 to ROS 2 (ROS, the Robot Operating System, is the standard robot middleware);&lt;/li>
&lt;li>an inference kit with my model file, which sends observations to PI and turns the replies into commands.&lt;/li>
&lt;/ul>
&lt;p>Each request carries the latest camera frames, the 25-value state and a short text prompt. The server resizes every image to 448x448, keeps a history of six frames per camera, and returns a chunk of 50 future commands. PI&amp;rsquo;s real-time chunking runner, a client that requests the next chunk while the current one still plays, asks for a new chunk every 25 steps and joins it onto the motion in flight, so the arm does not stop while it waits for the network. A healthy round trip takes 175 to 190 ms.&lt;/p>
&lt;h3 id="what-π07-is">What π0.7 is&lt;/h3>
&lt;p>π0.7 is a vision-language-action model (VLA), a model that maps camera images and a text instruction to robot actions. According to PI&amp;rsquo;s paper, it has about 5 billion parameters, a 4-billion-parameter vision-language backbone started from Gemma 3, Google&amp;rsquo;s open model family, and an 860-million-parameter action expert trained with flow matching, a training method for generative models. It predicts chunks of 50 actions and takes up to four cameras with up to six history frames each. I used the history variant of the base model and fine-tuned it through PI&amp;rsquo;s partner API, which opened as a beta on 2026-10-13.&lt;/p>
&lt;h2 id="data-collection">Data collection&lt;/h2>
&lt;p>I recorded every demonstration by teleoperating the S1, steering it remotely, in Astribot&amp;rsquo;s own data-collection mode. The task copies a packing line in three stages, and each stage has its own episodes and its own text prompt:&lt;/p>
&lt;ol>
&lt;li>&lt;code>grab a box on the left and place it on the table&lt;/code>, which takes an empty box from the side table on the robot&amp;rsquo;s left and sets it on the work table;&lt;/li>
&lt;li>&lt;code>put a red bag in the box until the box has 4&lt;/code>, which picks bags from the side table one at a time;&lt;/li>
&lt;li>&lt;code>push the box onto the conveyor on the right&lt;/code>, which slides the full box onto a roller conveyor.&lt;/li>
&lt;/ol>
&lt;p>The first 132 episodes, recorded on 2026-10-07, held 42,009 frames, about 23 minutes of robot time. The table below shows how the data grew over the four rounds.&lt;/p>
&lt;table>
&lt;thead>
&lt;tr>
&lt;th>Prompt&lt;/th>
&lt;th>v1&lt;/th>
&lt;th>v2&lt;/th>
&lt;th>v3&lt;/th>
&lt;th>v4&lt;/th>
&lt;/tr>
&lt;/thead>
&lt;tbody>
&lt;tr>
&lt;td>Grab a box&lt;/td>
&lt;td>45&lt;/td>
&lt;td>76&lt;/td>
&lt;td>110&lt;/td>
&lt;td>126&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Pack clear bags&lt;/td>
&lt;td>50&lt;/td>
&lt;td>103&lt;/td>
&lt;td>141&lt;/td>
&lt;td>141&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Pack red bags&lt;/td>
&lt;td>&lt;/td>
&lt;td>&lt;/td>
&lt;td>&lt;/td>
&lt;td>57&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Push onto the conveyor&lt;/td>
&lt;td>37&lt;/td>
&lt;td>47&lt;/td>
&lt;td>60&lt;/td>
&lt;td>80&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Center the box&lt;/td>
&lt;td>&lt;/td>
&lt;td>16&lt;/td>
&lt;td>16&lt;/td>
&lt;td>16&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Helper prompts (looks, idle, go home)&lt;/td>
&lt;td>&lt;/td>
&lt;td>16&lt;/td>
&lt;td>16&lt;/td>
&lt;td>16&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>Total episodes&lt;/td>
&lt;td>132&lt;/td>
&lt;td>258&lt;/td>
&lt;td>343&lt;/td>
&lt;td>436&lt;/td>
&lt;/tr>
&lt;/tbody>
&lt;/table>
&lt;p>The first product came in clear plastic bags. On 2026-10-10 I switched to red snack bags. That switch took 57 new packing episodes, plus 36 new grab and push episodes with the red bags on the table.&lt;/p>
&lt;p>Three data problems cost me time:&lt;/p>
&lt;ul>
&lt;li>The capture tool never exports the task text the operator types. I checked 3,785 episodes in nine task folders, and every one stored only its task ID. So the prompt must go into the upload metadata, which PI prefers over the dataset&amp;rsquo;s own field.&lt;/li>
&lt;li>Four episodes sat in the wrong task folder, because the operator kept recording under the old task ID. No metadata showed the error; only the video did.&lt;/li>
&lt;li>The merge tool ran in a Docker container as root and wrote the merged videos as root-only files. The upload then failed on the first video, although the dry run passed, because the dry run reads only metadata.&lt;/li>
&lt;/ul>
&lt;h2 id="method">Method&lt;/h2>
&lt;h3 id="a-test-run-first">A test run first&lt;/h3>
&lt;p>Before the packing work, I ran one fine-tune on 100 episodes of a simpler flip-the-box task, about 15.5 minutes of data, to test the path from upload to robot. That run trained in about 2.5 hours. I then pushed three recorded frames through the hosted model from my laptop, with no robot involved, and compared the returned chunks with the recorded actions. The mean error over arms, head and torso was 0.003 to 0.005 rad, close to the 0.002 to 0.009 rad of an earlier model for the UR5e, a six-joint industrial arm, which went on to work on its robot.&lt;/p>
&lt;p>The same check showed small base velocities of about ±0.006, although the training data held exact zeros. So the model was interpolating, and my model file zeroes the base on every row.&lt;/p>
&lt;p>I also trained on absolute joint targets, PI&amp;rsquo;s default, rather than on deltas. The base action is zero in every frame while the base state carries odometry noise, the jitter in the base&amp;rsquo;s estimate of its own motion. With deltas, the model would learn to predict that noise with the sign flipped, and the server would add the state back at run time, which turns noise into real base commands.&lt;/p>
&lt;h3 id="the-four-fine-tunes">The four fine-tunes&lt;/h3>
&lt;p>All runs made about 50 passes over their data at a batch size of 256, so the step count grew with the data. A run went from upload to a hosted checkpoint within hours, so I could try a change on the robot the same day or the next morning.&lt;/p>
&lt;table>
&lt;thead>
&lt;tr>
&lt;th>Run&lt;/th>
&lt;th>Trained (UTC)&lt;/th>
&lt;th>Episodes&lt;/th>
&lt;th>Prompts&lt;/th>
&lt;th>Steps&lt;/th>
&lt;th>Training time&lt;/th>
&lt;th>What changed&lt;/th>
&lt;/tr>
&lt;/thead>
&lt;tbody>
&lt;tr>
&lt;td>v1&lt;/td>
&lt;td>2026-10-08&lt;/td>
&lt;td>132&lt;/td>
&lt;td>3&lt;/td>
&lt;td>8,249&lt;/td>
&lt;td>3 h 39 min&lt;/td>
&lt;td>The three stages&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>v2&lt;/td>
&lt;td>2026-10-09&lt;/td>
&lt;td>258&lt;/td>
&lt;td>9&lt;/td>
&lt;td>15,699&lt;/td>
&lt;td>5 h 42 min&lt;/td>
&lt;td>Box centering, helper prompts, more stage data&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>v3&lt;/td>
&lt;td>2026-10-10&lt;/td>
&lt;td>343&lt;/td>
&lt;td>9&lt;/td>
&lt;td>19,899&lt;/td>
&lt;td>7 h 28 min&lt;/td>
&lt;td>85 more stage episodes&lt;/td>
&lt;/tr>
&lt;tr>
&lt;td>v4&lt;/td>
&lt;td>2026-10-11&lt;/td>
&lt;td>436&lt;/td>
&lt;td>10&lt;/td>
&lt;td>25,099&lt;/td>
&lt;td>9 h 04 min&lt;/td>
&lt;td>Red bags, renamed packing prompts, weighted mix&lt;/td>
&lt;/tr>
&lt;/tbody>
&lt;/table>
&lt;p>v2 added a prompt that centers the box and a few helper prompts, such as a scripted look to the left for the next box and a return to the working pose. It also renamed the push prompt. PI cannot rename a stored prompt, so all 258 episodes went up again.&lt;/p>
&lt;p>For v3, PI does not skip episodes it already holds, so I uploaded only the 85 new ones and trained on the v2 set plus the new set.&lt;/p>
&lt;p>For v4, I renamed the packing prompts so that each one names what the camera sees, &lt;code>put a clear bag in the box until the box has 4&lt;/code> and &lt;code>put a red bag in the box until the box has 4&lt;/code>. Renaming meant uploading all 436 episodes again, which took 24 minutes. I split the data into two sets, red-bag packing and everything else, and weighted them 0.289 and 0.711 so that red and clear packing get equal shares of each batch.&lt;/p>
&lt;p>PI reports a validation error on held-out episodes. v2 ended at a mean absolute error (MAE) of about 0.16. v3 and v4 each report two numbers, likely one per data set, and v4&amp;rsquo;s (0.49 and 0.47) sit well above v3&amp;rsquo;s (0.15 and 0.41). The sets, and so the held-out episodes, differ between runs, so these numbers do not rank the runs. Only robot trials can rank them.&lt;/p>
&lt;h3 id="running-the-policy">Running the policy&lt;/h3>
&lt;p>A small menu on the laptop sends the stage prompts. Each number key sends one prompt, and a switch reaches the robot within about 1 s while it keeps moving, with no reset. Before a stage starts from rest, a warmup walks the robot to the stage&amp;rsquo;s start pose, the median first frame of its training episodes, if the robot is more than 0.3 rad away.&lt;/p>
&lt;p>The menu also has an automatic mode. Gemini Flash-Lite, a small multimodal model from Google, looks at five recent head-camera frames and a reference picture of the station every 2 s and picks the next prompt. It switches only after two agreeing answers, it counts the bags in the box, and it costs about 4.3 USD per hour.&lt;/p>
&lt;p>Two settings shape the motion. A playback speed of 1.0 plays PI&amp;rsquo;s actions at the 30 Hz of the training data, and 1.2 plays them 20 percent faster. A smoothing filter, an exponential moving average (EMA) on the 20 joint commands, removes a small tick-to-tick jitter in the hosted model&amp;rsquo;s output.&lt;/p>
&lt;p>The rule that shapes my model file is simple. In the inference kit, an exception in the model makes the client return nothing, and the robot keeps executing the rest of the old chunk. So the model must never raise. The few lines below show the idea in my own simplified code.&lt;/p>
&lt;pre>&lt;code class="language-python">def infer(self, obs: dict) -&amp;gt; np.ndarray:
try:
chunk = self.client.infer(obs) # (50, 25) joint commands
chunk[:, 20:23] = 0.0 # the base never moves
return chunk
except Exception:
return np.tile(self.last_command, (50, 1)) # hold in place instead of failing
&lt;/code>&lt;/pre>
&lt;h2 id="results">Results&lt;/h2>
&lt;p>Figure 1 shows the v4 checkpoint on 2026-10-11, the run I filmed for the demo. In 92 seconds of real time, the robot packs two boxes and pushes both onto the conveyor. In the first cycle, the fourth bag lands at 37 s and the push starts at 40 s. Figure 3 shows a stretch of the same run at real speed. The demo ran with the playback speed at 1.3 and the smoothing at 0.5.&lt;/p>
&lt;figure class="post-clip" style="max-width: 100%; margin: 1.5rem auto;">
&lt;video controls muted playsinline loop preload="metadata" poster="realspeed.jpg" style="width: 100%; height: auto; border-radius: 6px;">
&lt;source src="realspeed.mp4" type="video/mp4">
&lt;/video>&lt;figcaption>Figure 3: The same v4 run at real speed, from grabbing the box to packing the first bag and reaching for the second.&lt;/figcaption>
&lt;/figure>
&lt;p>The robot is still slow, which is why Figure 1 runs at 2x.&lt;/p>
&lt;p>What the demo does show is that a few hundred demonstrations and four short fine-tunes got π0.7 through a three-stage task on the humanoid. The switch to red bags took one round of data and one overnight run.&lt;/p>
&lt;h2 id="what-broke">What broke&lt;/h2>
&lt;p>Most of the problems sat in the plumbing, not in the policy:&lt;/p>
&lt;ul>
&lt;li>The arm jerked back at every replan. In its original mode, the inference kit stopped sending commands during each call to PI, which happened every 20 rows. I measured 51 stalls of 180 to 240 ms in 45 s. PI&amp;rsquo;s real-time chunking runner, the same one the UR5e and the YAM arms (a two-arm research setup) use, fixed the stalls. A second bug made PI&amp;rsquo;s replies arrive one step late, because the kit reset the connection while the policy still used it. Now one thread talks to PI, and a reset only raises a flag.&lt;/li>
&lt;li>Before the packing work, the head and torso cameras reached the kit at 6 Hz. The bridge sent raw 2.76 MB frames over the network, and the messaging layer split each one into many packets. Switching to compressed images restored 30 Hz and cut outbound traffic from 117.74 to 0.28 MB/s.&lt;/li>
&lt;li>The laptop failed silently twice. Its firewall dropped the messages between the containers, and a virtual private network (VPN) interface gave the messaging layer a second network to send on. A new box between the laptop and the office network also turned out to be a router, not a switch, and cut the laptop off.&lt;/li>
&lt;li>From the standing home pose, the first chunk asked for a 0.40 rad torso move, because every stage starts bent forward, about 0.58 rad from home on that joint. The warmup to the start pose fixed it.&lt;/li>
&lt;li>At the bridge&amp;rsquo;s old filter setting, the arm trailed the policy by up to 0.42 rad on elbow swings. A higher setting fixed the whipping.&lt;/li>
&lt;li>The warmup first drove the grippers to 100, which is closed on the S1, and the policy then reopened them. Now the warmup opens them.&lt;/li>
&lt;li>During one home move, the automatic mode pressed a prompt, the waist jumped 0.65 rad, and the drivers shut off. Now the kit refuses any prompt until the home move ends.&lt;/li>
&lt;li>The hosted model&amp;rsquo;s joint commands step back and forth by about 1.3 mrad every tick, although the training actions are smooth. An EMA at 0.3 cuts that noise to about a quarter, at the cost of about 80 ms of delay.&lt;/li>
&lt;li>On the demo day, v4 showed bursts in 4 of about 26 runs. After a pause of 70 to 900 ms in the command stream, 10 to 16 joints jumped at once by 0.6 to 0.8 rad of raw command for two ticks. Slow replies that drain the chunk queue are the likely cause, made worse by the faster playback, but I have not confirmed it.&lt;/li>
&lt;li>Once the head camera simply stopped, and the automatic mode did nothing, because the policy had no full set of images. A restart of the camera stack fixed it.&lt;/li>
&lt;/ul>
&lt;h2 id="lessons">Lessons&lt;/h2>
&lt;ul>
&lt;li>Test the hosted model offline before the robot moves. Three recorded frames through the server caught the drifting base velocities before any rollout.&lt;/li>
&lt;li>Never let the model file raise. A crash should become a hold, not a silent replay of old commands.&lt;/li>
&lt;li>Plumbing faults look like bad policies. A stale camera stream feeds a six-frame history model old images, which fails quietly and looks like a weak policy.&lt;/li>
&lt;li>Put the prompt in the upload metadata, and check what the server stored. A prompt cannot be renamed later without uploading the data again.&lt;/li>
&lt;li>Name what the camera sees in the prompt. Splitting &amp;ldquo;clear bag&amp;rdquo; from &amp;ldquo;red bag&amp;rdquo; gave the model a visual cue for each product.&lt;/li>
&lt;li>Validation error does not rank runs that trained on different data.&lt;/li>
&lt;li>Fast fine-tunes change how you work. With runs of a few hours, each day held one round of data, training and robot tests.&lt;/li>
&lt;/ul>
&lt;p>Since the demo, the S1 has moved on to whole-task demonstrations cut into stage spans, stage ends set by the inference software and a MuJoCo digital twin of the station. The &lt;a href="https://www.alexkoven.com/project/humanoid_stage_policies/">follow-up post&lt;/a> covers that work.&lt;/p>
&lt;h2 id="links">Links&lt;/h2>
&lt;ul>
&lt;li>&lt;a href="https://www.linkedin.com/posts/alexander-nettekoven_excited-to-share-some-of-my-recent-work-at-ugcPost-7516241176472150016-WC4C/" target="_blank" rel="noopener">LinkedIn post of 2026-10-14&lt;/a>&lt;/li>
&lt;li>&lt;a href="https://www.pi.website/blog/api-beta" target="_blank" rel="noopener">Physical Intelligence API beta&lt;/a>&lt;/li>
&lt;li>&lt;a href="https://arxiv.org/abs/2604.15483" target="_blank" rel="noopener">π0.7 paper on arXiv&lt;/a>&lt;/li>
&lt;/ul></description></item></channel></rss>