Don't Retrain the Robot—Steer It

Every roboticist who has deployed a pretrained policy knows the moment: the model that looked brilliant on the benchmark meets your cell, your lighting, or your slightly different fixture—and fails, consistently, in the same way. FlowDAgger, introduced by Murray et al. at Microsoft Research in 2026, offers a useful reframe: adaptation does not have to mean touching the weights.

01 · Why: the adaptation trilemma

When a deployed generative policy—a flow-matching or diffusion model such as π0.5 or GR00T—meets a failure mode outside its pretraining distribution, the standard options are familiar:

• Collect demonstrations and fine-tune. This is effective, but expensive, operationally heavy, and vulnerable to forgetting behavior that already worked.
• Run reinforcement learning on hardware. Exploration is sample-hungry and can be unsafe; under a realistic correction budget, sparse reward carries far less information than a corrective action.
• Live with the failure. In practice, this remains the default more often than anyone would like.

All three options accept the same premise: behavior lives in the weights, so changing behavior means changing weights. FlowDAgger rejects that premise.

02 · The noise input is a control surface

A flow-matching policy does not map a state directly to a motor command. It learns a velocity field: given a state and a partially formed action at time t, which direction should that action move? At inference, the policy draws a random noise vector, treats it as raw material, and integrates a learned ordinary differential equation until an action emerges.

The action is therefore a deterministic function of two inputs: state s and noise w. The world gives us the state, but we can choose the noise. Hold the weights fixed, vary w, and the same frozen model traces the behaviors it already considers plausible.

• Prior preservation by construction. The steered behavior stays within the frozen model's learned support, so skills stored in untouched weights cannot be catastrophically forgotten.
• Adaptation becomes a search problem in noise space: for each state, which w produces the behavior we want?

Two distinctions matter. First, action inversion does not correct the base policy's output. It asks which noise would have made the frozen model produce the expert action. Second, this is not a last-layer fine-tune. A separate three-layer MLP—5.9 MB beside a multi-billion-parameter base—only proposes initial noise for each observation.

03 · How: three technical moves

• A latent policy replaces the Gaussian draw. At deployment, πʷ(s) proposes a noise vector instead of sampling from N(0, I); the frozen base decodes it into an action.
• Sparse DAgger-style interventions provide dense supervision. An expert supplies a corrective action only when behavior is unsatisfactory, covering the states the policy actually visits.
• Action inversion maps each corrective action back to a noise target. Reverse-time integration supplies a candidate, then local refinement finds w* such that πgp(s, w*) ≈ a*.

The result is a supervised dataset of state and target-noise pairs. There is no reward model, critic, or PPO machinery. Instead of fighting the frozen policy with an output residual, the steering head cooperates with the policy's generative structure.

04 · What happened on our UR10e

Our cell used a UR10e arm with a gripper, exterior RGB, and wrist RGB. The task was to grasp a vial and seat it in a rack pocket; success requires the vial to be lifted more than two centimeters and seated. The base was a π0.5 LoRA fine-tuned on 60 episodes, producing 10-step action chunks and frozen at its final checkpoint.

Our expert was not a person operating a teleoperation rig. It was the deterministic inverse-kinematics controller that generated the training data. In this setting, FlowDAgger becomes a way to distill a classical controller into the noise space of a learned policy.

Across 100 held-out states, the head closed 78% of the distance between the stock fine-tune and the best action this frozen model could produce, reducing mean chunk-reconstruction error from 0.00530 to 0.00124. A sibling GR00T head built with the same approach closed 80% of its stock-to-oracle gap.

The 59% win rate understates the shape of the result: losses clustered near zero on easy states, while the gains grew with difficulty. Steering also weakened through the task—closing 79% of the gap in REACH, 68% in GRASP, 67% in TRANSPORT, and 51% in PLACE. Placement is where even the oracle struggled, marking the boundary between what steering can recover from this checkpoint and what will require another lever.

05 · Where Odyssey fits

Odyssey gives this experiment a mission-shaped home: a hardware-agnostic, open-source mission specification, lifecycle engine, and runner framework. The frozen checkpoint, steering head, evaluation criteria, and Stage B trial can be treated as versioned mission artifacts rather than an isolated notebook result. The same mission semantics can then move into Lovell AI Command Center for training, evaluation, transcripts, and deployment-readiness decisions.

That matters because the final question is operational, not architectural. Can the robot complete the mission reliably in closed loop? Odyssey preserves the definition of success while the team changes the policy, steering head, or runner underneath it.

For now, the result is narrower—and still useful. A policy we did not touch became measurably better in an afternoon, on the GPU it already occupied, without forgetting anything stored in its weights. The weights never moved. Only the noise did.