Learning on Physically Adaptive Systems
Many manipulation tasks are too complex to hand-code: the behaviour is implicit, the state space too high-dimensional, or the environment too variable for classical controllers. Learning-based methods are one of the solutions — but they need data, and good data requires good control. Once a teleoperation platform is in place, it becomes a data collection engine. Human demonstrations captured through teleoperation feed imitation learning pipelines, and the resulting policies can execute tasks autonomously — closing the loop from human intent to robot behaviour.
A wide range of manipulation tasks — harvesting, inspection, leaf removal, monitoring, trimming, seeding — can be learned from human demonstrations and executed autonomously by a policy model. Compared to traditional visual servoing approaches, learning-based policies carry more implicit decision-making capacity: they generalise across object appearance, position variation, and scene complexity without explicit programming.
Detects objects and moves to them — but decisions are explicit and brittle to distribution shift.
The policy autonomously clears every object it has learned — no explicit object list or sequencing rules needed.
Under an architecture designed to minimise sensitivity to environment variation, the policy remains stable under severe lighting disturbances.
No policy is perfect from the start. When a robot fails — because the environment shifted, the training data was insufficient, or the task varied — the natural response is to dump more data and retrain. This work proposes a more efficient path: a human observes the failure, provides the minimal corrective input needed, and the robot continues autonomously. The result is an online learning loop that recovers from failures far more efficiently than passive data accumulation.
A human provides minimal corrective guidance the moment the robot reaches its limit.
After targeted correction, success rate exceeds what random data collection achieves with far more data.
Autonomy and human control are not opposites — they exist on a spectrum. The most capable systems know when to act alone and when to defer. This demonstrates a robot that operates autonomously while remaining aware of human presence and intent, adjusting its behaviour to collaborate naturally in a shared workspace.
The right arm is human-controlled via teleoperation; the left arm runs the same autonomous policy as above — both operating simultaneously in a shared workspace.
Three directions building on this foundation: