In a self-improving robot-learning loop, data-generation throughput rose as an emergent, unrewarded byproduct of reinforcement learning.
Sidekick Reports an Emergent Capability in Its Robotic Learning System. As one robot learned to fold a towel, it also got better at producing its own training data, an improvement no one programmed, and possibly the first of several.
Data-Generation Throughput
Rising as a byproduct of learning to fold
As the robot got better at folding a towel, the loop began producing more of its own verified training data per fixed session, more complete episodes, and more of the scarce successful examples that teach the system what success looks like. Nothing in the reward asked for it. The objective scores one thing: whether the towel ends up folded. It has no term for speed, for how many episodes run, or for how much data comes out. So when the loop's data yield rose, it rose for free.
“That is the finding worth naming. It is easy to say reinforcement learning makes a robot better at its task. This is stranger.”
That is the finding worth naming. It is easy to say reinforcement learning makes a robot better at its task. This is stranger: the rate at which the system supplies itself with training signal improved on its own, as a side effect of learning, a capability the system was never trained to produce. Self-improvement that partly funds its own acceleration, where each cycle the robot gets better, it also gets faster at producing the fuel for the next one, without anyone building that feedback in.
The direction is the interesting part, and the implication is larger than the single effect: if one capability can surface from the loop without being trained for, others may follow. The machine that makes the machine got faster, and we never told it to.
Implication
If one capability can surface from the loop without being trained for, others may follow. The machine that makes the machine got faster, and we never told it to.