Physical Intelligence, a hot robotics startup, says its new robot brain can figure out tasks it was never taught
Physical Intelligence, a rapidly emerging robotics startup based in San Francisco, has recently made headlines with its innovative approach to robotic intelligence. The company has published research demonstrating that its latest model, known as π0.7, can instruct robots to perform tasks that they have never been explicitly trained to execute. This capability has surprised even the researchers at Physical Intelligence, marking a significant advancement in the field of robotics.
The Concept of Compositional Generalization
The core claim of Physical Intelligence’s research revolves around a concept called compositional generalization. This refers to the ability of the robot to combine skills learned in different contexts to solve new problems. Traditionally, robotic training has relied on rote memorization, where specific data is collected for individual tasks, and models are trained accordingly. However, π0.7 breaks this pattern by allowing robots to remix learned skills in novel ways.
Scaling Capabilities
Sergey Levine, co-founder of Physical Intelligence and a professor at UC Berkeley, explains that once the model surpasses the threshold of merely executing pre-learned tasks, its capabilities increase more than linearly with the amount of data. This scaling property mirrors advancements seen in other domains, such as language processing and computer vision.
Demonstrating New Capabilities
One of the most striking demonstrations of π0.7’s capabilities involved an air fryer, a device that the model had not seen during training. The research team discovered that the training dataset contained only two relevant episodes related to the air fryer: one where a different robot pushed the appliance closed and another from an open-source dataset where a robot placed a plastic bottle inside the fryer based on instructions.
Successful Task Execution
Despite this limited exposure, the model managed to synthesize its understanding of the air fryer and successfully attempted to cook a sweet potato. With verbal instructions guiding it through the task, the robot performed effectively. This coaching capability is significant, as it suggests that robots could be deployed in new environments and improved in real time without needing additional data collection or retraining.
Limitations and Challenges
While the results are promising, the researchers are cautious about overstating the model’s capabilities. They acknowledge that sometimes failures arise from their own shortcomings in prompt engineering rather than the robot’s limitations. For example, an early experiment with the air fryer yielded a mere 5% success rate. However, after refining the task explanation, the success rate soared to 95%.
Complex Task Execution
Currently, π0.7 is not capable of executing complex multi-step tasks autonomously from a single command. As Levine points out, users cannot simply instruct the robot to “make toast.” Instead, they must provide step-by-step guidance, such as “open this part of the toaster, push that button, and do this,” for the robot to succeed.
Measuring Performance
Another challenge lies in the lack of standardized benchmarks for robotics, which complicates external validation of their claims. Instead, the company compared π0.7’s performance against its previous specialist models—systems designed for specific tasks. The generalist model matched the performance of these specialist models across various complex tasks, including making coffee, folding laundry, and assembling boxes.
Surprising Results and Future Implications
Perhaps the most notable aspect of the research is the degree to which the results surprised the researchers themselves. Ashwin Balakrishna, a research scientist at Physical Intelligence, expressed that he is rarely surprised by the model’s performance based on his knowledge of the training data. However, he experienced genuine surprise when the robot successfully rotated a gear set he randomly provided.
Comparison to Language Models
Levine draws a parallel between their findings and the early experiences with language models like GPT-2, which generated unexpected outputs. He emphasizes that while language models had access to vast amounts of internet data, robots do not have the same breadth of information to draw from. This raises questions about the limitations of robotic learning compared to language processing.
Criticism and Future Directions
Critics may argue that the tasks demonstrated by π0.7 are relatively mundane, lacking the excitement of more complex robotic feats. However, Levine counters this perspective by highlighting that the distinction between impressive demos and systems that genuinely generalize is crucial. Generalization may appear less dramatic but is significantly more practical for real-world applications.
Conclusion
The research surrounding π0.7 presents early signs of generalization and initial demonstrations of new capabilities in robotic learning. However, the researchers are careful to frame these findings as part of an ongoing exploration rather than a finished product. When asked about the timeline for real-world deployment of systems based on these findings, Levine refrains from speculation, expressing cautious optimism about the future of robotic intelligence.
Note: The developments in robotic intelligence are still in the research phase, and further validation and refinement are necessary before practical applications can be realized.
