Gemini Robotics-ER 1.6: Powering real-world robotics tasks through enhanced embodied reasoning
In the evolving landscape of robotics, the ability to perform tasks effectively hinges not merely on following programmed instructions but on the capacity to reason about the physical world. This concept, known as “embodied reasoning,” is pivotal for robots to navigate complex environments and interpret various inputs, such as reading a pressure gauge. Today, we introduce Gemini Robotics-ER 1.6, a substantial upgrade to our reasoning-first model that enhances robots’ understanding of their surroundings with remarkable precision.
What is Gemini Robotics-ER 1.6?
Gemini Robotics-ER 1.6 represents a significant advancement in robotic reasoning capabilities. This model is designed to improve spatial reasoning and multi-view understanding, thereby elevating the autonomy of physical agents. It specializes in critical reasoning capabilities essential for robotics, including:
- Visual and spatial understanding
- Task planning
- Success detection
Acting as the high-level reasoning model for robots, Gemini Robotics-ER 1.6 can execute tasks by utilizing tools such as Google Search for information retrieval, vision-language-action models (VLAs), or any third-party user-defined functions.
Key Improvements Over Previous Models
Gemini Robotics-ER 1.6 showcases significant enhancements compared to its predecessors, Gemini Robotics-ER 1.5 and Gemini 3.0 Flash. The upgrades focus on improving spatial and physical reasoning capabilities, which include:
- Pointing and counting accuracy
- Success detection in task execution
- New capability for instrument reading
The instrument reading feature allows robots to interpret complex gauges and sight glasses, a functionality developed in collaboration with our partner, Boston Dynamics.
Availability for Developers
Starting today, Gemini Robotics-ER 1.6 is accessible to developers through the Gemini API and Google AI Studio. To facilitate the integration of this model, we are providing a developer Colab with examples on how to configure the model and prompt it for various embodied reasoning tasks.
Benchmarking Results
Benchmark results illustrate the performance of Gemini Robotics-ER 1.6 against its predecessors. The evaluations, particularly in instrument reading, were conducted with agentic vision enabled, showcasing the model’s enhanced capabilities. The single-view and multi-view success detection evaluations were performed under different conditions, making direct comparisons challenging.
The Importance of Pointing in Spatial Reasoning
Pointing serves as a foundational capability for embodied reasoning models, evolving with each generation. It can express various concepts, including:
- Spatial reasoning: Precision in object detection and counting
- Relational logic: Making comparisons, such as identifying the smallest item in a set
- Motion reasoning: Mapping trajectories and identifying optimal grasp points
- Constraint compliance: Reasoning through complex prompts, such as “point to every object small enough to fit inside the blue cup”
Gemini Robotics-ER 1.6 utilizes pointing as an intermediate step to reason about more complex tasks. For instance, it can count items in an image or identify salient points to assist in performing mathematical operations, enhancing its metric estimations.
Demonstrating Pointing Capabilities
To illustrate the capabilities of Gemini Robotics-ER 1.6, consider a practical example where the model is tasked with identifying various tools in an image. The model successfully identifies:
- Hammers: 2
- Scissors: 1
- Paintbrushes: 1
- Pliers: 6
- A collection of garden tools (interpreted as a single group)
Importantly, it does not point to items that are absent from the image, such as a wheelbarrow or a Ryobi drill. In contrast, its predecessor, Gemini Robotics-ER 1.5, struggled to accurately identify the number of hammers or paintbrushes.
Conclusion
Gemini Robotics-ER 1.6 marks a transformative step in the realm of robotics, bringing enhanced embodied reasoning capabilities that empower robots to perform real-world tasks with greater autonomy and precision. As developers begin to integrate this model into their applications, we anticipate a new era of robotics where machines can understand and interact with their environments in increasingly sophisticated ways.
Note: The advancements in Gemini Robotics-ER 1.6 reflect ongoing research and collaboration within the field, aimed at enhancing the utility of robots in everyday tasks and complex industrial applications.
