RoboPrompt is a novel framework that allows text-only Large Language Models to directly predict robot actions through in-context learning without any additional training.

• Zero-shot capabilities: Leverages examples of robot actions within the prompt to enable prediction of new actions
• Keyframe identification: Automatically captures important moments from robot episodes to create effective prompts
• Cross-embodiment transfer: Successfully transfers knowledge between different robot types
• Real-world application: Demonstrates effectiveness in laboratory settings with physical robots

This research significantly advances robotics engineering by enabling off-the-shelf language models to control robots without specialized training, potentially accelerating the deployment of adaptable robotic systems in manufacturing and other domains.

In-Context Learning Enables Robot Action Prediction in LLMs