This research introduces the ASIMOV Benchmark for evaluating and improving semantic safety of foundation models in robotics applications.

• Creates 500+ robot constitutions - clear guidelines for safe robot behavior
• Establishes standardized benchmarks for testing model adherence to safety guidelines
• Evaluates existing models against various safety constraints in physical environments
• Provides framework to prevent hallucinations and unsafe robot behaviors

As LLMs increasingly control physical robots, this work addresses critical security concerns by establishing rigorous safety evaluation methods that help prevent dangerous robot behaviors and ensure responsible AI deployment.

Generating Robot Constitutions & Benchmarks for Semantic Safety