LangPert is a novel framework that enables robots to detect and respond to unexpected object changes during rearrangement tasks using Visual Language Models (VLMs).

• Addresses Task-Level Perturbations (TLPs) like object additions, removals, or displacements that disrupt robotic tasks
• Integrates VLMs to comprehensively monitor task execution in real-time
• Enhances robotic resilience in dynamic environments where unexpected changes occur
• Directly applicable to factory automation and engineering systems requiring adaptive object manipulation

This research bridges a critical gap in robotic manipulation systems, enabling more reliable automation in unpredictable real-world settings where objects may be unexpectedly moved or altered.

LangPert: Detecting and Handling Task-level Perturbations for Robust Object Rearrangement