This research introduces contextual decomposition as a novel technique for identifying circuits within transformer models, offering faster and more accurate insights into how large language models function internally.

• Speed Improvement: Addresses the slow runtime limitations of existing circuit discovery methods
• Reduced Errors: Minimizes approximation errors compared to activation patching techniques
• Enhanced Flexibility: Works without specific metric requirements like non-zero gradients
• Security Applications: Enables better understanding of model internals to identify vulnerabilities and ensure safer AI deployment

Efficient Automated Circuit Discovery in Transformers using Contextual Decomposition