This research develops a robust framework for evaluating and improving how Large Language Models translate natural language into secure Bash commands.

• Created a manually verified dataset of 500 natural language and Bash command pairs
• Developed a novel evaluation method that reliably determines functional equivalence of commands
• Demonstrated that LLM-supported translation can make complex command-line interfaces more accessible to non-experts
• Found that newer LLMs (GPT-4) significantly outperform previous models in generating correct Bash commands

Why it matters: Command-line interfaces represent a significant security risk when used incorrectly. This research helps bridge the gap between natural language and secure command execution, reducing potential vulnerabilities while making powerful CLI tools accessible to more users.

LLM-Supported Natural Language to Bash Translation