This survey examines how Large Language Models are revolutionizing mathematical reasoning, theorem proving, and optimization techniques across engineering applications.

• Evolution from statistical learning to transformer-based methodologies
• Advanced capabilities in mathematical problem-solving including mixed-integer programming
• Applications in engineering optimization and constraint generation
• New frameworks for formulating and solving complex computational problems

For engineering teams, this research offers valuable insights into how LLMs can enhance design optimization, systems control, and complex engineering problem-solving through more efficient mathematical reasoning capabilities.

A Survey on Mathematical Reasoning and Optimization with Large Language Models