LLMSched introduces a novel uncertainty-aware scheduling framework for compound Large Language Model applications that collaborate with external modules.

• Uses Directed Acyclic Graphs (DAGs) to model complex application workflows
• Employs Bayesian networks to handle duration and structural uncertainties
• Achieves up to 31% improvement in application completion time
• Maintains performance even with high variability in execution paths

This research significantly advances engineering capabilities for LLM service providers by tackling the unique scheduling challenges posed by modern AI applications with unpredictable execution patterns and variable resource needs.

LLMSched: Uncertainty-Aware Workload Scheduling for Compound LLM Applications