This research introduces a novel framework for evaluating reliability in recommendation systems powered by large language models (LLMs).

• Demonstrates that LLMs exhibit significant uncertainty in their recommendations
• Introduces methods to quantify predictive uncertainty for measuring recommendation reliability
• Proposes a framework to decompose uncertainty into different sources
• Enables more transparent and trustworthy AI recommendation systems

For security teams, this research provides essential tools to assess recommendation reliability, identify potential vulnerabilities in LLM-based systems, and build more trustworthy AI applications for end users.

Uncertainty Quantification and Decomposition for LLM-based Recommendation