RECKON introduces a novel evaluation framework that directly uses reference data to assess large language models' knowledge capabilities without relying on traditional benchmarks.

• Organizes unstructured data into manageable units for targeted question generation
• Eliminates resource-intensive benchmark development while reducing information loss
• Efficiently evaluates domain-specific knowledge across medical, legal, educational, and security domains
• Provides more comprehensive assessment of what LLMs actually know

For medical applications, RECKON enables precise evaluation of biomedical knowledge in LLMs, crucial for ensuring reliability and accuracy in healthcare applications where factual correctness is vital for patient safety.

RECKON: Large-scale Reference-based Efficient Knowledge Evaluation for Large Language Model