Retrieval-augmented generation

CRAG open model results

Comprehensive RAG benchmark for factual question answering across dynamic facts, long-tail entities, web pages, and mock knowledge-graph APIs.

Version
CRAG public test set (1,335 questions)
Task
End-to-end retrieval-augmented question answering
Primary metric
Answer accuracy (higher is better)
Source check
2026-08-13
Scores are from Table 5 of the CRAG paper. These rows isolate the open-weight Llama 3 70B Instruct reader under progressively richer retrieval settings; they are pipeline results, not model-only quality scores.
SystemModelScoreEvaluation scopeEvidence
Task 3: end-to-end web + KG RAG50 candidate web pages plus mock KG APIs; fixed context budget Llama 3 70B InstructOpen weights 40.6% CRAG public test, Task 3 Paper reported
Task 2: web + KG RAGFive candidate web pages plus mock KG APIs; fixed context budget Llama 3 70B InstructOpen weights 37.5% CRAG public test, Task 2 Paper reported
Task 1: web-only RAGFive candidate web pages; fixed 4K-token web context Llama 3 70B InstructOpen weights 35.6% CRAG public test, Task 1 Paper reported
LLM-only baselineBrief-answer prompt with abstention; no retrieved context Llama 3 70B InstructOpen weights 32.3% CRAG public test, no retrieval Paper reported

Primary sources

Official benchmark Original paper Evaluation repository