Retrieval-augmented generation

CRAG open model results

Name: CRAG open model results
Creator: LocalLLM Compare

Comprehensive RAG benchmark for factual question answering across dynamic facts, long-tail entities, web pages, and mock knowledge-graph APIs.

Version: CRAG public test set (1,335 questions)
Task: End-to-end retrieval-augmented question answering
Primary metric: Answer accuracy (higher is better)
Source check: 2026-08-13

Scores are from Table 5 of the CRAG paper. These rows isolate the open-weight Llama 3 70B Instruct reader under progressively richer retrieval settings; they are pipeline results, not model-only quality scores.

System	Model	Score	Evaluation scope	Evidence
Task 3: end-to-end web + KG RAG50 candidate web pages plus mock KG APIs; fixed context budget	Llama 3 70B InstructOpen weights	40.6%	CRAG public test, Task 3	Paper reported
Task 2: web + KG RAGFive candidate web pages plus mock KG APIs; fixed context budget	Llama 3 70B InstructOpen weights	37.5%	CRAG public test, Task 2	Paper reported
Task 1: web-only RAGFive candidate web pages; fixed 4K-token web context	Llama 3 70B InstructOpen weights	35.6%	CRAG public test, Task 1	Paper reported
LLM-only baselineBrief-answer prompt with abstention; no retrieved context	Llama 3 70B InstructOpen weights	32.3%	CRAG public test, no retrieval	Paper reported

Primary sources

Official benchmark Original paper Evaluation repository