RAG & source attribution: find it, and prove it.
At the RAG challenge of EvalLLM 2026, Kairntech ranked 1st in raw performance on source attribution and beat the participants’ median on every metric — on a closed corpus of nearly 374 documents from the defence and drones domain.
Answer correctly. And cite the exact source.
RAG (Retrieval‑Augmented Generation) lets an AI answer a complex question by drawing on thousands of pages. The real challenge isn’t just answering well: it’s citing precisely which document and page each piece of information comes from — so the answer is verifiable and trustworthy.
This edition ran on a corpus from the defence and drones domain: monitoring bulletins, parliamentary reports, technical sheets, theses, doctrine and open-source documents.
Indexing page by page, hybrid search BGE‑M3 (dense + lexical fused via RRF), query decomposition and orchestration n8n.
One foundation, two challenges.
Retrieval & generation
From a question, retrieve the relevant pages of the corpus then write a complete, faithful answer. Composite questions are broken into sub-queries to better cover each facet.
Source attribution
Link each sentence of an answer to its source pages — and flag those that have none. We compare a frugal lexical method (no AI) with an LLM-based approach.
From question to sourced answer.
Question decomposition
A composite question is split into 2–4 targeted sub-queries, one per sub-topic.
Hybrid search
Each sub-query hits the BGE‑M3 index (semantic + lexical), fused via RRF.
Factual generation
The answer is produced from the retrieved pages, in the question’s language, with no invention.
Source attribution
Each sentence is linked to its source page(s) — or flagged as unsourced.
Above the median, across the board.
In retrieval & generation, our three runs beat the participants’ median on every metric — the widest gap being on answer quality as judged by an LLM jury.
In source attribution, our best run also beats the median, with the LLM-based approach proving most precise.
| Task 1 — metric | Kairntech | Median |
|---|---|---|
| Answer quality (LLM jury) | 0.84 | 0.59 |
| Recall @10 | 0.60 | 0.51 |
| NDCG @10 | 0.52 | 0.44 |
| Task 2 — attribution | Kairntech | Median |
|---|---|---|
| F1 (best run, LLM) | 0.69 | 0.60 |
Preliminary official results provided by the organizers. Source attribution: 1st in raw performance (out of 20 systems); the final ranking then applies frugality criteria.
High-performing — and frugal
For attribution, we first measured what a lexical, AI-free method achieves: it produces attribution with zero LLM inference, turning to the model only where it brings a real gain. Measuring cost alongside quality is part of our approach: a setup can be high-performing without being frugal. Frugality (open-source, CO₂ footprint, compute resources) is a criterion we take seriously.
Get our RAG & attribution resources
Our poster and detailed article from the RAG challenge of EvalLLM 2026: defence corpus, method, results and carbon footprint. Leave your email — we'll send you everything. (Documents currently available in French only.)
The same rigour, working on your documents.
Sourced answers, no hallucination, full sovereignty — traceability at the heart of your RAG.