Evaluating AI models and agents: the 2026 field guide

The unit of measurement in AI broke faster than the models did.

OpenAI formally retired SWE-bench Verified, its own headline coding number, on February 23, 2026, publishing a post titled "Why SWE-bench Verified no longer measures frontier coding capabilities." Six weeks earlier, UC Berkeley's RDI lab showed that a ten-line conftest.py patch can score 100% on eight major agent benchmarks, SWE-bench variants included, without the agent solving a single task. Three LLM-observability vendors changed hands in 90 days.

The "best model" leaderboard, the lingua franca of 2024, is being quietly disowned by the labs that built it.

That makes model evaluation the highest-leverage skill on an AI team right now. The interesting question is no longer "which model is best." It is "what does your evaluation actually measure, and would it survive an audit?"

Model evaluation in 2026 is task-grounded, trace-aware, graded by a calibrated panel of judges, and run on CI before users see drift. Public benchmarks are inputs to your thinking, not the output of it.

TL;DR

Static public benchmarks have lost authority because they are contaminated, mis-graded, and gameable. The working alternative is an in-house eval program built on real production tasks, executable environments, a layered grader stack, and a CI pipeline that blocks merges on regression.

This guide maps the instruments, the failure modes, and a concrete harness you can build this quarter.

Key takeaways

• SWE-bench Verified was deprecated by OpenAI in February 2026 after an audit found ≥59.4% of problems had flawed tests. Treat any 2026 "X% on Verified" claim as the soft target.
• A 10-line grader exploit scored 100% on eight benchmarks. Test-based static grading is a security surface, not a measurement.
• Reliability is now a first-class metric: report pass^k, not just pass@1. A 50%-accurate agent often succeeds on all 8 of 8 calls less than a quarter of the time.
• For RAG, faithfulness is the metric to ship first. RAGAS is the default library; expect to build citation and refusal grading in-house.
• LLM judges are noisy, biased estimators. Use cross-family panels, rubrics, and a frozen human-labeled calibration set, or do not use them for defensible decisions.
• The eval harness is the product. Version everything, run it on every change, and treat it as your EU AI Act audit trail from August 2, 2026.

Why do AI benchmarks mislead?

Static benchmarks fail for three reinforcing reasons: the test set leaks into training, the grader is wrong, and the published number becomes a target. By mid-2026 each failure mode is quantified, not just suspected.

Contamination: the test is in the training set

MMLU was a useful instrument for about eighteen months. Edinburgh's MMLU-Redux re-annotation (Gema et al., 2024) found a 6.5% overall label-error rate across 5,700 questions, rising to 57% in the Virology subset.

The cleaner signal comes from a closed counterfactual. Microsoft Research's MMLU-CF, a never-released clean test set, drops GPT-4o's reported MMLU accuracy by 14.6 percentage points (to 73.4% 5-shot).

Take the model off the leaked distribution and 14.6 points vanish. That figure comes via a secondary survey of LLM-as-judge work, so treat it as a strong directional number rather than a vendor-reproduced one.

Scale AI's GSM1k, a held-out mirror of GSM8K, found drops up to 13 points, with Phi and Mistral families overfitting across nearly every size. LMSYS shipped the llm-decontaminator precisely because n-gram and embedding overlap is pervasive enough to need a reusable filter.

Even the contamination-proof successor leaked credibility. Humanity's Last Exam launched in January 2025 as a 2,500-question frontier benchmark. By July, FutureHouse reported that about 29% of HLE's text-only chemistry and biology answers had directly conflicting peer-reviewed evidence, and the HLE team conceded roughly 18% of items in a revised "Bio/Chem Gold" subset.

One honest counterpoint: Bordt et al. At ICML 2025 argue that small-scale contamination can be partially "forgotten" beyond 5x Chinchilla scale. It is a minority but rigorous position. Decontaminate anyway, and know the target moves.

Verifier error: the grader is the security surface

A benchmark is only as good as its grading function, and grading functions are exploitable.

The Berkeley RDI April 2026 audit is the watershed. A 10-line pytest conftest.py patch that monkey-patches the test runner scores 500/500 on SWE-bench Verified and 731/731 on SWE-bench Pro's public split, with no LLM in the loop.

The same paper logs an in-container parser.py overwrite on Terminal-Bench, a curl trojan that replaces the test binary before grading, and returning the literal string "{}" to clear 890/890 FieldWorkArena tasks.

The lesson is structural. Test-based grading is intrinsically a security surface, and the cost of finding exploits falls as coding models improve.

Verifier error also shows up without adversaries. OpenAI's own February 2026 audit of SWE-bench Verified found that ≥59.4% of audited problems had flawed test cases capable of rejecting correct patches, 35.5% over-specified the implementation, and 18.8% tested behavior the issue never required.

When the verifier error rate rivals the reported accuracy, a 65% headline is closer to a coin flip than a measurement.

Gaming: Goodhart on every public slide

A published leaderboard is a target, and targets saturate. HLE climbed from o1's 8.8% in January 2025 to Grok 4's 50.7% (with tools) by July, per the time-horizon and HLE research literature. GPQA Diamond is on the same arc.

Stanford's HELM tried to blunt this with multi-metric, multi-scenario reporting instead of a single rank. The durable fix is dynamic, executable evaluation: live graders, private held-out sets, fresh problems.

SWE-bench Pro's commercial split, the AISI's Inspect framework, and RDI's forthcoming "BenchJack" scanner all push in that direction. None is complete; together they raise the floor.

Is SWE-bench Pro the replacement for SWE-bench Verified?

SWE-bench Pro is the least-bad public coding benchmark in 2026, and OpenAI recommends it, but it is not a clean one. The family's timeline is the clearest case study in benchmark deprecation the field has.

• August 13, 2024: OpenAI launches SWE-bench Verified, a 500-task human-validated subset with three expert reviewers per task.
• September 2025: Scale AI releases SWE-bench Pro (arXiv:2509.16941): 1,865 problems across 41 repos and four languages, averaging 107.4 lines and 4.1 files per fix, split into a 731-task public subset, a 12-repo held-out split, and an 18-repo commercial split behind a paid license.
• February 23, 2026: OpenAI deprecates Verified and recommends Pro.
• April 12, 2026: Berkeley RDI publishes the 100%-via-exploit paper.

At launch, Pro Pass@1 scores landed under 25% for every frontier model (GPT-5 23.3%, Claude Opus 4.1 22.7%, Gemini 2.5 Pro 13.5%), a deliberate departure from saturated Verified numbers.

The catch is harness divergence. On Scale's standardized SEAL leaderboard (public 731-task split, June 2026), GPT-5.4 (xHigh) leads at 59.1%, with Opus 4.6 (thinking) at 51.9% and Muse Spark at 55.0%. Vendor self-reported numbers on the same subset run 20 to 30 points higher.

Trust the standardized column. The self-reports use different harnesses, prompts, and infrastructure, so they are not comparable, and the public subset is structurally leak-prone. The harder signal is the commercial 18-repo private split, where Opus 4.6 scores 47.1% and GPT-5 14.9%. It is not independently re-runnable, which is the honest limitation.

A team that reports "X% on SWE-bench Verified" in 2026 has chosen the soft target. That is a strategic choice, not a measurement one. For a second live coding signal, the Terminal-Bench 2.0 leaderboard grades CLI and shell work under an agentic lens, currently in the 40 to 60% band.

Pick the subset that resembles your production environment and say which one you ran.

How is agent evaluation different from single-shot evaluation?

Agent evaluation grades a process, not an output. Early benchmarks treated an agent as a single-turn function. The 2025 to 2026 generation treats it as a trajectory: multi-step, stateful, costly, and prone to fail mid-run for reasons unrelated to the task.

The benchmark generation that matters

τ-bench (Yao et al., ICLR 2025) is the canonical reference. It grades against the final database state after a multi-turn retail or airline conversation, and it introduced pass^k. The headline: GPT-4o's retail pass@1 of ~50% collapses below pass^8 of 25%.

τ²-bench (Barres et al., Sierra, June 2025) is the dual-control successor, a Dec-POMDP telecom scenario where two agents coordinate against an evolving user. Mid-2026 standardized scores reach 0.993 for Claude Opus 4.6 on Telecom.

GAIA (466 real-world reasoning tasks) was authoritative for two years and is now saturated; RDI's audit clears about 98% via a public answer-key exploit. OSWorld is the computer-use benchmark, a real VM with real apps and 369 state-inspected tasks.

Its progression is the cleanest capability signal of the era: GPT-4V ~7.8% to o1 38.1% to Claude 3.5 Sonnet 61.4% in December 2024, with frontier systems now in the 70 to 80% band. RDI also found a 73% exploit rate on OSWorld via grader manipulation, so even environment benchmarks need sandboxing.

Use this saturation map to pick instruments:

Reliability: hitting versus working

The most important conceptual move of 2026 is treating reliability as a metric. pass@1 is one attempt. pass@k is at least one of k succeeding, which flatters a model. pass^k is all of k succeeding, which is what a production system needs.

When you read a vendor score, ask which one it is. A pass@k number suits an offline coding agent with a retry budget. It is irrelevant for a real-time customer-facing agent. A defensible eval reports pass@1 for capability and pass^k for reliability, at a k that matches your production retry budget.

Cost per successful task

Leaderboards now report cost per successful task, not just accuracy. RDI's Pareto curves show the most accurate model is rarely the most cost-effective, often by 5 to 20x. METR's time-horizon research frames the complement: the task length a model completes with 50% reliability is doubling every 4 to 7 months.

So a production decision needs three numbers, not one: success rate, time-horizon, and cost per successful task. The question shifted from "which model is best at coding" to "which model gives me the most successful tasks per dollar at the reliability I need."

How do you evaluate RAG and answer quality?

RAG evaluation is narrower and better quantified than general agent eval. The field has converged on a small metric catalogue; the open work is choosing a framework and filling the gaps it leaves.

The metric catalogue

If you ship one RAG metric, ship faithfulness. It is the dominant failure mode and the metric most correlated with user trust.

The framework landscape

RAGAS (EACL 2024, Apache-2.0) is the de facto standard, around 14.3k GitHub stars, metric-focused with no orchestration. TruLens is now Snowflake first-party, OpenTelemetry-native, and its "RAG Triad" of context relevance, groundedness, and answer relevance is its defining abstraction. DeepEval 4.0 brings 50+ metrics and pytest-style tests for LLM outputs.

ARES is a Stanford project (often misattributed to NVIDIA) and the only RAG framework that ships statistical confidence intervals natively, via Prediction-Powered Inference. Patronus AI's open-source Lynx 70B (arXiv:2407.08488) hits 87.4% on HaluBench, beating GPT-4o and Claude-3-Sonnet on hallucination detection.

The selection rule: RAGAS for breadth, TruLens for tracing and OTel alignment, DeepEval for pytest CI, ARES for confidence intervals, Patronus Lynx for the hardest faithfulness edge cases. Most teams run RAGAS as the primary library and pull Lynx for the difficult cases.

The metric gap

Two metrics practitioners ask for are not first-class anywhere: citation precision/recall and true refusal correctness. The Refusal Index (arXiv:2510.01782, May 2026) is a research proposal, not a built-in. Plan to write 10 to 20% of your RAG eval logic in-house until the frameworks absorb these.

What are the pitfalls of LLM-as-judge?

LLM-as-judge is the most used and most misused technique in modern eval. The biggest mistake is treating the judge as a measurement instrument. It is a noisy, biased estimator with a known error structure.

Four bias mechanisms

Position bias: the judge favors answer A or B. The MT-Bench paper (Zheng et al., 2023) put swap-inconsistency at 15 to 25%. Mitigation: score each pair twice with positions swapped, then average.

Length bias: longer answers win, which is why "be more thorough" quietly degrades judge agreement.

Self-preference bias: a judge prefers its own family's outputs. This is the strongest argument for cross-family judging.

Sycophancy bias: a judge prefers hedged, agreeable answers even when the sharp answer is correct. Anthropic's "Towards Understanding Sycophancy" (2023) is the reference. Mitigation: prompt the judge to take a position and penalize hedging.

Judge capability ceiling

A judge reliably grades only at or below its own capability. Evaluate a frontier model on a frontier task and you need a frontier judge, whose self-evaluation is tautological.

Use the strongest available model, then validate it against 50 to 200 human-labeled examples. Frontier judges now reach 85 to 90% human agreement on well-rubric'd tasks, above the 80% MT-Bench floor.

The mitigation playbook

1. Cross-family judges. Judge Claude with GPT, GPT with Claude, never a model on its own family's output.
2. Position-swap and average. Run pairwise judgments twice with positions flipped.
3. Reference-based judging. Hand the judge the ideal answer; this roughly halves variance.
4. Rubric-anchored scoring. Name what each level means. Rubric judges run 10 to 20 points more reliable than open-ended ones.
5. Structured outputs. Force JSON with score, reasoning, evidence_spans, and reject on schema failure.
6. Panel of 3 to 5 diverse judges. Take median or majority; track per-judge agreement.
7. Frozen calibration set. Keep 100 to 200 human-scored examples and re-run weekly to catch drift.
8. Agreement metrics. Report Cohen's κ, Krippendorff's α (≥0.67 acceptable, ≥0.80 publishable), and Spearman on continuous scores.
9. Statistical significance. For A/B capability claims require n ≥ 100 tasks, n ≥ 200 for judge-based scores, and a paired test.
10. Human review on the borderlines. Route top and bottom deciles plus 10 to 20% of borderline cases to humans.

LLM-as-judge is a force multiplier on a small human-labeled calibration set. If you cannot maintain that set with quarterly refresh, do not use judges for decisions that must be defensible.

Which LLM observability stack should you pick?

Observability is the third leg of the eval stool, and it turns offline metrics into a production feedback loop. The 2026 stack has consolidated around OpenTelemetry as the wire format, with a fast-moving M&A landscape on top.

Three of the most-used brands changed hands in 90 days. ClickHouse acquired Langfuse on January 16, 2026. Mintlify acquired Helicone on March 3. Cisco announced intent to acquire Galileo on April 9, folding it into Splunk. The columnar storage layer is the moat; the LLM-specific features are commoditizing.

The OpenTelemetry GenAI semantic conventions are the emerging standard, covering LLM, embedding, tool, and the new agent and MCP span kinds. As of June 2026 they are still labelled "Development," so attribute names may shift.

The pragmatic move: pick an OTel-native platform like Langfuse or Phoenix, and let the semconv be your API contract. Not every acquisition will survive; the wire format will.

What is task-grounded, trace-based evaluation?

A task-grounded eval sources its tasks from real production traffic, runs them in a real or realistic executable, grades the state of the world rather than output text, records a replayable trajectory, and logs cost per task. A static MMLU question is none of these.

An OSWorld task scored by inspecting the final desktop is all of them.

Five concurrent shifts produced this consensus.

Process supervision replaces outcome-only grading. OpenAI's "Let's Verify Step by Step" (2023) showed process reward models adding 10 to 20 points on math reasoning at equal compute. For agents, this is trajectory grading.

On-policy evaluation replaces fixed held-out sets, sampling from the live distribution so the benchmark moves with the model.

Trajectory evaluation scores final state, intermediate state, tool-call correctness, error recovery, and cost. Arize's agent-trajectory evaluators and Inspect's Task/Solver primitives are the best-documented implementations.

Environment-based eval replaces static test files. Making the environment the test is the structural answer to grader exploits.

Time-horizon and economic eval replaces the single accuracy number, per METR and RDI's Pareto curves.

Two vendor signals point the same way. OpenAI's GDPval is a 220-task benchmark of real knowledge work, 14 days of expert time per task, graded by expert comparison. Anthropic's Project Glasswing ships an agent-eval framework and threat model together. The direction is longer, expert-graded, and task-grounded.

The UK AI Safety Institute's Inspect framework is the most rigorous open-source option: a 5-component task definition, 200+ pre-built evals, sandboxing via Docker, K8s, Modal, or Proxmox, and MCP support. AISI mandates it for frontier pre-deployment eval.

Use Inspect as the default for any agent that touches real tools or the real internet, and treat its pre-built evals as a baseline.

How do you build your own eval harness?

The methodology below is opinionated. The goal is a harness that survives contact with production.

Start with the golden set

The golden set is the most important artifact, and production guidance from Hamel Husain, TinkerLLM, and the Phoenix docs converges on the same shape.

• Size: 30 to 50 for smoke, 100 to 200 for capability, 500+ for regression.
• Source: real production traffic. Synthetic-only sets saturate within a quarter.
• Segment: by intent, difficulty, customer segment, and data source. One overall number hides what matters.
• Refresh: monthly minimum, weekly if traffic moves fast. Version it like code.
• Difficulty: if it passes 100%, it is too weak. Aim for 60 to 85% on your best model.
• Criteria drift: per Husain, grade 20 to 30 outputs loosely before writing the rubric. The failure modes you see should drive the rubric; you cannot enumerate them from first principles.

Layer the graders

Production eval needs a three-tier stack, not one judge.

• Tier 1, code-based: schema validation, regex, exact match, tool-call assertions. Deterministic, cheap, the only graders that run on every commit.
• Tier 2, LLM-as-judge: the Section above's playbook for faithfulness, relevance, and quality signals.
• Tier 3, human: a 100 to 200 example calibration set for judge calibration, deciles, and "are we still measuring the right thing."

If you can write a code-based grader, write it. Reserve the expensive graders for the harder signals.

Run it as Eval-Driven Development

Eval-Driven Development is the AI analogue of TDD: the eval is the spec, and the build fails when the spec is violated. The five gates:

1. Pre-commit smoke (≤30s): 5 to 10 examples, code graders only.
2. PR gate (≤5min): 50 to 100 examples, code plus judge, blocks merge on regression, n ≥ 100 for capability claims.
3. Nightly full regression (≤2hr): full golden set, all graders, contamination check.
4. Pre-deploy shadow (≤1hr): new model on 500 to 1,000 production-shaped examples; block if cost-of-success rises >2x or accuracy drops >2 points.
5. Production A/B (continuous): 1 to 10% of live traffic, randomized, graded online.

The rule from the EDD canon: if your evals do not run on every change, they do not exist. A notebook reviewed quarterly is not an eval program. A CI gate that blocks merge is.

Close the loop with drift detection

Define each scorer once in a shared registry and call it from both offline (golden set) and online (live traces), as in Langfuse's core concepts. Drift becomes a differential question: did the online distribution shift versus the offline baseline?

Instrument five drift signals: input statistics (token length, intent mix), embedding distribution (Wasserstein or Mahalanobis distance), output quality (golden-set graders on a live sample), per-step agent behavior (step counts, tool-call mix, recovery frequency), and downstream KPIs (thumbs-up, retention, ticket rate). Alert on change-points with CUSUM or Bayesian detection, because the noise floor on any single online signal is high.

# One scorer, two call sites. Offline experiment and online stream share logic.
def faithfulness_scorer(answer: str, context: list[str]) -> float:
    """Tier-2 judge: fraction of answer claims supported by context."""
    claims = extract_claims(answer)
    supported = [c for c in claims if judge_supported(c, context)]
    return len(supported) / max(len(claims), 1)

offline_results = run_experiment(golden_set, scorers=[faithfulness_scorer])
online_stream.attach(faithfulness_scorer, sample_rate=0.05)  # same function

Govern it like a product

Version everything: dataset, rubric, judge prompts, judge model snapshot, harness code, target model snapshot. Pin all hashes. Run a quarterly harness audit. Pair every deploy with a documented rollback path.

This is also compliance infrastructure. The EU AI Act's high-risk audit-trail provisions apply from August 2, 2026. For any system touching employment, credit, education, or law enforcement in the EU, the eval harness is the audit trail.

What this means for you

Stop optimizing for a public leaderboard number. It is the most gameable surface in your stack.

Build the golden set first, from real traffic, and version it. Everything else is downstream of having a representative test set you trust.

Pick an OTel-native observability stack and treat the wire format, not the vendor, as the durable bet. The acquisitions will keep coming.

Report three numbers for any agent going to production: success rate, pass^k reliability at your retry budget, and cost per successful task. One accuracy figure is a marketing artifact.

If you must use LLM judges for a defensible decision, you need a human calibration set. No exceptions.

The differentiator in mid-2026 is not model capability. It is the quality of the program that decides which model ships. A model that is well-evaluated at 60% accuracy is worth more in production than one that is poorly evaluated at 90%.

What the field is still arguing about

A few debates remain live, and confident consensus on any of them is a signal to read further.

Does contamination matter at scale? Bordt et al. Say small-scale leakage can be forgotten; practitioners say it matters at the unit level. Decontaminate regardless.

Is pass@1 still the right headline? It is universally reported and a poor proxy for reliability. Report both pass@1 and pass^k.

Does the RDI exploit generalize? The specific conftest.py and curl tricks target weakly-sandboxed test graders. The structural lesson holds, but the exploits may not port to OSWorld or Inspect scenarios in the same form. Read it as "static test-based grading is a dead end," not "all eval is broken."

Is the static-benchmark era ending or bifurcating? Public benchmarks are losing authority with practitioners while private and regulator-mandated ones rise. The trustworthy signal now lives in private, agentic, environment-based eval, not the public leaderboard.

Sources

• Why SWE-bench Verified no longer measures frontier coding capabilities, OpenAI
• Introducing SWE-bench Verified, OpenAI
• How We Broke Top AI Agent Benchmarks, UC Berkeley RDI
• SWE-bench Pro paper, Scale AI
• SWE-bench Pro standardized leaderboard, Scale SEAL
• Terminal-Bench 2.0 leaderboard
• τ-bench (Yao et al., arXiv:2406.12045)
• τ²-bench (Barres et al., arXiv:2506.07982)
• Are We Done with MMLU? (arXiv:2406.04127)
• A Survey on LLM-as-a-Judge (arXiv:2411.15594)
• llm-decontaminator, LMSYS
• HLE chemistry/biology answer audit, FutureHouse
• Measuring AI Ability to Complete Long Software Tasks (arXiv:2503.14499)
• How Much Can We Forget about Data Contamination? (ICML 2025)
• Judging LLM-as-a-Judge with MT-Bench (Zheng et al., 2023)
• Towards Understanding Sycophancy in Language Models, Anthropic
• RAGAS (arXiv:2309.15217)
• TruLens
• DeepEval, Confident AI
• ARES (NAACL 2024)
• Inspect, UK AISI
• OpenTelemetry GenAI semantic conventions
• Agent and framework spans, OpenTelemetry
• ClickHouse acquires Langfuse
• Cisco buys Galileo
• Helicone pricing and alternatives
• Agent Trajectory Evaluations, Arize
• GDPval, OpenAI
• Project Glasswing, Anthropic
• METR time-horizon research
• Eval-Driven Development
• Langfuse evaluation core concepts
• Your AI Product Needs Evals, Hamel Husain
• EU AI Act regulatory framework