# JEV cascade keeps 99% of GPT-6 accuracy at about 57% of the fee

A Jev-as-a-Judge paper reports that accepting TypeSafe AI’s confident JEV verdicts and escalating the rest to GPT-6 Astra kept 99% of GPT-6 accuracy on 510 held-out preference pairs, at about 57% of GPT-6’s fee. The authors say the escalation threshold did not transfer to every fallback model and should be set on one’s own data.

Language: en
Time zone: America/Los_Angeles
Current time (UTC, cached as of): 2026-11-10T06:08:00.000Z
Query date (America/Los_Angeles): 2026-11-09

HTML: https://didcodexreset.com/news/d787ece8d1514ca69442772f.html

Content language: en
Localization state: sameLanguage

Source: DAIR.AI · Published 9/24/2026, 08:48:35

A paper introducing Jev-as-a-Judge finds that most evaluations should use a cheap judge, with only uncertain calls sent to a frontier model. It tests that approach with JEV, TypeSafe AI’s decision-only judge. On 510 held-out preference pairs, a cascade that accepted JEV’s confident verdicts and escalated the rest to GPT-6 Astra kept 99% of GPT-6’s accuracy at about 57% of its fee.

JEV returns a verdict and label probabilities and does not produce reasoning text. It costs $0.044 per 1,000 judgments, with a median latency of 0.152 seconds, compared with $12.182 and 1.885 seconds for GPT-6, about 277 times cheaper.

On ordinary preference and evidence-grounded factuality, JEV stays within 3 points of GPT-6: 92.2% versus 93.5% on RewardBench, and 87.5% versus 86.7% on HaluEval. The gap grows to 9 to 20 points on tasks that require checking a derivation or rejecting an elaborately written wrong answer, including 78.6% versus 93.1% on JudgeBench. On several benchmarks, JEV’s gap to GPT-6 is concentrated in its low-confidence decisions. The escalation threshold did not transfer to every fallback model, so the authors recommend setting it on one’s own data. The paper is [Jev-as-a-Judge: accept when confident, escalate when unsure](https://academy.dair.ai/papers/jev-as-a-judge-accept-when-confident-escalate-when-unsure-2609.26550).

Tags: AI evaluation, JEV, LLM judges, TypeSafe AI, GPT-6

[View original post](https://x.com/dair_ai/status/2103147453717545278)