Mistral AI reports Mistral Large 4 as top open-source model on Harvey's Legal Agent Benchmark
Mistral AI stated that Mistral Large 4 ranked as the number one open-source model on Harvey's Legal Agent Benchmark.
#benchmark
View all AI newsHassan releases GeoGuess Bench to evaluate AI models on GeoGuessr
GeoGuess Bench evaluates AI models on 210 worldwide geolocation photo tasks. Claude Opus 5.5 ranked first overall, while open models delivered competitive accuracy at lower inference costs.
Jiantao Jiao announces SWE-Serve, a 53-task NVIDIA benchmark
Jiantao Jiao says an NVIDIA team built SWE-Serve to test whether AI agents can develop an inference engine and serve real models. The benchmark has 53 tasks drawn from real sgl_project engineering work.
Tencent Hy introduces WebCraftBench for AI-built web apps
WebCraftBench has agents use a live app, then scores aesthetics, usability, and whether the original request was met. Tencent Hy reports an 85.3% match with human preference on 197 validated pairs.
MentalHealthBench covers everyday support through acute crises
OpenAI says MentalHealthBench is designed for the full range of mental health conversations people bring to AI, from everyday support to more acute crisis scenarios, rather than only emergencies.
OpenAI releases MentalHealthBench for realistic mental health conversations
OpenAI has openly released MentalHealthBench, a new benchmark built with input from more than 80 mental health clinicians. OpenAI says it shows frontier models continuing to improve in realistic mental health conversations.
Gemini 3.8 Flash TTS ranks first on Artificial Analysis pronunciation benchmark at 89.5%
On Artificial Analysis’s Pronunciation Robustness benchmark, Gemini 3.8 Flash TTS scores 89.5%, ahead of Gemini 3.1 Flash TTS, SpaceXAI TTS, and Gemini 3.8 Flash-Lite TTS. Leadership on individual pronunciation categories is split among several models.
Polymarket says GPT-6 Astra finished a real Toyota Corolla obstacle course
Polymarket says GPT-6 Astra was the only AI tested by DrivingBench to complete the course.
NVIDIA AI says Nemotron 3 Diarization ranks first of 12 on initial Diarization-Bench
NVIDIA AI reports a 14.72% error rate for Nemotron 3 Diarization on voicearena_ai’s initial Diarization-Bench, about 24% lower than the runner-up.
AssemblyAI Universal 3.5 Pro shows 1.93% word error rate in a 15-model test
Tarush Agarwal reported a speech-to-text benchmark of 15 models on the same audio and pipeline. On 1,000 read-aloud clips from Pipecat FLEURS, AssemblyAI Universal 3.5 Pro had a 1.93% word error rate and a 489 ms median time to first text, both the best results in that test.
Lovable now builds with GPT-6 Sol
Lovable says GPT-6 Sol scores 6 to 12% higher than GPT-5.6 Sol at every effort level on its 0-to-1 building benchmark, with larger gains in intent alignment and design fundamentals.
Qwen introduces RecreationBench for hybrid computer-use agents
ModelScope said Qwen has introduced RecreationBench, a 250-task benchmark for Hybrid Computer-Use Agents across Ubuntu, macOS, Windows, Android, and Web. Agents must explore a running app, recreate it in code, and pass programmatic tests plus VLM-based visual evaluation.
Bailey Jennings compares Jev and Luna on ONET job classification
In a quick demo, TypeSafe AI's Jev scored 80.5% exact at about 200 milliseconds and $122 per 1M, against Luna's 84.9% exact at about 2 seconds and $482 per 1M.