# Best Taste-Bench model chooses the better direction 59.7% of the time

A paper from Microsoft and colleagues reports that the best model answers 59.7% of Taste-Bench questions correctly when choosing the better open direction in long engineering and research tasks. Forks decided by later evidence are much harder, a larger reasoning budget does not raise accuracy, and distilling a teacher’s outcome judgment improves held-out SWE-bench Pro success.

Language: en
Time zone: America/Los_Angeles
Current time (UTC, cached as of): 2026-11-10T07:30:00.000Z
Query date (America/Los_Angeles): 2026-11-09

HTML: https://didcodexreset.com/news/7ecd25ff37594380818276ef.html

Content language: en
Localization state: sameLanguage

Source: elvis · Published 9/25/2026, 07:35:23

A paper from Microsoft and colleagues measures how often frontier agents choose the better direction at a decision point in a long task. On Taste-Bench, each question is a point in a trajectory where several directions are open and one leads to a better outcome. The forks are mined automatically from parallel attempts and detours in engineering and research runs. The best model answers 59.7% correctly.

Forks whose deciding evidence appears later in the trajectory are much harder, and a larger reasoning budget does not raise accuracy. The authors distill a teacher’s judgment of the outcome into a student model, which improves end-to-end success on held-out SWE-bench Pro tasks.

The paper is available on [arXiv](https://arxiv.org/abs/2609.25804), with a companion [Chat with Paper](https://academy.dair.ai/papers/the-tasteful-agent-measuring-and-improving-taste-in-long-horizon-tasks-2609.25804) page.

Tags: Taste-Bench, AI agents, Benchmarks, Microsoft, SWE-bench

[View original post](https://x.com/omarsar0/status/2103478384584278502)