Did Codex Reset
GitHub

Jiantao Jiao announces SWE-Serve, a 53-task NVIDIA benchmark

Jiantao Jiao

Jiantao Jiao says an NVIDIA team built SWE-Serve to find out whether AI agents can develop an inference engine and make it serve real models. The benchmark consists of 53 tasks taken from real sgl_project engineering work.

#swe-serve#nvidia#benchmark#ai-agents#inference

Best Taste-Bench model chooses the better direction 59.7% of the time

A paper from Microsoft and colleagues reports that the best model answers 59.7% of Taste-Bench questions correctly when choosing the better open direction in long engineering and research tasks. Forks decided by later evidence are much harder, a larger reasoning budget does not raise accuracy, and distilling a teacher’s outcome judgment improves held-out SWE-bench Pro success.