Did Codex Reset
GitHub

Critical-State RL trains only the tool call that changes the outcome

DAIR.AI

Critical-State RL targets multi-turn tool use by training the single call where the action changes the outcome, rather than spreading reward across the whole trajectory. When the reward depends on later turns, much of its variation comes from what happens downstream.

The method uses nested sampling to separate reward variation caused by the current action from that downstream noise, then trains only the selected call with contextual-bandit updates. On BFCL v4 missing-function tasks, training the selected turn adds about 14 points, while training the other candidate turn leaves accuracy flat or lower.

DAIR.AI points to the Critical-State RL paper.

#critical-state-rl#salesforce#reinforcement-learning#bfcl#tool-use

Only 35.8% of the cleanest public terminal-agent RL collection passed a Salesforce AI Research audit

Salesforce AI Research put that collection at 35.8% clean and two others at 10.1% and 3.3%, after finding reward errors in both directions. The authors say RIVER, which filters defective environments, increases RL gains by 106% on Terminal-Bench-Lite and 30% on Terminal-Bench v2.1 while using fewer than 30% of TMax's environments.