# Critical-State RL trains only the tool call that changes the outcome

DAIR.AI describes a Salesforce AI Research method that isolates the multi-turn tool call whose action changes the later reward, then updates only that call. On BFCL v4 missing-function tasks, training the selected turn adds about 14 points.

Language: en
Time zone: America/Los_Angeles
Current time (UTC, cached as of): 2026-11-17T15:23:00.000Z
Query date (America/Los_Angeles): 2026-11-17

HTML: https://didcodexreset.com/news/dc278faa185d402084875219.html

Content language: en
Localization state: sameLanguage

Source: DAIR.AI · Published 9/28/2026, 08:48:22

Critical-State RL targets multi-turn tool use by training the single call where the action changes the outcome, rather than spreading reward across the whole trajectory. When the reward depends on later turns, much of its variation comes from what happens downstream.

The method uses nested sampling to separate reward variation caused by the current action from that downstream noise, then trains only the selected call with contextual-bandit updates. On BFCL v4 missing-function tasks, training the selected turn adds about 14 points, while training the other candidate turn leaves accuracy flat or lower.

DAIR.AI points to the [Critical-State RL paper](https://academy.dair.ai/papers/critical-state-rl-diagnosing-trainable-states-for-multi-turn-tool-use-2609.24985).

Tags: Critical-State RL, Salesforce, reinforcement learning, BFCL, tool use

[View original post](https://x.com/dair_ai/status/2104597506835530112)
