Ankush Gola says smithtune pulls, selects, and processes trajectories from LangSmith, trains on FireworksAI_HQ and baseten, and evaluates the results with LangSmith. The code is in the langchain-ai GitHub repository.
LangChain says LangSmith Custom Apps can create interfaces from agent data with a prompt, including views for labeling runs, comparing experiments, and reviewing traces.
Fireworks says it is a launch training partner for LangChain’s LangSmith Fine-Tuning and smithtune, describing a one-CLI path from LangSmith traces to managed SFT on Fireworks, then evaluation and deployment, without separate training infrastructure.
LangChain announced LangSmith Engine v2 at Interrupt. It identifies agent issues before production, tests proposed fixes before showing them to users, and surfaces inefficient agent work along with cost and latency trends.
LangChain announced LangSmith Engine v2 at Interrupt and says it can proactively find agent issues and resolve them before users see them.
The session covers the key elements of online evals and how teams can use Tuned Evaluators to analyze agent interactions and attach feedback to production traces.
LangSmith provides visibility into every step for supported decision models, including Jev and SemIf.
LangChain says Jev-as-a-judge is now available in LangSmith. The company claims it can score every production trace, check more criteria per trace without rising cost, and catch safety or security issues fast enough to trigger an automated response.
LangChain said it is making SemIf, an open-source decision model, free for the next week in LangSmith LLM Gateway. In the quoted post, Jacob Lee also said SemIf is being served free for the next week and is compatible with the TypeSafe SDK.
