Tagged “evaluation”
Adoption
Imago's Warranty-Policy App Puts Knowledge Graphs to a Practical Test
Imago says graph-based work got its warranty-support model answering correctly, but the published account has no accuracy figures, schema or validation method. Here is what to ask for before you copy the approach.
Agents
Anthropic's Dynamic Workflows: A Practical Test for Claude Agent Coordination
Anthropic's dynamic workflows let Claude plan and coordinate its own agents. The source shows a demo, not results, so here is what to test before you adopt it.
Research
Context language models put models in charge of their own context
A Meta, MIT and University of Washington approach lets models edit their own context. The reported gains come without figures, so here is what to ask and measure before changing a pipeline.
Agents
When Agents Run Unattended: Testing for Retries That Cost Money and Outputs That Vary
Pilots hide manual recovery. Before an agentic workflow runs unattended, test the retry caps, per-unit costs and acceptance rules that a human was quietly covering.
Research
Sakana AI's 73% core-claim result is a benchmark, not a deployment decision
Sakana AI's Multi-Layered Review caught most inserted core-claim contradictions in research papers, but the benchmark says little about approving company documents. Here is what it shows and how to pilot it.
Industry
Standard Bots' AI learning loop: demonstrations, corrections, and the evidence factories need
A Latent Space profile describes how Standard Bots adapts models with demonstrations and human corrections, but reports no measured reliability gains. Here is what it shows and what buyers should ask for.