DuplexLedger Tests Whether Voice Agents Keep Their Memory Straight While Still Talking

DuplexLedger is a speech agent memory benchmark for full-duplex voice AI. It tests whether agents update persistent memory while listening and speaking. It addresses a gap the authors identify in existing benchmarks, which separate interaction timing from memory outcomes. ¹

The bilingual benchmark includes 1,094 persistent sessions across four full-duplex scenarios, with event-level Write, Update, Ignore and Retain supervision, explicit pre- and post-memory states, and 6,564 same-session questions and checks. ¹ The authors evaluated five full-duplex models. They report that none combined strong boundary control with coherent final-state memory, and that the best final-state fact coverage was 2.01%. ¹

That is a stark headline figure, but the abstract does not provide the scoring rubric, individual model results, details of the four scenarios, the languages used, or how sessions were constructed. ¹ Those omissions make it harder to interpret or reproduce the result from the abstract alone.

Why test voice agent persistent memory inside the live exchange

The authors argue that separating interaction timing from memory outcomes obscures how an agent's memory changes as events unfold. ¹ DuplexLedger addresses this by aligning scenarios with event-level memory actions and explicit before-and-after memory states. ¹

The distinction is practical: an agent can manage interaction boundaries yet still produce stale assertions or adopt distractor information. The paper reports evaluating memory outcomes as well as interaction boundaries, rather than treating one as a substitute for the other. ¹

What to check before using this speech AI evaluation

One reading is that the 2.01% figure says more about the difficulty of the task than about any single model, but the abstract gives no per-model breakdown to test that. ¹ Teams considering this kind of speech AI evaluation will want the full paper's scoring definitions and scenario descriptions before treating the number as a procurement signal.

It may also help to compare it with how other work evaluates speech and agent memory. For speech stacks, see IBM Granite Speech 5.0: What Speech Teams Should Verify Before Switching. For text-based agents, see ALTK-Evolve Improves GPT-4.1 on τ²-Bench—Without a Matched Model Comparison. If you are shortlisting models to test against your own memory checks, the model catalog is a place to start.