36+ Hours Is a Demonstration, Not a Reliability Case [1]

OpenAI's title describes Third Moment Research running 10,000 agents for 36+ hours; its summary connects the lab's agents with cybersecurity. ¹ The supplied excerpt does not say whether the output was correct, how much human intervention it took, or what the run cost. For a CTO deciding whether to hand long-running AI agents work that spans days instead of minutes, that gap is the whole story.

Duration and agent count describe an operating pattern. They say nothing on their own about whether a workflow is dependable enough to put in front of a board.

What Third Moment Research reported about long-running AI agents

OpenAI published the account on October 14, 2026, titled 'How Third Moment Research runs 10,000 agents for 36+ hours.' ¹ Its summary names GPT-6 Astra, GPT-6.1 Sol, Codex, and Daybreak in connection with the lab's cybersecurity agents, and says they 'focus for days and improve with use.' ¹

That is a capability claim, not a measurement. The excerpt doesn't say how improvement was tracked or what tasks the agents performed. It gives an agent count, a duration, named systems, and a work domain. ¹ That's enough to make the account relevant to anyone evaluating long-running agents. It's not enough to establish output quality or business value.

What's missing from the AI agent reliability case

The supplied excerpt doesn't specify the tasks performed, success criteria, human intervention, failure recovery, run cost, or repeatability. ¹

Without those details, there's no way to know from this excerpt if the agents did useful work, how often someone had to fix what they produced, or what happened when part of the workflow broke.

Analysis: Treat this as evidence of an operating pattern, not proof the pattern is production-ready. A production case links defined work to acceptable results, controls, and cost. Runtime is one property of a system. It is not a stand-in for evidence about outcomes. That doesn't make the account worthless — it's a prompt to build your own evaluation, not a substitute for one.

Define the work before you measure it

'Cybersecurity work' isn't a test specification. Before running anything, name the task, the inputs the workflow is allowed to use, the output it must produce, and the condition under which that output counts as correct. The supplied excerpt doesn't specify which cybersecurity tasks were run or how success was judged. ¹

Separate completed tasks from attempted ones. Track whether results were correct and verifiable — not whether the agent produced something plausible. A workflow that keeps running isn't the same as a workflow that finishes the work, and heavy human correction can erase whatever economics looked good on paper.

Analysis: Use a fixed evaluation set and the same success criteria across repeated runs. Otherwise an apparent improvement might just mean the tasks got easier, the inputs changed, or review got looser — not that the agents got better. This bears directly on OpenAI's statement that agents 'improve with use': the excerpt gives no measurement method or size of improvement. ¹

For each trial, log:

• Whether the defined task was completed, and whether the result was correct
• How often a person intervened, reviewed, or approved an action
• The types of errors, retries, and escalations
• Elapsed time and total cost, including human review and recovery work

The excerpt provides none of this trial data. ¹ It's the minimum needed to compare a long-running workflow against a shorter, more conventional one doing the same job.

Agent oversight: build in supervision and recovery before you scale permissions

A workflow running for days needs an explicit control model: who can stop it, which actions need approval, what gets logged, and what it does after an interruption. For a starting list of what to build, see runtime governance controls for always-on AI agents.

OpenAI's summary says the agents 'focus for days.' It doesn't describe how they were supervised or how unsafe actions, failures, and interruptions were caught and handled. ¹ Runtime alone tells a buyer nothing about whether the system stayed inside its authority.

Analysis: Test under failure conditions before widening permissions. Interrupt the workflow on purpose. Remove a tool. Deny a permission. Feed it a wrong result that should trigger review. Watch whether it stops safely, resumes from a known state, or needs a manual restart — and measure the human effort to get it back under control.

Cap the scope of unattended action explicitly. Long duration doesn't imply broad authority. Define what the agent can do without sign-off and what must go to a person, regardless of how long it's been running. The question of where agent permission boundaries should sit is worth settling before the trial, not after.

AI agent costs: cost per outcome, not cost per run

The supplied excerpt doesn't report what the 10,000-agent, 36+ hour run cost, or how cost moved over its course. ¹ That makes cost an open variable for any team considering this, not a validated result.

A real cost model goes beyond model usage: track spend by task, retry, escalation, and elapsed time, then add human review and recovery. A workflow that looks efficient by automated runtime alone can still require substantial human effort to verify or fix its output.

Analysis: Compare the long-running design against shorter, checkpointed workflows on the same task, holding success criteria constant — otherwise you're rewarding duration, not value. Set a budget ceiling and a stop condition tied to measured progress. If more runtime stops producing better results, the workflow should halt or escalate, not keep going. One option is to enforce budget ceilings at an enterprise AI gateway layer rather than relying on each agent to police itself.

Judge the business case on cost per acceptable outcome. That's the basis for comparing an agent workflow against alternatives without mistaking a longer run for a more capable one.

What would justify production use of enterprise AI agents

Before production, answer these questions: does the workflow complete a defined task, stay within policy, remain supervisable, recover from failure, and fit the business case? Answer with run-level evidence — not a headline about duration and agent count.

The supplied excerpt doesn't say how repeatable the reported run was under comparable conditions. ¹ That matters: one successful run doesn't predict consistent performance on similar future work.

Ask for records showing interventions, failures, retries, recoveries, approvals, and final outcomes. That's what separates a system that completes work with controlled exceptions from one that simply runs a long time while its actual state stays unclear.

Analysis: Production readiness is specific to the task and the permissions in question — evidence from one workflow doesn't transfer automatically to another. The question isn't whether agents can run for days. It's whether this workflow produces acceptable results under controls your company can operate and defend to its board.

Next steps

Pick a bounded cybersecurity workflow with a measurable outcome and limited permissions. Define success, what needs human approval, and when the run stops.

Run a controlled trial: use a fixed evaluation set, deliberate interruption tests, and a documented recovery procedure. Log task success, error types, human intervention, elapsed time, retries, and total cost for every run. Repeat under matching conditions before expanding scope.

Use the Third Moment Research account as a reason to build that evaluation — not as evidence that you can skip it. ¹ Expand permissions only after the results show the workflow is reliable for the specific task, supervised at the right level, recoverable when it fails, and cheaper per acceptable outcome than what you're running today.