OpenAI's Low-Resource Coding Work Puts COBOL Migration to the Test

On October 19, 2026, OpenAI said it is improving ChatGPT across low-resource programming languages, with new evaluations for software engineering, SQLite, and COBOL migration ¹. If your company still runs COBOL in production, that's a reason to build a controlled test of your own—not evidence that a model can safely touch the mainframe.

The gap between those two things is the whole story. An evaluation tells you a task is being studied. The supplied excerpt does not say what was tested, how outputs were scored, or whether the results say anything about your codebase ¹. Before you expand AI-assisted work on legacy systems, you need evidence about behavior, integration, maintainability, and how much human review the output demands.

What the low-resource programming languages announcement says

OpenAI's announcement names three evaluation areas—software engineering, SQLite, and COBOL migration—as part of its work to improve ChatGPT across low-resource programming languages ¹. The supplied excerpt identifies evaluation coverage, but does not specify which languages or COBOL dialects are represented, what the migration tasks involve, what datasets or metrics were used, or how results compare to earlier model versions ¹.

Treat it as a prompt to investigate, not as grounds to approve production changes or cut back on review. The excerpt confirms that COBOL migration is an evaluation area; it does not provide organization-specific evidence for a decision ¹.

What to measure when you evaluate AI code migration

“Migration” covers several distinct jobs, and conflating them can lead an evaluation to measure the wrong thing. Code translation, behavioral preservation, integration with surrounding systems, and whether engineers can maintain the result are separate questions. The supplied excerpt does not report on these questions ¹—they are areas for an internal evaluation to define before code is generated:

• Behavior: Does the changed program produce the same outputs and enforce the same business rules on representative inputs?
• Validity: Does it parse or compile in the target environment, and does it work with the interfaces around it?
• Coverage: Do the tests hit the cases that matter, or just the easy path?
• Maintainability: Can a reviewer follow the control flow, naming, and error handling later?
• Review effort: How long does it take an expert to find defects and sign off?
• Failure severity: What's the blast radius when the output is wrong, incomplete, or missing context?

Compilation is useful as a validity check, but it's a weak proxy for correctness: code can be syntactically valid and still violate a business rule that no test covers. That's why an evaluation should include executable tests against expert-reviewed expected behavior, not just a check that the output looks like code.

The supplied excerpt does not answer these questions ¹. They are issues for your own evaluation to address.

AI SQLite coding is not a proxy for COBOL

OpenAI lists SQLite as a separate evaluation area, but the supplied excerpt gives no task definitions, datasets, metrics, or results for it ¹. That makes it a separate line of inquiry, not evidence of COBOL capability.

A SQLite test might cover schema changes, query behavior, migrations, or how an application talks to its database. Each requires its own inputs and expected outcomes. Without details about the evaluations, a result in database work would not establish how a model performs on COBOL migration, or vice versa.

“SQLite coding” could mean several different tasks, and the supplied excerpt doesn't specify which were included ¹. Without that detail, you can't map the evaluation onto your actual database workload.

The questions the announcement leaves open

The supplied excerpt confirms new evaluations in software engineering, SQLite, and COBOL migration, but does not provide the detail needed to interpret their results ¹. Before you weight this announcement in any decision, get answers to:

• Which languages and COBOL variants are represented, and how close are they to your environment?
• What are the exact tasks—for software engineering, SQLite, and COBOL migration?
• What context does the model see, and what counts as a correct output?
• Are correctness, maintainability, failure severity, and review effort scored separately, or rolled into one number?
• What baseline or prior system is the comparison against?
• How representative are the tasks of your code, integrations, and business rules?

A single aggregate score can hide failures you care about. A plausible-looking change that quietly rewrites business logic is a different problem from one that fails to parse. Ask for failure categories and how they were detected, not just a headline pass rate.

The limiting factor is portability. An evaluation set can tell a team what to test next; by itself, it cannot establish how a model performs on your codebase. That is an inference from the gap between the evaluation areas named in the excerpt and the organization-specific evidence it does not provide ¹. The same question of whether reliability on one evaluation transfers to another setting comes up well beyond code.

How to run your own COBOL migration test

Start small and representative, not broad and open-ended. Pick a bounded slice of work that exposes business logic, data handling, and interface boundaries—enough for experts to review in full. Don't open with “convert this codebase”: if it fails, you won't know whether the cause was the model, missing context, or an acceptance test nobody defined.

Fix the baseline before generating anything. Preserve the existing implementation, its tests, and an expert-reviewed statement of expected behavior for the target. Then run the evaluation and keep the record: prompts, source context, generated changes, tool actions, test results, reviewer decisions, and rework. That record lets you compare attempts and investigate failures.

If you compare several models on the same slice, keep prompts and source context identical, and decide up front which providers may see your source code. An enterprise AI gateway that applies routing constraints on provider and data policy can enforce that choice, and a model catalog shows what is available to compare.

Gate the output through checks that test different properties rather than repeating the same check:

• Parsing or compilation for basic validity
• Automated behavioral tests for expected outcomes
• Static analysis for issues that don't require running the full workflow
• Expert review for business assumptions and maintainability
• Integration review for effects on the systems around the change

These are recommendations, not a description of OpenAI's methods—the supplied excerpt says nothing about how its evaluations were run ¹.

Report more than a pass rate. Track defects by severity, instances of missing context, reviewer effort, maintainability complaints, and cases where the model should have asked a clarifying question instead of guessing. Code that's technically valid but consumes substantial senior review time may not be a win.

Making the call on AI for COBOL migration

Use the announcement as a trigger to investigate, especially if “the model doesn't handle COBOL” has been your reason for not testing AI-assisted maintenance ¹. Don't let it become the decision itself: the supplied excerpt gives no information about risk, savings, or review capacity for your organization ¹.

A reasonable starting point is bounded, reviewable work: explaining a section of legacy code, proposing tests against existing behavior, or making a limited transformation. Expand only when your own results show acceptable behavior, a review load your team can sustain, and output someone else can maintain.

Set the go/no-go line before you start, not after you see the output. Decide which failure modes are unacceptable, how much review capacity you have, and what evidence would change your mind. Then decide from your results—not from the fact that an evaluation exists.

FAQ

Does OpenAI's announcement prove ChatGPT can migrate COBOL safely? No. The supplied excerpt confirms a COBOL-migration evaluation exists, but gives no task definitions, results, correctness evidence, or review requirements ¹.

What does the announcement say about SQLite coding? Only that SQLite is one of the named evaluation areas; the supplied excerpt gives no tasks, datasets, metrics, or results ¹.

Can evaluation results predict performance on a company's legacy codebase? Not by themselves. They can help identify what to test next. Assessing performance on your own code requires representative internal tasks, tests, and expert review. That's an inference from what the supplied excerpt does not cover, not a claim it makes ¹.

What should teams measure when evaluating AI for COBOL migration? Behavioral correctness, code validity, maintainability, failure severity, and human review effort are useful measures. The supplied excerpt reports no results for these measures ¹.

When was OpenAI's low-resource programming languages announcement published? October 19, 2026 ¹.

Before your next planning cycle, pick a bounded COBOL task, such as a module or query pattern, and run it through the checklist above. Your own test results are what you can use to support a decision about production work.