LLMON in Granite 4.1: A New Markup Format, and a Benchmark Claim IBM Hasn't Backed With Numbers

IBM announced two experimental preview models, granite-4.1-3b-LLMON-preview and granite-4.1-8b-LLMON-preview. They are derived from Granite 4.1 base models and trained to consume and generate LLMON-marked text. IBM says the format nearly eliminates indirect prompt injections on some benchmarks, but the announcement does not name the test sets or give sample sizes or baseline configurations. That makes the claim difficult to evaluate against the risk it is meant to address. ¹

What changed in Granite 4.1 3B and 8B

The two preview models are derived from Granite 4.1 base models and trained specifically to consume and generate LLMON-marked text. ¹

LLMON (LLM Object Notation) is an LLM-native markup language. It uses meaningful tags wrapped in reserved special-token delimiters to communicate structure and semantics. Its schema supports nesting, and unique IDs let an instruction be linked to the data payload it should act on. ¹

That's the core idea: relationships that might otherwise be implicit in prompt ordering or convention can be represented explicitly. This could help when a model needs to act on specific content without treating nearby text as a command. But making intent explicit is not the same as demonstrating better or safer production outcomes; the announcement does not provide data establishing that. ¹

LLMON vs. chat templates vs. JSON

LLMON doesn't replace a chat template. A chat template labels roles such as system, user, and assistant. LLMON adds information about what each component means and how components relate, including across roles. ¹

IBM's case against JSON is about how models handle its structure. The announcement says JSON was designed for parsers, and that its delimiter characters can also appear as ordinary text in data. It also says transformers can be weak at tracking bracket nesting, particularly over long spans. LLMON instead uses reserved special tokens as structural delimiters and states nested structures locally. ¹

Format	Primary purpose	Relationship modeling	Open question for teams
Chat templates	Label roles (system, user, assistant) 1	LLMON adds component meaning and relationships within and across roles 1	Does LLMON improve outcomes on your actual prompts?
JSON	Structured data; IBM says it was designed for parsers 1	Expressed through nested objects and syntax 1	How reliably do these models handle your schemas and nested outputs?
LLMON	Mark structure and semantics with tags and reserved delimiters 1	Explicit instruction-to-data binding via IDs 1	Does binding reduce errors enough to justify conversion work?

This table compares design descriptions, not a head-to-head benchmark. The announcement does not report results comparing chat templates, JSON, and LLMON on the same tasks. ¹

IBM says its conversion tools and example notebooks translate JSON to LLMON and back without loss. Treat that as a migration path to verify against your own inputs; the announcement does not report tests of optional fields, deep nesting, or malformed inputs. ¹

Does LLMON actually cut prompt-injection risk?

IBM reports that LLMON models mostly ignore indirect prompt injections hidden in user data, attributing this to explicit instruction/data separation. It says attack success rates drop to nearly zero on some benchmarks, while regular instruction-tuned models are hijacked with high probability. ¹

The announcement does not name the benchmarks or provide the attack corpus, sample sizes, or baseline configurations. That makes it difficult to assess how the results might transfer to a retrieval pipeline, an agent loop, or a tool-use workflow. ¹

The proposed mechanism is instruction-to-data binding: an instruction can point to the data it governs. IBM says unrelated passages, old chat turns, and tool outputs do not get pulled into the answer because each instruction is bound to the data it applies to. ¹ That is a design goal, not evidence of a proven security boundary. Before relying on it, test it against the untrusted content in your own workflows, including retrieved documents, tool results, stale history, and text that mixes legitimate data with instruction-like phrasing. Our piece on how stale repository notes can distract coding agents covers a related failure mode: outdated context competing with the current task.

The announcement provides no figures for latency, overall token overhead, complex-schema adherence, or long-context behavior in model evaluations. ¹ Those are relevant to assessing whether any injection-resistance gain is worth the operational cost.

What changes in an existing stack

The preview models ship with their own chat template. IBM says developers can build the usual messages list and call apply_chat_template; message parts can be marked for inclusion in an execution block that specifies which instruction runs against which data. ¹

So the existing message abstraction may survive, while selected content inside it gets tagged differently. The integration work may involve handling message parts, execution blocks, IDs, and the JSON/LLMON conversion boundary. Check the released toolkit and your orchestration code before estimating the effort. ¹

IBM says teams can define custom tags and tune a LoRA so a model learns them. The repository released alongside the previews includes an LLMON toolkit and example workflows. IBM also says its Mellea framework has native LLMON support, automatically converting model calls. Teams using other orchestration layers should verify conversion, observability, and error handling in their own stacks. ¹

One concrete detail: LLMON block IDs are two characters drawn from a fixed set, and the tokenizer encodes them as single tokens. IBM says addressing a block therefore costs two tokens regardless of the ID. ¹ That is a tokenizer detail, not evidence that a full LLMON prompt is cheaper than an equivalent JSON or plain-text prompt; the announcement provides no such comparison. ¹

What evidence is still missing

IBM says the preview models maintain general capability relative to Granite 4.1 Instruct models, but the announcement supplies no numbers. That is a claim to verify, not a quantified result. ¹

A useful test would compare each LLMON preview with its Granite 4.1 Instruct counterpart on identical tasks, prompts, context, decoding settings, and scoring. That would help isolate whether the format changes outcomes or mainly changes syntax. Small-model evaluations have their own traps; see why the benchmark table isn't the hard part for a small on-device agent.

Four dimensions worth tracking:

• Structured-output validity: do outputs conform to schema, including nested and optional fields?
• Instruction-to-data binding: does the model apply each instruction to the right content when multiple instructions and payloads are in play?
• Prompt-injection resistance: what is the attack success rate on inputs that resemble yours?
• Operational cost: what are the latency and token overhead? IBM reports neither. ¹

Test long contexts, irrelevant history, tool output, mixed trusted/untrusted content, and failed JSON conversions. The announcement does not report how the models perform on those cases. ¹

What to do next

Pick one bounded workflow where instruction scope and untrusted input both matter. Build paired test cases, one in LLMON and one in your current format, and compare task quality and injection outcomes side by side. This is a contained way to test the claim without rewriting every prompt in your stack. ¹

Keep the trial reversible. Use IBM's converters and toolkit, keep your JSON or message format intact at system boundaries, and diff outputs before changing downstream contracts. IBM describes JSON-to-LLMON conversion as lossless; verify that against your own schemas before relying on it. ¹ If your traffic already runs through an enterprise AI gateway, keep your existing routing and data-policy constraints in force during the trial rather than loosening them for an experiment.

Set your adoption bar before reviewing results: a measurable drop in successful injections, no unacceptable hit to task quality, and overhead your system can absorb. Those thresholds are decisions for your team, not figures supplied by IBM.

And don't let LLMON stand in for input handling, tool authorization, or output validation. The announcement describes a context-representation format and reports benchmark behavior; whether it is sufficient for any security purpose needs to be evaluated in your own system. ¹

Bottom line: LLMON gives models a more explicit way to mark what's instruction, what's data, and how the two connect. The Granite 4.1 previews make that idea testable, but the announcement leaves the performance and security results underspecified. Test it against your own attacks and schemas before using it in production.

Frequently asked questions

What is LLMON markup language?

LLMON, or LLM Object Notation, is an LLM-native markup language that uses meaningful tags and reserved special-token delimiters to represent structure and semantics in model context. ¹

Is LLMON a replacement for chat templates?

No. Chat templates represent roles such as system, user, and assistant; LLMON adds component meanings, nesting, and relationships within and across those roles. ¹

Which Granite models support LLMON?

The announcement introduces granite-4.1-3b-LLMON-preview and granite-4.1-8b-LLMON-preview, derived from Granite 4.1 base models and trained to consume and generate LLMON-marked text. ¹

Does LLMON prevent prompt injection?

IBM reports that LLMON models mostly ignore indirect prompt injections hidden in user data and that attack success fell to nearly zero on some benchmarks. The announcement does not provide the benchmark names, sample sizes, or exact results, so teams should validate the claim in their own workflows. ¹

Can teams convert JSON to LLMON?

Yes. IBM provides converters and example notebooks for translating JSON to LLMON and back, and says the conversion is lossless. Teams should test that behavior on their own schemas and malformed-input cases. ¹

Sources

¹ “LLMON: An LLM-Native Markup Language, now in Granite 4.1 3B and 8B,” Hugging Face, published 2026-11-12. https://huggingface.co/blog/ibm-granite/llmon-an-llm-native-markup-language-now-in-granite