Semantic Text versus Machine-Oriented JSON for Tool Results

Abstract

Tool-using language models act on observations supplied by software. Those observations are often serialized as JSON because JSON is convenient for the application, even when the model is the intended reader. An alternative is to retain structured data for machine consumers while presenting the model with a deterministic semantic-text projection that makes relationships, direction, absence, and available actions explicit.

This study compares two end-to-end representation bundles in a read-only knowledge-graph agent: semantic text with instructions written for that representation, and a machine-oriented JSON envelope with its corresponding instructions. The unit of comparison is therefore the complete bundle, not serialization syntax alone. A paired battery of 1,181 questions with database-derived expected answers was evaluated over a large public-transport knowledge graph. Four phases produced 4,301 retained turn-level records, covering a primary operating condition, a constrained tool budget, a smaller-model condition, and a repeatability check.

In the primary condition, strict terminal-answer accuracy was 60.0% with the semantic-text bundle and 35.1% with the JSON bundle, a paired difference of +24.9 percentage points (95% cluster-bootstrap interval: +21.5 to +28.0). On the fixed 200-question pressure subset, accuracy was 43.0% versus 9.5% at the standard budget and 41.8% versus 8.3% at the constrained budget; the observed gap did not widen when the budget was reduced. Under the smaller-model condition, accuracy was 30.0% versus 17.5%, and the gap narrowed.

The primary difference combined two effects. The semantic-text bundle produced a final answer more often: 408 records ended in an unanswered clarification, compared with 711 for JSON. It was also more accurate when it did answer: 87.4% versus 77.5%. Semantic text yielded fewer emitted tool-result characters, but provider-reported input tokens per question and latency did not show a clear difference. The observed ratio of retained input tokens to correct answers favoured semantic text, although that ratio excludes failed retry attempts and is not a complete cost estimate.

The findings support an operational preference between the two evaluated bundles under the tested conditions. They do not establish that prose is generally superior to JSON, isolate representation from instruction and projection effects, or predict the performance of a compact semantic JSON designed specifically for model consumption.

Keywords: tool-using language models; agent evaluation; tool-result representation; semantic text; JSON; knowledge graphs; clarification behaviour.

Executive summary

The study asks a practical interface question: when an application already understands the meaning of its data, should it pass a machine-oriented serialization directly to a language model, or render that meaning explicitly for the model while preserving structured data for the rest of the system?

The comparison is deliberately end to end. Each arm includes the model-facing representation and the instructions associated with it. The underlying data, read operations, permissions, execution limits, and expected answers are held constant. This design measures the configurations as complete bundles, but it cannot attribute the result to text syntax alone.

Four conclusions follow.

1. The semantic-text bundle was more accurate in every evaluated paired condition. The main paired difference was large and appeared across the broad task groups rather than arising from one isolated category.
2. Budget pressure did not produce the predicted interaction. On the exact pressure subset, reducing the tool budget changed both arms by only about 1.2 percentage points. These questions were already difficult under the standard budget.
3. The smaller-model result did not support the predicted interaction. The semantic-text score fell while the JSON score rose relative to the primary model on the same questions. The experiment establishes a smaller observed gap in that condition, not a general explanation for it.
4. Answer production is central to the result. The JSON bundle generated substantially more clarification interruptions without a final answer. Conditional on answering, the semantic-text bundle still retained a ten-point advantage. Reporting both measures prevents clarification behaviour from being mistaken for factual error alone.

The efficiency evidence is mixed. Semantic text was shorter under the study's character counter and produced fewer observed procedural errors. Mean provider-reported input tokens per question and mean latency were inconclusive. A lower observed token total per correct answer favoured semantic text, but it is an outcome-dependent ratio and omits failed retries. It should not be presented as a complete percentage reduction in operational cost.

The bounded practical conclusion is that the semantic-text bundle is the stronger of the two evaluated configurations for this read-only graph-agent workload. The highest-value next experiment is a controlled ablation comparing semantic text with compact semantic JSON while holding selected facts, framing, and instructions constant.

1. Problem and motivation

1.1 The design thesis

The evaluated system is a metadata-driven knowledge-graph platform, and those two capabilities are what make the question of this paper askable at all. The metadata model of a domain is explicitly authored by the people who actually know what the data means — and it is configuration, not a fixed schema: it is extended and revised as the domain evolves, and downstream views follow it. Every object type carries a business definition, every relationship type carries its meaning as a pair of directional readings (when the TfL graph was modelled, Line Service was given serves from the line's side and is served by from the station's side), and every constraint carries its cardinality (a station sits in at most one fare zone; a line may have timetables). Every data record is then classified against that model — so the semantics exist at both levels, in the metadata and stitched to the data, as first-class, queryable content. The meaning of the data is governed alongside the data rather than left in a separate document or a veteran's head.

Most enterprise data has nothing like this. A foreign key has no name for itself; a join table called line_stop_xref does not know whether the line serves the station or the station serves the line — and neither does any AI pointed at it, which is why so much "chat with your data" tooling burns tokens guessing at semantics nobody ever wrote down.

That made the default tool-result design look backwards. Handing a language model the raw JSON control envelope asks the most expensive, least reliable interpreter in the stack to reconstruct meaning that the platform already holds — probabilistically, per call, forever: mapping a relationship identifier to a name, a direction: incoming to a verb reading, a count to a claim. The design thesis: since the platform masters the semantics, render them at the source — a deterministic pure function turns the captured semantics into semantically explicit text, the semantic datasheet — and hand the model finished understanding, so it does not have to guess. Every inference pre-computed where the meaning lives is an inference the model cannot fluff; the typed envelope continues to serve the machine consumers (UI, guards) on its own channel, unchanged.

The evaluated configuration used this semantic datasheet as its model-facing representation. The question this paper answers is whether it is, in fact, better suited to this AI-agent workload than the machine-oriented JSON envelope used as the control, rather than merely appearing elegant in principle. The stakes are operational: correctness (comprehension and deduction errors, schema-versus-data confusion), efficiency (context payload and token cost on every call), governance (internal machine fields leaking into user-facing answers; a deterministic, auditable rendering versus improvised summarisation), and maintainability (owning a second, versioned rendering path).

1.2 The representation problem

The evaluated design is one instance of a problem every agentic system has, whether its builders notice or not: a tool result has two consumers with opposite needs. The application is a machine — its guards, UI, and next-move machinery need exact identifiers, typed fields, counts, and provenance, and JSON serves it perfectly. The model is a reader — its training is overwhelmingly human text, and everything it says next is built from what it comprehends of the result. Almost every agent framework collapses both consumers onto one artifact: the machine's serialization is dropped into the model's context, and the model is left to reconstruct meaning from structure. Each reconstruction is a small probabilistic step; an agent chains dozens of them per question; and the attention spent parsing braces is attention budgeted away from reasoning.

Talking precisely about this problem requires separating four things that "format" usually conflates:

• Serialization — how facts are encoded for a reader: a JSON envelope versus deterministically rendered text. The narrowest of the four.
• Semantic projection — which facts are selected, framed, and prioritised for the model: suppressing internal bookkeeping, capping long lists, grouping empty relationship types, adding framing the structure cannot carry ("zero links today — never report these as existing"). A projection is deliberately not byte-equivalent to the envelope; it is an editorial act, performed deterministically.
• Instructions — the system prompt and tool descriptions that teach the model to read what it receives. In an operating system they co-evolve with the representation; together, they form a bundle.
• Transport channels — which artifact reaches which consumer. Nothing forces one artifact to serve both readers: the typed envelope can continue to drive the machine unchanged while the model reads a projection, so choosing text for the model surrenders nothing to the machine.

The present study compares two complete operating bundles: serialization, projection, and instructions vary together, while the tools, governed-read rules, machine channel, and everything beneath the rendering are held identical. A claim about serialization alone would require holding the projection and instructions fixed across arms — a different experiment, stated as future work.

1.3 Evidence gap

Prior work establishes that representation choices can materially change how language models reason over graphs and structured data. Graph-encoding studies show that performance varies with the encoding, task, and graph structure (Fatemi, Halcrow and Perozzi, 2024), while KG-LLM-Bench extends that observation across several textualization strategies, tasks, and models (Markowitz et al., 2025). KG-to-text work reports gains from verbalizing retrieved graph evidence (Wu et al., 2023), and StructGPT shows the value of specialized interfaces that separate evidence collection from reasoning (Jiang et al., 2023). The closest direct evidence concerns tool outputs themselves: Kate et al. (2026) find that processing complex JSON tool responses remains difficult even for strong models, and that the best processing strategy depends on output size, content, and reasoning demand.

Together, these studies establish that representation matters, but leave a narrower operational question unresolved: when one application can provide either its machine-oriented envelope or a deterministic semantic projection, how do the complete representation-and-instruction bundles compare inside the same governed agent on the same questions? This study addresses that gap.

2. The evaluated agent system in context

The evaluated agent navigates a knowledge graph through governed read operations rather than direct query-language or database access. It searches for a record, reads that record's typed properties and relationship overview, and follows a relationship offered by the preceding result. Each result therefore contains both evidence and a constrained set of available next actions. Three functional tool roles support that walk:

• Search — find candidate records for a name or phrase.
• Summary — orient on one record, its properties, and the distinction between relationships that contain records and relationships that are merely permitted by the metadata model.
• Related-record retrieval — follow one available relationship to neighbouring records, with pagination and a reported total.

The model decides whether to read again, ask for clarification, or produce a final answer. The surrounding routing, budget enforcement, and read controls are deterministic and apply equally to both representation arms. Every question also has a hard step budget, making unnecessary reads observable and allowing the same workload to be tested with standard and constrained budgets.

All measured questions used the same authenticated, permission-scoped, read-only access context. The two arms traversed the same visible graph through the same operations; neither received privileged database access. Conversation state was preserved within multi-turn items so that references such as "which lines serve it?" could resolve against earlier turns; state was reset between unrelated items, and a retry began from a fresh conversation.

The model-facing semantic datasheet presents one object's facts as a disciplined page of text: what it is, its properties, the relationships it actually has, and what the metadata permits but the data does not currently contain. It is produced deterministically from the canonical result, without a second language model in the rendering path. Directional readings become verb phrases — a count of 6 on the Line Service relationship becomes "This Stop Point is served by 6 Lines" — while business definitions and cardinalities shape the surrounding explanation. Relationships with records appear under LINKED; relationships allowed by the metadata but empty in the current data appear under CAN BE LINKED with a warning not to report them as existing.

3. Competing representations

3.1 Semantic rich text

The rich-text arm delivers the semantic datasheet as the tool message the model reads. Its construction is fixed by the renderer, not the run: a subject header (title, type, and business definition), typed properties, a LINKED section with one authored sentence per relationship type holding records, and a CAN BE LINKED section listing schema-allowed, record-free relationship types. The publication-safe excerpt below shows the form of the semantic result for King's Cross St. Pancras. Internal references and nonessential fields are redacted.

King's Cross St. Pancras Underground Station — a Stop Point
Object type "Stop Point" (schema): A line-serving NaPTAN stop point:
station, bus stop, pier, tram stop or cable-car station

PROPERTIES
Public identifier: 940GZZLUKSX
[Additional properties omitted]

LINKED — actual data records
1. This Stop Point is served by <Line> (6 linked records).
   [Navigation reference withheld]
2. This Stop Point is in zone <Fare Zone> (1 linked record).
   [Navigation reference withheld]
[Additional linked relationship types omitted]

CAN BE LINKED (schema only, zero links today — never report these as existing)
As a Stop Point, this object could:
- has car park: Car Park
[Additional allowed relationship types omitted]

3.2 Control-envelope JSON

The JSON arm delivers the machine's rendering to the model: the canonical result serialized with its typed properties, relationship data, available actions, identifiers, counts, directions, cardinalities, and provenance. It retains machine fields that the projection suppresses and does not add natural-language framing. It is an existing control-oriented representation, not a JSON design optimized for model consumption. The schematic below generalizes internal field names and withholds references while preserving the nesting and cross-referencing burden. It contains the same answer-bearing facts as the semantic excerpt.

{
  "subject": {
    "type": { "title": "Stop Point", "description": "A line-serving NaPTAN stop point" },
    "title": "King's Cross St. Pancras Underground Station",
    "properties": { "public_identifier": "940GZZLUKSX", "additional_properties": "[omitted]" },
    "relationships_allowed": [
      { "related_type": "Car Park", "direction": "incoming", "present_count": 0, "reference": "[redacted]" }
    ]
  },
  "available_actions": [
    { "action": "retrieve_related", "reference": "[redacted]",
      "expects": { "relationship": "Line Service", "direction": "incoming", "count": 6,
                   "related_type": { "title": "Line" } } },
    { "action": "retrieve_related", "reference": "[redacted]",
      "expects": { "relationship": "Fare Zone Membership", "direction": "incoming", "count": 1,
                   "related_type": { "title": "Fare Zone" } } }
  ]
}

The answer-bearing correspondence is visible: the first available action carries a count of 6, the related type Line, and an incoming direction; the datasheet states that fact as "is served by 6 Lines." The two representations are intentionally not byte- or field-equivalent — which is why the result must be interpreted as a bundle comparison rather than a pure syntax comparison.

3.3 Shared invariants, intended differences, and confounds

Every row of the comparison is classified in advance as an invariant (identical across arms), an intended difference (the thing being measured), or a bundled difference (varies with the arm by design, and therefore confounds any claim narrower than the bundle).

3.4 Strongest case for each representation

Semantic rich text — strongest where comprehension is the bottleneck: multi-fact synthesis across many results (fan-out, deep chains), explicit negatives the structure cannot say ("zero links today"), directional readings already resolved into verbs, and a smaller payload that leaves budget and attention for reasoning.

Control-envelope JSON — strongest where verbatim fidelity and explicit structure beat readability: exact long values reproduced character-for-character, totals carried in an explicit field rather than a sentence (pagination totals), the machine-level distinction between a property stored false and a property absent, and identifier fidelity. A subtler expected strength: opacity resists the garden path — structure that does not read fluently cannot be fluently misread.

4. Research questions, hypotheses, and design

4.1 Research questions and hypotheses

• RQ1 / H1 — accuracy at slack. Under the primary model and standard budget, are the two bundles equivalent within a predeclared margin of ±3 percentage points?
• RQ2 / H2 — budget-pressure interaction. Is the format effect larger at the constrained budget than at the standard budget?
• RQ3 / H3 — model-pressure interaction. Is the format effect larger under the small model than under the primary model?
• RQ4 / H4 — efficiency. At equivalent accuracy, does the rich-text arm consume fewer payload bytes/tokens and produce fewer procedural failures?
• RQ5 — where each wins (exploratory). Across the battery's blocks, where does each representation hold the advantage?

4.2 Claim and interpretation bounds

This study supports bundle-level, per-condition claims about the two evaluated configurations. Explicitly out of scope: serialization-only causal claims (instructions are bundled with each format by design); claims about JSON representations designed for model consumption (no such arm exists in this round); confirmatory per-category claims (category effects are exploratory); and generalization beyond the evaluated graph shape, domain, permission profile, model families, and budgets. A paired interval wholly on one side of zero supports a condition-specific accuracy advantage; an interval wholly inside ±3 points supports practical equivalence.

4.3 One paired comparison, four phases

The design called for 4,302 records; one follow-up could not be produced, leaving 4,301 analysed records. Budgets of 15 (standard) and 8 (constrained) were enforced as strict maxima. Provider-default decoding settings were used.

5. Methods

5.1 Dataset

The evaluation graph is built from Transport for London's official open data, extracted via the TfL Unified API through a three-stage pipeline: extraction, metadata modelling (41 object types, 25 relationship types with directional readings, typed properties, and cardinality constraints), and import. The resulting evaluation space holds 380,248 objects, including 36,922 stop points, 63,172 route segments, 726 lines, 917 platforms, 96 interchanges, 16 fare zones, 50,626 accident records, and associated infrastructure. The graph is multi-typed, with high-degree hubs, long-tail records, and schema-versus-data asymmetries — for example, the metadata permits line timetables, but only 6 of the 726 line records in the snapshot carry one. The snapshot remained read-only from the start of the first measured phase to the end of the last.

5.2 Question battery

Questions were generated programmatically from the graph itself, rather than hand-written, to reduce author selection bias and ensure that every question carries a machine-checkable expected answer. The battery holds 1,181 questions (load 481, probes 240, structure 60, baseline 400).

Load categories (comprehension under pressure): fan-out synthesis (max / aggregate), deep chains (multi-hop), direction pairs, multi-turn threads, needle-in-clutter property lookups, and full enumeration. Behavioural probes (judgement under temptation): schema-versus-data traps, absent entities, deliberate ambiguity, live status, and comparison. Structure-favouring categories (home field for JSON): exact long values, pagination totals, and null-versus-false fidelity. Baseline: a 400-question single-fact sample drawn deterministically from a 763-question pool regenerated from the graph.

Every expected answer was computed from the graph before any agent run. Sixteen ground truths — one per category across all four blocks — were independently re-derived from the frozen snapshot; all 16 matched.

5.3 Measurement and grading

For every analysed turn, the driver captured the final answer, the exact tool-result text delivered to the model, in-band tool errors, provider-reported token usage, clarification events, model call count, steps used, and wall-clock time.

Every run record flowed through one grading pipeline: a deterministic grader produced an initial verdict and confidence level; low-confidence cases went to three representation-blind judges with majority vote; and a separate sample of confident automated verdicts was independently audited. The battery, grade specifications, grader, and judge rubric were frozen before outcomes were opened. Interval estimates use a cluster bootstrap over question families, because turns within a thread, direction pairs, fan-out members, and repetitions are not independent.

6. Results

A positive difference means that semantic text produced more correct final answers than machine-oriented JSON on the same questions.

6.1 Integrity checks and sample accounting

The frozen-snapshot check passed, the hand-checked ground-truth sample was exact on all 16 items, and budget-enforcement and model-input-capture checks passed before outcomes were opened.

One phase-one semantic-text turn failed after all retries and was scored incorrect; the following turn in that conversation could not be run, accounting for the one-record difference. The paired primary analysis therefore uses the 1,300 records available under both representations.

6.2 Headline outcomes

H1 asked whether the two bundles were close enough to treat as equivalent within ±3 points. That criterion was not met. This is stronger than merely failing to prove parity: a separate 95% interval also excludes zero by a wide margin, supporting a superiority reading on this battery. The result stays within its claim boundary — it does not isolate rendering from the accompanying instructions, and does not imply that prose is universally preferable to structured data.

6.3 What changed under budget and model pressure

The pressure comparison uses the same fixed set of 200 questions in every row.

Budget pressure. Tightening the budget did not widen the observed gap; both accuracies fell by about 1.2 points, leaving the difference unchanged at 33.5 points. This does not support H2 on this subset. Model pressure. The smaller model still favoured semantic text, but the gap narrowed from 33.5 to 12.5 points — semantic-text accuracy fell by 13 points while JSON rose by 8. Because the arms moved in opposite directions, the result should not be read as a simple "capability floor"; the mechanism remains unresolved.

6.4 Producing an answer was part of the outcome

Strict accuracy combines two questions: did the model produce a final answer, and was it correct? Phase one differed on both.

Most stalls happened after the system had already executed at least one tool — 364 of 408 semantic-text stalls and 652 of 711 JSON stalls occurred after tool use. That pattern is compatible with the representation and instruction wording affecting whether the model treats the available evidence as sufficient to answer. It is not a causal mediation estimate.

6.5 Efficiency and procedural reliability

The clearest result is narrower than the original H4 claim: semantic text emitted about 1,450 fewer tool-result characters per question, with an interval excluding zero. Mean input tokens and latency showed no clear paired difference. Observed retained input tokens per correct answer were 14,914 versus 23,866 (37.5% lower for semantic text) — a useful operational ratio, but outcome-dependent and excluding failed retries, so not a complete cost estimate. Rare procedural events pointed the same way but were too sparse to explain the accuracy gap: 24 versus 54 tool errors, and budget exhaustion in 6 versus 15 records. H4 is not testable exactly as written, because its accuracy-equivalence premise failed and the declared byte/token measures were not retained.

6.6 Exploratory category patterns

Category findings are exploratory, but the broad pattern is useful: the primary difference appears in every major block, including the block designed to give structured representation its strongest hearing.

Directionality questions produced the largest separation (96/120 vs 19/120), followed by live-status (26/40 vs 1/40) and needle-in-clutter (64/100 vs 21/100). JSON took two narrow category wins — pagination totals (7/20 vs 6/20) and deliberate ambiguity (28/40 vs 26/40) — both too small to generalise. Absent-entity questions were a shared failure (1/40 for both).

6.7 Robustness and disposition of the hypotheses

On the closing 100-question repeatability control, 76 outcomes matched between the opening and closing runs (13 correct only at the close, 11 only at the opening), arguing against a large endpoint shift. A separate grader audit agreed with 95 of 102 sampled confident verdicts (93.1%). Of 272 adjudicated items judged three times, 243 were unanimous.

7. Discussion

The two bundles were not operationally interchangeable. Semantic text produced correct final answers on 60.0% of matched records versus 35.15% for JSON; the 90% interval lay entirely beyond the ±3-point equivalence region. The predicted widening under budget pressure did not appear, and the model-pressure change ran against the prediction. The appropriate conclusion is a substantial semantic-text advantage on this battery — not a universal claim about prose and JSON as data formats.

What appears to drive the difference is the production of a final answer: 408 semantic-text records versus 711 JSON records ended in clarification stalls, most after at least one tool execution. Correctness among answered records also differed (87.4% vs 77.5%). Both channels are affected by the experimental bundle, so neither is independently identified — the safest reading is that the bundles differed both in answer production and in the quality of the answers produced.

7.1 Rival explanations

• Instruction bundling — the strongest rival. Each representation used the prompt and tool descriptions designed for it; that prevents attribution to rendering alone. A common-instruction ablation is required before any format-only claim.
• Editorial selection — the projection selects, orders, labels, and suppresses fields; its advantage may come from those editorial decisions rather than natural-language syntax.
• Model-specific decision behaviour — the primary model may have been unusually willing to request clarification given structural output; the smaller-model condition shows the effect size is not stable across models.
• Run sequencing and drift — arms ran sequentially; the repeatability control bounds but cannot exclude a transient change between its two measurements.
• Benchmark construction — battery and projection came from the same development context; independent authorship and replication remain necessary.

7.2 The strongest case for JSON

The evaluated JSON envelope was designed primarily for deterministic machine and interface use — not as the best possible structured representation for a language model. The dominant observed difference was also a behavioural end state (JSON asked for clarification more often), which a revised prompt might reduce without changing serialization. JSON retained narrow advantages where explicit structure or caution aligned with the rubric, and continues to offer deterministic parsing and stable field identity outside the model-facing channel. This defines the claim boundary: the evidence supports choosing between these two evaluated bundles; it does not establish that every carefully designed structured representation would perform worse. The most informative next comparison is the same selected semantics expressed once as controlled natural language and once as compact semantic JSON, under identical instructions.

7.3 Boundary conditions

The result covers one production-shaped graph snapshot, one agent architecture, one primary model, one smaller model, two step budgets, one language, and a read-only workload. Several boundaries matter especially: metadata quality (a sparse or stale metadata layer would produce weaker text and could reverse the comparison), task type (reading and answering only — not writes, transactions, or autonomous actions), clarification policy (the strict outcome treats unnecessary clarification as failure), and metadata-rich source (the semantic representation benefits from an authored semantic layer; systems without one must first create and maintain it).

7.4 Architectural implications

The strongest architectural lesson is not "replace JSON with prose." It is: do not require one representation to serve every reader. A canonical structured result can remain the stable machine-and-interface contract while a separate projection prepares the same result for a model. That projection should be treated as governed product logic — its selection, ordering, wording, omission rules, identifier policy, and instructions versioned and tested together. The canonical structured result stays the source of truth; model-facing text is a derived view, making semantic rendering replaceable, auditable, and reversible. The identifier-leakage finding adds a safety requirement: model-facing references should use opaque, non-sensitive handles.

8. Limitations and threats to validity

• Strict terminal-answer accuracy embeds a product judgement — a clarification request on a determinable task is counted incorrect; a safety-critical assistant might reasonably prefer asking.
• Bundle, not serialization — representation, instructions, and editorial selection changed together; a pure prose-versus-JSON effect is not identified.
• Efficiency measurement — emitted string characters are not bytes, model tokens, or cumulative context exposure; retained totals omit failed attempts before recovered retries.
• Sequential execution — runs were not question-by-question randomized; a transient provider or infrastructure change could remain undetected.
• Stochastic outputs — only the tight-budget phase was repeated; other conditions provide one response per representation and question.
• Curated battery — intervals resample question-family clusters within the constructed battery; they are not survey intervals for a sampled population of future questions.
• Two model conditions and one frozen snapshot do not support a model-independent or domain-independent conclusion.
• Leakage is a screening signal, not a security assessment; the detector may produce false positives or miss domain-specific leakage.
• No maintenance-cost measurement — a separate projection introduces code, metadata, testing, and versioning work the study does not measure.

These limitations narrow the conclusion without erasing it. The evidence supports a substantial advantage for the evaluated semantic-text bundle on the evaluated battery. It does not identify a universal winning serialization, a single causal mechanism, or a complete cost advantage.

9. Conclusion and future work

9.1 Scoped conclusion

In the primary condition, semantic text produced correct final answers on 60.0% of 1,300 matched records, compared with 35.15% for JSON — a paired difference of +24.85 percentage points (95% cluster-bootstrap interval +21.53 to +27.99), with the predeclared parity criterion not met. Semantic text also led under the tight-budget and smaller-model conditions, though the pressure analyses do not support the original predictions that the difference would widen. The experiment found fewer clarification stalls and shorter emitted tool-result strings under semantic text, but did not establish a complete cost advantage.

For this evaluated agent, dataset snapshot, model pairing, and question battery, the semantic-text bundle produced substantially more correct final answers than the machine-oriented JSON bundle.

That statement supports a local design decision based on answer accuracy. It does not show that prose is universally superior to JSON, that rendering alone caused the difference, that a model-optimized semantic JSON would lose, or that the observed character reduction is a complete cost saving.

9.2 Future work

• Separate presentation from the bundle — hold facts, ordering, tool descriptions, and instructions fixed while varying only surface representation; then compare semantic text with compact, model-oriented JSON carrying the same selected semantics.
• Test clarification as an independent policy — vary only the answer-versus-ask instruction, then resume clarification turns with controlled user replies.
• Replicate independently and repeatedly — a broader model panel, interleaved run order, an externally authored battery, and a fully crossed human audit.
• Measure complete cost and broaden safety coverage — retain every attempt, tokenizer-specific payload counts, cumulative context exposure, monetary cost, and end-to-end latency; extend to sparse metadata, multilingual data, permission denials, write actions, and identifier leakage.

Until those studies are complete, the result should be used as evidence for the evaluated bundle and setting, not as a general hierarchy of data formats.

References

1. Fatemi, B., Halcrow, J. and Perozzi, B. (2024). "Talk Like a Graph: Encoding Graphs for Large Language Models." International Conference on Learning Representations 2024.
2. Jiang, J., Zhou, K., Dong, Z., Ye, K., Zhao, X. and Wen, J.-R. (2023). "StructGPT: A General Framework for Large Language Model to Reason over Structured Data." EMNLP 2023, 9237–9251. aclanthology.org/2023.emnlp-main.574
3. Kate, K., Rizk, Y., Ghosh, P., Gulati, A., Chakraborti, T., Wright, Z. and Agarwal, M. (2026). "How Good Are LLMs at Processing Tool Outputs?" EACL 2026, 2918–2941. aclanthology.org/2026.eacl-long.134
4. Markowitz, E., Galiya, K., Ver Steeg, G. and Galstyan, A. (2025). "KG-LLM-Bench: A Scalable Benchmark for Evaluating LLM Reasoning on Textualized Knowledge Graphs." arXiv preprint arXiv:2504.07087
5. Transport for London (2026). "Our open data." Accessed 29 July 2026.
6. Wu, Y., Hu, N., Bi, S., Qi, G., Ren, J., Xie, A. and Song, W. (2023). "Retrieve-Rewrite-Answer: A KG-to-Text Enhanced LLMs Framework for Knowledge Graph Question Answering." arXiv preprint arXiv:2309.11206