Module 2 · Structured Outputs and Tools · 55 min

Structured Outputs With JSON Schema

Lesson 4 of 18

Is lesson ke baad tum

Structured output is a versioned contract, not a truth machine: define the consumer's schema, reject malformed and unexpected data, verify source evidence separately, and keep consequential actions behind authorization and human review.

In this lesson

Make a support-triage result safe for another program to read. You will run a supplied JSON Schema against three local fixtures, then prove that a structurally valid result can still cite invented evidence. You need Node from Lesson 1.1 and one free package download; no model account is needed for the core task.

JSON Schema describes permitted fields and values. Parsing JSON only checks its syntax: {"priority":"extremely urgent"} parses successfully even when the queue accepts only low, normal, or high.

Start With the Consumer, Not the Prompt

Before writing a schema, identify the next component. Suppose a small ecommerce team wants to triage synthetic support messages. The queue accepts three priorities, requires a short reason, and needs an explicit boolean showing whether a human must review the result. That downstream need determines the shape:

json37 lines

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "title": "SupportTriage",
  "type": "object",
  "properties": {
    "category": {
      "type": "string",
      "enum": ["delivery", "product", "payment", "other"]
    },
    "priority": {
      "type": "string",
      "enum": ["low", "normal", "high"]
    },
    "reason": {
      "type": "string",
      "minLength": 10,
      "maxLength": 180
    },
    "needs_human_review": {
      "type": "boolean"
    },
    "evidence_phrases": {
      "type": "array",
      "items": { "type": "string", "minLength": 1, "maxLength": 80 },
      "minItems": 1,
      "maxItems": 3
    }
  },
  "required": [
    "category",
    "priority",
    "reason",
    "needs_human_review",
    "evidence_phrases"
  ],
  "additionalProperties": false
}

Every required field has a type. Enumerations stop category drift. Length limits reduce unbounded output. additionalProperties: false prevents an apparently helpful model from inventing fields the consumer silently ignores. The evidence array asks the extractor to point back to supplied text; your code must still confirm each phrase actually appears in that text.

Do not represent “unknown” by quietly omitting a required field unless the consumer understands omission. Choose a deliberate design: a nullable field, an unknown enum member, or a separate needs_human_review flag. Required does not mean “the model must guess.”

Worked Example: Validate Fixtures Before Calling a Model

Save the schema as triage.schema.json, then install a general JSON Schema validator:

powershell1 line

npm install ajv

Create validate.mjs:

javascript50 lines

import { readFile } from "node:fs/promises";
import Ajv2020 from "ajv/dist/2020.js";

const schema = JSON.parse(await readFile("triage.schema.json", "utf8"));
const ajv = new Ajv2020({ allErrors: true });
const validate = ajv.compile(schema);

const fixtures = [
  {
    name: "valid",
    value: {
      category: "product",
      priority: "normal",
      reason: "The synthetic message reports that the received colour is wrong.",
      needs_human_review: true,
      evidence_phrases: ["wrong colour"]
    }
  },
  {
    name: "invalid-priority-and-extra-field",
    value: {
      category: "product",
      priority: "urgent-now",
      reason: "Wrong colour was reported.",
      needs_human_review: true,
      evidence_phrases: ["wrong colour"],
      promised_refund: true
    }
  },
  {
    name: "invalid-empty-evidence",
    value: {
      category: "product",
      priority: "normal",
      reason: "The synthetic message reports that the received colour is wrong.",
      needs_human_review: true,
      evidence_phrases: []
    }
  }
];

let failed = false;
for (const fixture of fixtures) {
  const ok = validate(fixture.value);
  console.log(fixture.name, ok ? "PASS" : "FAIL");
  if (!ok) console.log(validate.errors);
  if ((fixture.name === "valid") !== ok) failed = true;
}

if (failed) process.exitCode = 1;

Run node validate.mjs. The first fixture should pass. The second should fail for the unsupported enum value and extra property. After the one-time Ajv download, these fixture tests run locally without an API request. They check the contract before introducing a model.

Now add a semantic check that JSON Schema cannot express easily:

javascript10 lines

function evidenceExists(sourceText, result) {
  if (!Array.isArray(result.evidence_phrases) || result.evidence_phrases.length === 0) {
    return false;
  }
  const source = sourceText.toLocaleLowerCase();
  return result.evidence_phrases.every((phrase) => {
    if (typeof phrase !== "string" || phrase.trim().length === 0) return false;
    return source.includes(phrase.trim().toLocaleLowerCase());
  });
}

Run it against the exact approved input. A model could return schema-valid evidence that never appeared in the source. Syntax, schema, and evidence are three separate gates.

Trace the Three Gates With One Message

Imagine the receiving program as a clerk with three different questions. First: can I read this envelope? Second: does it contain the fields my queue understands? Third: does its claim match the message? A successful answer to one question supplies no answer to the next. This is why catching a JSON parsing error and retrying the model does not fix a mistaken classification that is perfectly valid JSON.

Use this additional teaching fixture: The colour is correct. I did not request a refund. A proposed result says category payment, priority normal, reason The customer requested a refund., review true, and evidence request a refund. Trace it:

Gate	Observation	What you have learned
JSON parsing	The object can be parsed	Its punctuation and value syntax are usable
Local schema	The permitted fields, enum values and lengths pass	The queue can read the object without guessing its structure
Phrase occurrence	`request a refund` occurs inside the source	Those characters were present
Meaning comparison	The source says did not request	The reason reverses the customer's meaning

The mistake is not an obscure edge case. Extracting a short phrase can remove negation, a date, a speaker, or an exception. Requiring a longer quotation helps a reviewer see context, but length alone does not prove the conclusion. The repair here is to describe the actual message and retain the full relevant sentence as evidence. It is not to loosen the schema or rename the priority.

After adding evidenceExists to validate.mjs, append this local check. It uses the existing first fixture as its base, so it belongs in that file after the helper:

javascript10 lines

const negatedSource = "The colour is correct. I did not request a refund.";
const misleading = {...fixtures[0].value,
  category: "payment",
  reason: "The customer requested a refund.",
  evidence_phrases: ["request a refund"]};
if (!validate(misleading)) throw new Error("Expected schema-valid fixture");
if (!evidenceExists(negatedSource, misleading)) {
  throw new Error("Expected substring occurrence despite false meaning");
}
console.log("Negation fixture: structure and occurrence pass; meaning needs correction.");

This passing check deliberately demonstrates a limitation. It is not an automated claim that the reason is correct. Write the corrected reason yourself, such as The customer says they did not request a refund., and compare the entire sentence with the source.

Now change just one condition: the source becomes Please refund my demo order. Predict whether the old evidence phrase will pass before running the helper. It fails: the meaning is related, but the exact words request a refund do not occur. Repair the evidence to Please refund my demo order.; the occurrence check then passes. Also revise the reason to reflect a request, not an approval. These two cases show opposite limitations: a substring can pass while the conclusion is false, and a reasonable paraphrase can fail an exact-quotation check. Decide which contract you need and test that contract explicitly.

Optional: Connect a Provider

Providers expose structured outputs differently. OpenAI's Responses API uses SDK helpers to pair a schema with a request and return parsed output. This optional route uses API quota/credits or billing and the protected credentials from Lesson 1.3. Install openai and zod in the practice folder. Save the following as model-triage.mjs beside the local schema. The provider schema below uses a small supported subset; Ajv enforces the full local contract after parsing.

javascript40 lines

import OpenAI from "openai";
import { z } from "zod";
import { zodTextFormat } from "openai/helpers/zod";
import { readFile } from "node:fs/promises";
import Ajv2020 from "ajv/dist/2020.js";
const localSchema = JSON.parse(await readFile("triage.schema.json", "utf8"));
const validateLocal = new Ajv2020({ allErrors: true }).compile(localSchema);

const Triage = z.object({
  category: z.enum(["delivery", "product", "payment", "other"]),
  priority: z.enum(["low", "normal", "high"]),
  reason: z.string(),
  needs_human_review: z.boolean(),
  evidence_phrases: z.array(z.string()),
});

if (!process.env.OPENAI_API_KEY || !process.env.OPENAI_MODEL) {
  throw new Error("Set OPENAI_API_KEY and OPENAI_MODEL before running");
}
const client = new OpenAI({ maxRetries: 0, timeout: 30_000 });
const response = await client.responses.parse({
  model: process.env.OPENAI_MODEL,
  max_output_tokens: 1024,
  store: false,
  input: [
    { role: "system", content: "Classify only from the supplied synthetic message. Mark uncertainty for human review." },
    { role: "user", content: "My demo order arrived with the wrong colour." }
  ],
  text: { format: zodTextFormat(Triage, "support_triage") },
});

if (response.status !== "completed" || !response.output_parsed) {
  throw new Error("No completed structured result; review refusal or incomplete status");
}
if (!validateLocal(response.output_parsed)) throw new Error("Local schema rejected result");
const source = "My demo order arrived with the wrong colour.".toLowerCase();
if (!response.output_parsed.evidence_phrases.every(phrase => phrase.trim().length > 0 && source.includes(phrase.trim().toLowerCase()))) {
  throw new Error("Evidence was not found in the supplied message");
}
console.log(response.output_parsed);

Choose a model available to your project that supports Structured Outputs. Provider schema subsets differ from general JSON Schema, and some constraints differ for fine-tuned models; the local validator remains the application boundary. Treat refusals, incomplete responses, parse failures, timeouts, and empty outputs as explicit branches. Validate again at the boundary if the parsed object crosses into a different service. Never let “the SDK parsed it” become authorization to refund, charge, publish, or message somebody.

Schema Design Failure Cases

8 cases to diagnose

Valid structure, false content
require source pointers and verify them. Use human review for consequential decisions.
Schema too loose
free-form strings for status or currency create dozens of spellings. Use controlled enums where the business rules are genuinely finite.
Schema too rigid
a city enum copied from today's service area rejects a newly approved location. Version the contract and update it through a controlled change.
Numbers used for identifiers
phone numbers, CNIC-like identifiers, postal codes, and order IDs can contain leading zeros or punctuation. Store identifiers as strings and minimize sensitive fields.
Money stored as floating point
represent amounts using an agreed smallest unit or a decimal-safe strategy, along with an explicit currency code. Do not ask the model to infer exchange rates.
Unknown forced into a guess
include an uncertainty path and escalation field.
Schema changed without consumers
adding a required field can break older workers. Version schemas and test both producer and consumer.
Validator errors hidden
record the schema version, failure path, and safe reason; do not log the private payload by default.

Hands-On Exercise: Build a Contract That Rejects Bad Data

4 steps

Keep the supplied support-triage contract. Extend validate.mjs; do not build a second application.

Run the three supplied fixtures. Expect valid PASS, then two FAIL lines. Here FAIL means the bad fixture was correctly rejected; the command should still exit successfully because those failures were expected.
Copy the valid object and change only needs_human_review to the string "true". Predict the outcome, add it as an invalid fixture, and rerun. It must fail the boolean type check.
Copy the valid object again, but use evidence_phrases: ["refund approved"]. Append the semantic function above to the same file. Call both validate(candidate) and evidenceExists("My demo order arrived with the wrong colour.", candidate), using your copied object as candidate. Expect schema true, evidence false.
Add assertions for those two expected booleans; throw an error if either differs. Correct the phrase to wrong colour and change the evidence expectation to true. Keep the model's output separate from any refund action.

The evidence check proves a phrase occurs, not that the whole reason is true. A reason can still reverse the meaning of a source; a reviewer must compare the claim with its context. For a later contract change, changing a required field needs consumer tests and a schema version.

Completion Rubric

The original valid fixture passes and all three invalid fixtures fail for the expected field/constraint.
Invented evidence passes the schema but fails the separate source check; corrected evidence passes both.
Your assertions fail the command if observed results differ from the predictions.
You can explain why a valid triage object does not authorize a refund or verify every claim.

Sources

Practice quiz

Check yourself

4 questions · practise what you just learned

0/4

1 / 4 · diagnose
A triage object passes the schema but cites "refund approved", absent from the supplied message. What should happen?

Apna kaam check karo.

Practice complete ki? Result ko example aur lesson ke checks se compare karo.

This is your own completion record, not a graded assessment.

Kuch samajh nahi aaya? Ask a question →