AI School Pakistan
← Applied AI Builder0 of 18 complete
LessonCourse overview

Module 5 · Agents and Reliability · 40 min

Agents vs. Scripts: Choose the Simplest Reliable System

Lesson 13 of 18

Is lesson ke baad tum

Professional AI engineering is knowing where model judgement helps—and refusing to add autonomy where ordinary code is clearer, safer, and easier to operate.

In this lesson

Choose the Smallest Useful System

Run the supplied return router, change one input, and choose where a model would help. The core task uses Node and fictional business rules; it makes no API request and cannot issue a refund.

Use a script for known rules, a model-assisted workflow for a bounded interpretation step, and an agent when choosing the next tool/step is itself part of the problem. Each added degree of autonomy creates more behavior to test and control.

The three levels

Level 1: deterministic script

Inputs follow a known schema, rules are explicit, and the same input should take the same path. Examples include renaming files, validating CSV columns, calculating invoice totals, sending a reminder at a fixed time, or calling an API after a form submission. Ordinary code gives you exact conditions, straightforward tests, and predictable cost.

Level 2: model-assisted workflow

The sequence remains controlled by code, but a model performs one bounded judgement: classify a support message, extract fields from a receipt, summarize a document, or draft a response. The model returns a structured result; code validates it and chooses the next step. This covers a large share of useful business AI without an open-ended agent loop.

Level 3: bounded agent

The application gives the model a defined goal and a small tool set. The model may decide which tool to call next, inspect the result, and continue until a stop condition or approval point. This is justified when the next correct step cannot be fully known in advance—for example, investigating a support incident across a knowledge base, order system, and status service. Even here, permissions, budgets, maximum steps, and side effects remain controlled by application code.

OpenAI's current Agents guide makes the ownership choice explicit: use the Responses API when your application should own the loop, and the Agents SDK when you want the SDK to manage recurring tool calls, handoffs, sessions, tracing, guardrails, or resumable approvals. That is an implementation choice after you establish that an agent is necessary.

A decision test before you build

Can ordinary code express the requirement? Keep it deterministic. Does one fixed step need language interpretation? Add a bounded model step and validate its result. Does an investigation need to adapt its tool sequence to findings? Consider a bounded agent only after defining tools, permissions, maximum steps, elapsed/cost limits, and escalation.

Do not choose an architecture from its label. Compare the observable task: a script can validate an order age without a model, while a free-text complaint may need interpretation.

Worked build: handling ecommerce return requests

A fictional seller receives a return form with order ID, delivery date, reason, requested remedy, and optional message. The first prototype is often pitched as a “returns agent.” Break the job apart:

  • Validate required fields: deterministic code.
  • Check whether the order exists: deterministic API call.
  • Calculate days since delivery: deterministic code.
  • Look up the applicable policy: deterministic retrieval with version filters.
  • Classify an unstructured customer explanation: bounded model call.
  • Decide whether the case is automatically eligible: deterministic rules.
  • Draft a courteous reply: bounded model call using verified facts.
  • Approve a refund above a threshold: human decision.

There is no reason for an agent to invent this sequence. A controlled workflow is easier to test and safer around money.

Here is a minimal runnable JavaScript version of the deterministic spine:

javascript32 lines
function routeReturn({ deliveredAt, reason, amountPkr, orderFound }) {
  const allowedReasons = new Set(["damaged", "wrong_item", "missing_parts", "changed_mind", "other"]);

  if (typeof orderFound !== "boolean") {
    return { state: "ESCALATE", reason: "INVALID_ORDER_STATUS" };
  }
  if (!orderFound) return { state: "ESCALATE", reason: "ORDER_NOT_FOUND" };
  if (!allowedReasons.has(reason)) {
    return { state: "ESCALATE", reason: "INVALID_RETURN_REASON" };
  }
  if (!Number.isFinite(amountPkr) || amountPkr < 0) {
    return { state: "ESCALATE", reason: "INVALID_AMOUNT" };
  }

  const now = Date.parse("2026-10-05T12:00:00.000Z"); // fixed evaluation time
  const delivered = typeof deliveredAt === "string" ? Date.parse(deliveredAt) : NaN;
  if (!Number.isFinite(delivered) || new Date(delivered).toISOString() !== deliveredAt || delivered > now) {
    return { state: "ESCALATE", reason: "INVALID_DELIVERY_DATE" };
  }
  if (now - delivered > 14 * 86_400_000) return { state: "REVIEW", reason: "OUTSIDE_POLICY_WINDOW" };
  if (reason === "changed_mind") return { state: "REVIEW", reason: "POLICY_CHECK" };
  if (reason === "other") return { state: "REVIEW", reason: "UNCLASSIFIED_REASON" };
  if (amountPkr > 20_000) return { state: "APPROVAL", reason: "HIGH_VALUE" };
  return { state: "ELIGIBLE", reason: "RULES_PASSED" };
}

console.log(routeReturn({
  deliveredAt: "2026-09-30T12:00:00.000Z",
  reason: "damaged",
  amountPkr: 7_500,
  orderFound: true,
}));

The thresholds here are fictional for the lab: the window is at most fourteen elapsed 24-hour periods, and timestamps use the exact UTC ISO form shown. They are not marketplace rules; a real seller must encode its approved current policy. A model can classify a free-text reason into a fixed enum, but the code should validate that enum and handle uncertainty. It should not be allowed to call issueRefund merely because the customer's message sounds convincing.

When might an agent become justified? Perhaps an exception investigator must look up shipment scans, ask a courier-status tool, retrieve a policy clause, identify contradictory records, and assemble an evidence packet. Even then, let the agent investigate, not transfer money. Give read-only tools by default and place a human approval gate before the side effect.

Tool design decides agent quality

Tools should do one understandable job, accept a strict schema, return structured results, and enforce permissions internally. Prefer get_order_status(order_id) over run_sql(query). Prefer draft_refund_case(...) over manage_customer(...). Tool names and descriptions help the model choose, but authentication and authorization must live in code.

Separate read tools from write tools. Log every call. Set timeouts. Return typed errors such as NOT_FOUND, TEMPORARY_FAILURE, and FORBIDDEN. Define whether a call is safe to retry. An agent cannot compensate for ambiguous tool contracts.

Failure cases to test

  • The agent repeats the same search because no maximum step count exists.

  • Two tools overlap, and the model chooses the risky write tool instead of the read tool.

  • A user message contains instructions to ignore policy and trigger a refund.

  • The model returns prose where the application expected structured arguments.

  • A tool times out after completing a side effect, and the agent retries it.

  • The agent reaches a plausible conclusion from incomplete evidence rather than escalating.

  • A multi-agent design adds handoffs but no measurable improvement over one controlled workflow.

  • A cheap-looking model choice increases retries and total cost per successful task.

Hands-on deliverable

Save the supplied code as returns.mjs and run it. Expect ELIGIBLE / RULES_PASSED for the fictional five-day-old damaged item at PKR 7500. This is a routing result, not permission to transfer money.

Change only amountPkr to 25000. Predict APPROVAL / HIGH_VALUE, then run it. Restore the amount and change orderFound to false: expect ESCALATE / ORDER_NOT_FOUND, regardless of the complaint's persuasive wording.

Now choose a design for this changed input: the reason arrives as "dabba khola to item toot gaya tha" instead of a dropdown enum. A reasonable design adds a bounded classification step proposing damaged, keeps uncertainty as other/review, and leaves order/age/amount rules in code. It does not need an agent to choose their sequence.

Write four lines mapping input validation, free-text interpretation, deterministic routing, and refund authorization to code, model, code, and authorized human/system respectively. If you propose an adaptive investigator later, name the uncertainty that requires choosing another tool and its stop condition.

Completion rubric

  • The original, high-value, and missing-order inputs produce the three expected routes.
  • The free-text change adds interpretation without moving arithmetic or permissions into the model.
  • The hypothetical investigator has a bounded purpose/tool budget, or you justify why it is unnecessary.
  • No eligibility label is presented as a refund execution or a real merchant policy.

Sources

Practice quiz

Check yourself

4 questions · practise what you just learned

0/4
  1. 1 / 4 · sequence

    The five-day-old damaged-item fixture has orderFound true. You change only amountPkr from 7,500 to 25,000. What route should the code return?

Apna kaam check karo.

Practice complete ki? Result ko example aur lesson ke checks se compare karo.

This is your own completion record, not a graded assessment.

Kuch samajh nahi aaya? Ask a question →