Module 4 · Grounded Knowledge Systems · 45 min
Build a Cited Assistant That Can Abstain
Is lesson ke baad tum
The application must decide which evidence is eligible and when to abstain or escalate; a citation is useful only when its permitted excerpt supports the claim.
In this lesson
Build a small local answer gate over the fictional fee policies from Lesson 4.1. It returns a cited answer, abstains, or escalates. You will run it, introduce conflicting active evidence, and observe the changed state. Node is the only prerequisite; no model or paid service is called.
This is the deterministic skeleton of a cited assistant. Questions use fixed keys so you can test the boundary before adding natural-language retrieval or generation. It is not an embedding search or proof that a language model will always follow the rules.
Define the Three Outcomes
ANSWERED means eligible, consistent evidence supports this bounded lookup. ABSTAINED means no eligible evidence answers it. ESCALATED means an authorized person must resolve the requested exception, access issue, or conflict. The result includes only citations the same user may see.
Use verified server identity to select permitted sources. A browser field saying role: "manager" is not authentication. This lab hardcodes a public user; a real server must supply the identity and tenant/campus scope independently.
Worked Example
Save as cited-assistant.mjs and run node cited-assistant.mjs:
import assert from "node:assert/strict";
const sources = [
{ id: "A3", key: "admission", audience: "public", from: "2026-01-01", to: null,
rule: "one-payment", label: "Demo Fees v3 / Admission fee",
text: "Admission fees must be paid in one payment, including the evening program." },
{ id: "T3", key: "tuition", audience: "public", from: "2026-01-01", to: null,
rule: "two-installments", label: "Demo Fees v3 / Tuition",
text: "Tuition may be paid in two installments. This rule does not cover the admission fee." },
{ id: "A2", key: "admission", audience: "public", from: "2025-01-01", to: "2025-12-31",
rule: "two-installments", label: "Demo Fees v2 / Admission fee",
text: "Admission fees may be paid in two installments." },
{ id: "M1", key: "internal_exception", audience: "manager", from: "2026-01-01", to: null,
rule: "manager-review", label: "Internal exception procedure",
text: "Managers review exceptional fee arrangements through the internal process." }
];
function answer(key, actor, asOf, registry = sources) {
if (!actor || !["public", "manager"].includes(actor.role)) {
return { state: "ESCALATED", message: "Ask the policy owner for access.", citations: [] };
}
if (!/^\d{4}-\d{2}-\d{2}$/.test(asOf) ||
!Number.isFinite(Date.parse(asOf)) || new Date(asOf).toISOString().slice(0, 10) !== asOf) {
return { state: "ESCALATED", message: "The evaluation date needs correction.", citations: [] };
}
if (key === "approve_exception") {
return { state: "ESCALATED", message: "Only the policy owner may decide an exception.", citations: [] };
}
// Permission and date filters run before selecting candidates by question key.
const eligible = registry.filter(s =>
(s.audience === "public" || actor.role === "manager") &&
s.from <= asOf && (s.to === null || asOf <= s.to));
const hits = eligible.filter(s => s.key === key);
if (!hits.length) {
return { state: "ABSTAINED", message: "No approved source available for this question; ask the policy owner.", citations: [] };
}
if (new Set(hits.map(s => s.rule)).size > 1) {
return { state: "ESCALATED", message: "Active sources conflict; ask the policy owner.", citations: [] };
}
// The fixture answer copies a complete approved clause, rather than adding claims.
const source = hits[0];
return { state: "ANSWERED", answer: source.text, asOf,
citations: [{ id: source.id, label: source.label, excerpt: source.text }] };
}
const actor = { role: "public" }; // trusted test fixture, not a request parameter
const tests = [
["admission", "ANSWERED", "A3"],
["tuition", "ANSWERED", "T3"],
["scholarship", "ABSTAINED", null],
["approve_exception", "ESCALATED", null],
["internal_exception", "ABSTAINED", null]
];
for (const [key, state, citation] of tests) {
const result = answer(key, actor, "2026-10-05");
assert.equal(result.state, state);
assert.equal(result.citations[0]?.id ?? null, citation);
console.log(key, JSON.stringify(result));
}
console.log("Five local cases passed; no model request or approval was made.");The public internal_exception lookup abstains without exposing the confidential title. The missing scholarship clause also abstains. approve_exception escalates because evidence cannot make a discretionary decision. A citation shows the actual supporting excerpt, not merely a plausible filename.
Hands-On Exercise
Append one test below the loop. Make conflicting = structuredClone(sources), then push a copy of A3 with a new ID A4, rule: "two-installments", and text allowing admission installments. Keep its active date unchanged. Call answer("admission", actor, "2026-10-05", conflicting) and assert ESCALATED. Two incompatible active rules must stop the answer.
Now set A4's from to 2026-11-01. On October 5 it is not yet active, so the result should return to ANSWERED with A3. This repairs the test-date conflict only: on November 1 both remain active until the owner gives A3 an end date or defines explicit precedence. Do not silently pick the newest ID.
Finally change the original question's date to 2025-12-15. Expect A2 as its citation. Inspect the excerpt as well as the state; a correct state with the wrong source is still a defect.
Where the Full Assistant Adds Work
An actual retrieval system also needs a source registry, ingestion checks, permission-scoped retrieval, and tested freshness. Natural-language generation adds claim-support checks: a real file citation can still point to text that does not support the answer. Retrieved instructions are untrusted data; they cannot grant permissions or authorize actions.
If you later connect hosted file search, use authorized API access, protected keys, explicit model/timeout/output/call budgets, documented storage/search charges, and current file metadata filters. Keep confidential and public sources separate where the provider's filter boundary requires it. Validate and render citations server-side without leaking titles, links, or excerpts the user cannot access. The local fixture does not test those hosted controls.
Completion Rubric
- Five starter cases pass, including missing evidence, requested approval, and private-source exclusion.
- Active contradictory A4 causes escalation; its future date restores the October answer.
- The earlier date cites A2, with the matching excerpt.
- You can name the unimplemented components: authentication, natural-language retrieval, model generation, and production citation/access handling.
Sources
3 reference sources. Open the source to check the details.
Practice quiz
Check yourself
4 questions · practise what you just learned
1 / 4 · diagnose
On 2026-10-05, you add active A4 allowing admission installments while active A3 requires one payment. What should answer('admission', ...) return?
Apna kaam check karo.
Practice complete ki? Result ko example aur lesson ke checks se compare karo.
This is your own completion record, not a graded assessment.