Module 6 · Prepare the Capstone for Review · 60 min
Capstone: Test a Local Workflow and Prepare Its Evidence Pack
Is lesson ke baad tum
A credible capstone makes its local behavior reproducible and its evidence inspectable, while leaving unimplemented production controls visibly untested.
In this lesson
Bring the cited-assistant lab from Lesson 4.2 into one reviewable local package. The 60-minute guided session starts after the earlier lab works; installation, catching up on prerequisites, and a hosted deployment take additional time. Use the same fictional fee policies and fixed question keys. You do not need an account, API key, or paid request.
Your deliverable is a tested local prototype with an honest statement of what it proves. It is not a deployed language-model assistant, assessed professional credential, or production security approval. Course completion remains self-reported.
Define the Scope
Use this starter contract: “For a learner using the fictional demo fee policies, the local program returns an approved clause with its source ID, abstains when no eligible clause exists, and escalates a requested exception or active conflict.”
The fixed-key lookup does not accept arbitrary natural-language questions, authenticate real users, or issue approvals. Keep the full registry synthetic. A real app would need independent authentication, tenant/scope controls, ingestion, actual retrieval evaluation, safe generation, and operations review before publication.
Worked Example
Start with cited-assistant.mjs from Lesson 4.2 and its five starter assertions. Preserve the source text and expected citation IDs. Add the three checks already practiced there: incompatible active A4 → ESCALATED, future A4 → A3 on October 5, and date 2025-12-15 → A2. If you have not implemented those checks, mark them NOT RUN until you do.
Create review.json from your actual run. This starter deliberately blocks review because several items have not been checked:
{
"scope": "local-fixture",
"normal_citations": "NOT RUN",
"unsupported_and_private": "NOT RUN",
"exception_and_conflict": "NOT RUN",
"date_boundaries": "NOT RUN",
"synthetic_data_only": "NOT RUN",
"public_authentication": "NOT RUN",
"hosted_retrieval": "NOT RUN"
}The first four fields correspond to executable observations. synthetic_data_only requires inspecting the code, output, and included files. Set a field to PASS only after its specific check; use FAIL for an observed mismatch. Keep the final two fields NOT RUN for this local project.
Save this as review-gate.mjs:
import { readFile } from "node:fs/promises";
const report = JSON.parse(await readFile("review.json", "utf8"));
const required = ["normal_citations", "unsupported_and_private", "exception_and_conflict", "date_boundaries", "synthetic_data_only"];
if (report.scope !== "local-fixture") throw new Error("This gate covers local-fixture review only");
const blocked = required.filter(key => report[key] !== "PASS");
if (blocked.length) throw new Error(`Local review blocked: ${blocked.join(", ")}`);
console.log("Recorded local checks are complete; inspect their evidence before review.");
console.log("This report does not authorize deployment or establish production readiness.");Running the untouched report must fail. After actual checks and truthful updates, it can pass. This gate reads your recorded statuses; it cannot independently know whether you ran a test or judged a claim correctly. Keep the run output beside it so a reviewer can verify the entries.
Turn a Passing Demonstration Into an Explanation
An evidence pack should let another learner reconstruct why a result happened. Begin with the supported admission question: the input key identifies the question, the evaluation date selects the active policy, the public scope excludes private material, and A3 supplies the clause. Record that chain beside the observed result. If you save only PASS, a reader cannot tell whether the program returned A3, guessed the same words, or silently skipped the check.
Use an illustrative review row with separate columns for input, expected result, actual result and evidence location: admission / public / 2026-10-05 → A3 → NOT RUN → pending. Replace the last two fields only after execution. The expected result is your specification; the actual result is an observation. Keeping them separate prevents copying an attractive expected answer into the evidence column.
The eight checks cover different failure reasons. Normal questions test the intended lookup. Unsupported/private questions test whether absence is preserved. Exception/conflict questions test whether incompatible answers are escalated. Date checks test which version applies. Five local review fields summarize these observations, but those five labels are not five independent measurements of system quality. Count test cases and assertions accurately without converting them into an invented accuracy score.
Diagnose an Apparently Successful Review
Consider this fictional failure: all eight cases passed yesterday, then somebody changed the source registry. The saved output still says success, and review-gate.mjs still passes because its five strings remain PASS. The gate examines the report, not the relationship between the report and the current program. A hash is useful here as a fingerprint: matching hashes indicate the same file bytes; a different hash means the old run no longer describes that exact file. A hash does not prove the file is correct or identify its author.
Repair the trail by saving the code/source version with the command and output, rerunning after the change, and updating the report from the new run. Do not edit yesterday's terminal text. Preserve the old result as evidence for the old version. A reviewer can then compare the changed input with the changed behavior.
Predict this changed-condition scenario: the date moves to 2025-12-15, while the public admission question remains unchanged. Expected citation: A2, because A3 is not active yet. If the result remains A3, inspect the date filter before changing answer wording. If the program crashes because it assumes A3 always exists, inspect the changed-candidate branch. If it returns A2 but your assertion expects A3, the test expectation is wrong for this scenario. Those failures require different repairs.
For another diagnostic check, keep the current date and introduce the incompatible active A4 fixture already specified in Lesson 4.2. The useful result is escalation, not whichever policy appears first in an array. Array order does not establish which conflicting policy is authoritative. A future-dated A4 should not create the same conflict on October 5. Comparing these cases tests the interaction between validity dates and conflict detection.
Before handing over, ask a peer to locate one failing condition from the README alone. They should be able to run the gate with a required field set to FAIL, see it block, restore the recorded truthful result, and understand the reason. The packet teaches the behavior through evidence, not through the number of documents it contains.
Hands-On Exercise
Create one folder containing the program, review.json, the gate, saved test output, and a short README.md. The README needs only setup/run commands, the scope contract, the fixture source IDs, observed results, and current limitations.
- Run the five starter cases and three added checks. Save the command, date, file version/hash, and observed output; do not write invented accuracy percentages.
- Fill the five local report fields from that evidence and file inspection. Keep public authentication/hosted retrieval
NOT RUN. - Run the gate. Then change
date_boundariestoFAILand rerun: it must block. Restore the truthful result from your saved evidence. - Write a three-sentence demonstration: one supported query/citation, one safe refusal or conflict, and the boundary of the local prototype. Identify one next implementation experiment, such as tested natural-language retrieval, without claiming it has run.
If a test fails, preserve the failing observation and correct the implementation before marking it passed. If an earlier lab is missing, keep the record incomplete and return to that lesson. A folder of polished documents cannot replace the executable checks.
Optional: Preparing for a Hosted Build
A later hosted project needs authorized accounts and a budget, protected server credentials, real authentication and scope enforcement, source lifecycle controls, request/output/call limits, safe logs, useful alerts, version compatibility, and tested recovery. Reuse Lessons 5.2, 5.3, 6.1, and 6.2 to define those checks. Label real provider usage separately from local fixture behavior and include charges, latency, privacy controls, and actual failures when measured.
Publish no secrets or private records. Do not claim users, client results, earnings, production reliability, or a completed security assessment from this synthetic lab.
Completion Rubric
- The eight specific local checks have saved observations and the expected source IDs/states.
- The untouched report blocks; a truthful completed local report passes; changing a required field to FAIL blocks again.
- The package is reproducible from its README and contains only synthetic sources and safe output.
- The demonstration names what is implemented, what was checked, and what remains untested.
Sources
3 reference sources. Open the source to check the details.
Practice quiz
Check yourself
4 questions · practise what you just learned
1 / 4 · sequence
All eight local checks run successfully and the five required review fields are supported by saved evidence. What should public_authentication and hosted_retrieval say in this local project?
Apna kaam check karo.
Practice complete ki? Result ko example aur lesson ke checks se compare karo.
This is your own completion record, not a graded assessment.