RAG, Embeddings, Chunking, and Retrieval in Plain English

Retrieval-augmented generation (RAG) searches approved material and supplies it to a model before an answer. Retrieval supplies candidates, not proof. A keyword search can support RAG; embeddings are a useful option, not a requirement.

An embedding is a numerical representation used to compare patterns in content. Related meaning often has nearby representations. It is neither a fact store nor a confidence score. Keyword retrieval helps exact codes such as FORM-17B; hybrid retrieval combines keyword and semantic search.

A chunk is one retrievable unit. Keep a complete rule with its heading, scope, and exception. A short sentence without its exception can be dangerously incomplete; an entire handbook can bury the relevant rule. Generation turns the selected evidence into an answer, with citations and an option to abstain.

First filter by trusted user scope, authority, and effective dates. Search only eligible material, cap the candidate count, then inspect whether it supports the question. A similarity threshold must be chosen from labeled tests for your corpus; there is no universal score that proves an answer.

For managed retrieval, OpenAI vector stores can chunk, embed, and index uploaded files. Hosted storage, search, and model usage have their own access and cost requirements; none is needed here. File attributes/filters operate at the documented file level, so do not mix confidential and public sections in one uploaded file and assume a chunk label creates an access boundary. Separate scopes before indexing. Wait for ingestion to complete; provider deletion can be eventually consistent, so revoke application access immediately instead of assuming a delete request instantly removes every search hit.

Retrieval and answering solve different problems. Retrieval asks which passages are worth inspecting. Answering asks whether those passages establish the requested fact. In our fee corpus, both A3 and T3 discuss payments. T3 may look attractive for a query containing split because it mentions installments, yet its exception explicitly excludes admission fees. Better wording in the final prompt cannot recover an exception that was removed from the retrieved text.

An embedding turns text into a list of numbers called a vector. Think of it as a position used for comparison, not a compressed paragraph you can read back. One common comparison, cosine similarity, compares vector directions. To make the arithmetic visible, use invented two-dimensional vectors, not real model outputs: question Q=(1,0), passage X=(0.8,0.6), and passage Y=(0.6,0.8). Each has length 1 because, for example, 0.8² + 0.6² = 1. Their cosine scores against Q are 0.8 and 0.6. X ranks higher under this toy comparison, but neither number tells us whether X is current, permitted, or actually answers the question. Real embeddings have many more coordinates and must be evaluated on actual queries.

Return to the fee question and follow the intermediate states:

1. The request supplies an evaluation date and a trusted public-reader scope.
2. Eligibility removes A2 because it expired and M1 because it is private.
3. Search compares the question with A3, T3 and F1.
4. Evidence inspection identifies A3 as decisive; T3 addresses a different fee.
5. The answer uses A3's complete rule and names A3 as its citation.

Eligibility removes material the answer must not use. Search removes less relevant candidates to reduce reading work. Evidence inspection removes unsupported interpretations. Combining them into one vague instruction to “find the best answer” makes failures harder to diagnose.

Suppose an importer creates chunk T3a containing only Tuition may be paid in two installments. and T3b containing only This rule does not cover the admission fee. T3b lacks the subject of “this rule.” If search returns T3a alone, a model may extend installments to every fee. Repair the boundary by retaining the heading, both sentences and their shared source metadata. Adding unrelated pages would make the packet longer without repairing this missing relationship.

Try a second scenario: Can I split tuition and admission together into two installments? Predict the evidence packet before reading the answer. One source is insufficient: retain T3 for tuition and A3 for admission. The worked answer distinguishes the charges: tuition may use two installments under T3, while admission requires one payment under A3. It cannot calculate a combined schedule because the corpus supplies neither amounts nor such a schedule.

To diagnose a wrong result, save three observations: which eligible IDs were searched, which IDs were retrieved, and which claims the answer made. If A3 never reached the model, investigate retrieval. If A3 arrived intact but the answer contradicted it, investigate generation or validation. If A3 was excluded by a wrong date, repair eligibility first. This trace tells you where the next experiment belongs.