Grounding an eligibility assistant without giving the model the keys

Policy questions in public benefits are not a chatbot demo. SNAP, Medicaid, and TANF answers have to be explainable, current, and tied to Federal and State of Maryland source text. The E&E Eligibility Assistant was built around that constraint.

The diagram above is the product shape: ingestion → embeddings → FDE to build the corpus, then retrieve-and-generate at ask time, with MCP only when a live case or rule fact is required. The chat model does not get AWS keys, database credentials, or a whole case file in the default prompt.

Ingestion layer

Ingestion is how policy becomes processable text, not how the assistant answers.

Federal and Maryland sources (statute, regulation, agency manuals, curated extracts) land in object storage the same way other large files do — off the application servers. A processor picks up new or replaced objects, records a document version, and extracts text from PDF/HTML. Boilerplate, headers, and scanned junk get dropped so embeddings are not spent on page chrome.

Chunking is part of this layer. A benefits manual is too large for one vector and too large for one prompt. We split on section boundaries first, then fall back to token windows with overlap so a sentence that straddles a cut still retrieves. Every chunk keeps source identity from the start: program (SNAP / Medicaid / TANF), jurisdiction, document id, section heading, effective date. Ingestion does not call Claude or Nova. If you mix generation into ingest, you cannot tell whether a later citation is policy or model paraphrase.

Re-ingest is a first-class path. When a manual is superseded, the old version is marked not-for-retrieval; FDE then stops serving those chunks. That is how “current policy” stays a data problem instead of a prompt-engineering problem.

How embeddings work

An embedding is a numeric fingerprint of meaning. Cohere Embed on Amazon Bedrock turns each ingested chunk into a dense vector. Nearby vectors mean nearby language: “income disregard” and “what income can we ignore for SNAP” sit closer than either does to a TANF work-requirement paragraph.

Those vectors live in PostgreSQL with pgvector, next to the chunk text. At ask time the question is embedded with the same Cohere model. Switching models without re-embedding the corpus silently breaks retrieval. Similarity search returns the nearest passages; we then filter by program and access metadata so a Medicaid question does not get a SNAP-only paragraph just because the wording is close.

Embeddings are not the answer. They only rank candidates. We batch embed on ingest (idempotent on document version + chunk id) so the hot path is a query embedding plus an index lookup, not a crawl of S3. POCs on Claude and Nova were for generation, classification, and routing — not for replacing Cohere as the vector model.

How FDE is done

FDE (full-document enrichment) is the step that turns “a vector exists” into “this passage may be shown to a model and cited to a human.” A raw embedding without FDE is not served.

For each chunk, FDE writes the citation spine the UI and the prompt both need:

• parent document and version (so we cite the manual, not an orphan fragment)
• section / page or paragraph locator
• program and effective date
• ACL / role labels (applicant vs caseworker vs internal-only)
• hash of source text (so a quiet edit forces re-embed)

Parent and child stay linked: retrieval can return a tight chunk for the model and still display the parent title in the citation. FDE also rejects chunks that failed extraction (empty text, unknown program, missing effective date). Those stay in an ingest-error state, not in the k-nearest list.

Only after FDE succeeds does pgvector plus metadata become the retrieval record. That is why the assistant can return grounded answers with citations, and why thin retrieval is a product event (guardrail: do not invent policy) rather than a confident paragraph with no source.

Ask-time: still not giving the model the keys

When someone asks a question, Spring AI embeds it, retrieves FDE-approved passages, and instructs the Bedrock model to answer only from those passages. Live case context is a different channel: MCP tools call controlled E&E case-data and eligibility-rule services when the conversation actually needs a fact from the system of record. Existing E&E systems remain that record.

Latency, token use, retrieval hit rate, and citation ids go to Splunk. The useful lesson: RAG is a product over documents and APIs — ingestion, embeddings, and FDE are the corpus factory; the model is the last mile, not the keys to the kingdom.