A legal research assistant with machine-checked citations

This legal-research assistant can only cite law that actually exists. Every answer names a real article with a clickable source url, or it says plainly that it can’t. The chat is table stakes. The part worth writing about is the code that makes the model structurally unable to cite a law it can’t prove.

A lawyer needs the exact article by number, the one that holds up when a regulator reads it back. A summary won’t do, and a regulator won’t accept “the tool said so” either, so a research tool that can’t name the article and prove it exists has done nothing worth paying for. I built this over one jurisdiction’s legislation.

The rule the whole design rests on: the model never writes a citation. It emits a request to cite, and plain code verifies that request against data I control. Retrieve, then verify.

0 hallucinated citations reach the user.

A single fabricated citation ends the product

When a lawyer reads “Article 12 of Decree No. 4 of 2022” inside a confident paragraph, they act on it. If that article doesn’t exist, or its number is off by one, the tool has planted a landmine in someone’s advice.

Language models are fluent by design, and that fluency is the failure mode here: a wrong answer looks identical to a right one. Ask a raw Gemini call for “the article on early termination penalties” and it’ll happily produce “Article 34, Decree No. 9 of 2019”. Correct format, plausible number, total confidence, and no such article on the books.

In a regulated field the bar sits higher than “usually correct”. One fabricated reference, shown a single time, ends trust in the whole product.

The whole design rests on one opinion: you never let the model produce the citation. It reasons about the question, and it never gets to assert what the law says.

The model reasons, the database owns the words

I built the assistant as a Gemini agent on Google’s Agent Development Kit (ADK), driving the loop with Gemini 2.5 Flash for tool selection and a Pro pass for the final drafting. It never speaks freely about the law; it works through three tools, and only three:

# The agent never receives free text it can paraphrase as "the law".
# It gets tools. Each returns structured data rather than prose.
tools = [
    search_legislation,  # Vertex AI Search over the corpus, returns law ids + snippets
    open_article,        # exact (law_id, article_no) read from Postgres, verbatim text
    cite_law,            # emits a citation request, which runs the gate before render
]

Search runs on Vertex AI Search over the corpus. Every verbatim quote of legal text comes back from a Postgres 15 database through Prisma, keyed by (law_id, article_no), never from the model’s memory. The tools are exposed to the agent over MCP, so the same verified surface backs the chat UI and any downstream integration.

That separation is the whole lever: the model does the reasoning while the database owns the words. On top of it sits what I call the citation gate, three checks that every citation clears before it can render. None of them ask the model anything; they’re plain code run against retrieval output and the corpus.

Every answer cites a real article, or says it cannot.

Gate check 1: the allowlist drops invented laws

When the agent decides to cite, I don’t trust the law id it hands me. I intersect it with the exact set of law ids retrieval just returned for that question, and anything outside that set never came from the corpus, so it can’t be cited.

def enforce_allowlist(citations, retrieved_ids):
    # retrieved_ids: law ids Vertex AI Search returned for THIS query, this turn
    allowed = set(retrieved_ids)
    kept = []
    for c in citations:
        if c.law_id in allowed:
            kept.append(c)
        else:
            # invented, or pulled from training data. It never surfaced. Drop it.
            log.warning("dropped citation %s: not_in_retrieval", c.law_id)
    return kept

The set is scoped to the turn rather than the whole corpus. A law that exists but that this query didn’t retrieve still gets dropped, because a citation the model couldn’t reach through search is a citation it guessed at. And guessing is exactly what I’m removing.

Gate check 2: prove the article number or drop it

A real law doesn’t guarantee a real article. The model will attach a crisp “Article 47” to a law that stops at Article 40. So for every surviving citation I look up the precise (law_id, article_no) pair in the versioned corpus, and if the pair is absent, I keep the law reference and null the number rather than let a fabricated article stand.

async def verify_article(law_id, article_no):
    row = await db.article.find_first(where={
        "law_id": law_id,
        "number": normalize_article_no(article_no),  # see the war story below
        "status": "current",
    })
    if row is None:
        # law is real, this article number is not proven. Downgrade, do not invent.
        return Citation(law_id=law_id, article=None, downgraded=True)
    return Citation(law_id=law_id, article=row.number, source_url=row.source_url)

A citation is only as specific as what I can prove. A downgraded citation still points at a real law on the official portal, without naming an article it can’t find.

War story. Legislation numbering breaks the clean-integer assumption: the corpus holds “47”, “47 bis”, “47 ter”, and reissued articles, so when the model emitted “Article 47 bis” my lookup compared it as the integer 47, missed, and downgraded a real article for two days. The fix was normalize_article_no, which canonicalizes the suffix forms and matches on the stored string instead of a parsed int.

Gate check 3: broaden the jurisdiction, but flag it

The corpus doesn’t cover every regime equally, and passing one emirate’s law off as another’s is the classic silent error. When a filtered search matches nothing, it auto-broadens to a library-wide search and returns a broadened flag, which the model is forced to disclose.

def search_legislation(query, jurisdiction):
    hits = vertex_search(query, filter=f'jurisdiction = "{jurisdiction}"')
    if hits:
        return {"hits": hits, "broadened": False}
    # nothing under the asked jurisdiction. Fall back, but never pretend coverage.
    hits = vertex_search(query, filter=None)
    return {"hits": hits, "broadened": True}

A broadened: True result changes the answer contract. The model must state that its jurisdiction filter matched zero results, and that these are library-wide hits rather than a match for what was asked. Silent broadening is how you tell a lawyer in one emirate that a rule from the emirate next door applies to them. The flag makes that failure impossible to hide.

Three independent checks run before a citation renders.

SHA-256 at the article level keeps an answer reproducible after the law changes

Content is ingested only from official government legislation portals, parsed into laws and articles, and fingerprinted with SHA-256 at the article level. When a source republishes, the digest changes, a new version row is written, and the old one is retained. Nothing enters the answer path that didn’t come from an official text with a traceable version.

digest = hashlib.sha256(raw_text.encode("utf-8")).hexdigest()
existing = await db.article.find_unique(law_id=law_id, number=article_no)

if existing is None:
    await db.article.create(law_id, article_no, text=raw_text, sha256=digest, version=1)
elif existing.sha256 != digest:
    # official source republished the text. Version it, keep the prior one.
    await db.article.create(law_id, article_no, text=raw_text,
                            sha256=digest, version=existing.version + 1, status="current")
    await db.article.update(existing.id, status="superseded")
# identical digest -> no write. The corpus stays byte-stable across re-ingests.

Because the fingerprint is content-addressed, a re-ingest of an unchanged portal is a no-op, and a changed article becomes a new version instead of an in-place overwrite. Every citation the gate emits carries the source_url and the version of the exact bytes it quoted, so an answer stays reproducible even after the law is amended.

On top of that, the prompt rules forbid inventing law, decree, or article numbers, and force a clean split: “the law states”, which is sourced through the tools, versus “generally”, which is the model’s own knowledge, labelled as such.

The prompt is the weakest layer and I treat it that way: a courtesy to the model, nothing I’d rely on. The real control is the three gates.

Every legal quote is fingerprinted and versioned, source to answer.

What it refuses to do

Honest limit: the gate makes some answers less specific than a sharp human would give. When the article number won’t verify, I drop it and keep only the law, so a reader sometimes gets “this law governs the point” instead of “Article 47 governs it”. I chose a correct, coarser answer over a precise-looking guess. Coverage is the other edge. The assistant won’t answer for a regime the corpus doesn’t hold; it broadens the search, flags that it did, then defers rather than reach for a law it doesn’t have.

The cost of this stance is real. A confident competitor that skips the gate will look sharper on a demo, because it always names an article. It’s right up until the one time it’s wrong, and that one time is the whole liability. I’d rather ship the tool that occasionally says it can only confirm the law without the article than the one that’s impressive and wrong once a quarter.

Footnotes that resolve, in about two seconds

The assistant answers with footnotes that resolve to real laws on an official portal, at a p95 around two seconds for a single-part question. When it can’t verify something, it says so, whether that’s no matching legislation, an article number it won’t confirm, or a jurisdiction outside its coverage.

Ask about a jurisdiction outside the corpus and you’re told the coverage doesn’t extend there, never handed a confident paragraph quoting a law that governs somewhere else. An answer with a footnote can be checked in one click; an answer without one is telling you where its limits are.

Research time on a cited point is down about 60%.

Who has this problem

From the first line, I treat grounding as the shape of the system: can this claim be verified against a source I control? Retrieve, then verify, is the architecture, and the model is a component inside it.

If your team answers questions out of a body of rules, this is your problem wearing a different suit. A compliance officer who must cite chapter and verse to a regulator. Medical guidelines, tax rules, safety standards, financial disclosures: wherever a wrong-but-fluent answer carries real cost, the same pattern holds.

The recipe is the same everywhere: retrieve from a trusted, versioned corpus, let the model reason without ever letting it invent, intersect every citation with what search actually found, prove the specifics against a database, and make the system say plainly when it doesn’t know.