📄 Advanced Document Retrieval System

A loan-file copilot for home-loan (mortgage) officers: upload a borrower's packet, get an automatic review of the application against its supporting documents, record a decision on every flag, and ask anything else with page-cited answers. Under the hood it is a multi-user RAG (Retrieval-Augmented Generation) application for PDF documents. Built with a React frontend and a FastAPI backend, featuring accounts, persistent per-document conversations, PDF extraction via AWS Textract or PyMuPDF, and hybrid sparse-dense search.

One account → many chats → one uploaded document per chat. Chats share the account's Pinecone namespace; each chat's vectors are identified by its chat id. Neon/Postgres owns identity, ownership and conversation history; Pinecone holds vectors.

---

🚀 Key Features

• Accounts & persistent chats: JWT auth over bcrypt, DB-backed login lockout, and conversations that survive a restart — including their source citations. A login lasts as long as the tab is open: tokens live in sessionStorage, so a reload keeps you signed in, closing the tab signs you out, and a new tab asks you to log in.
• Session rehydration: the in-process retriever is a pure cache. On a miss it rebuilds from Neon + Pinecone in ~11s instead of re-ingesting the document (~180s), by persisting the fitted BM25 encoder and recomputing centroids from the index.
• Queued ingestion: upload returns 202 and enqueues a job that a worker runs, with a pollable status — as its own process, or inside the API with RUN_WORKER_IN_PROCESS=1. A queued job is a row that outlives the API process, so a deploy or crash does not destroy work in flight.
• Pluggable Extraction: AWS Textract (TABLES + FORMS, default) or PyMuPDF (local, no-AI, text-layer only) via EXTRACT_METHOD. Contextual chunking attaches each chunk's document identity so entity-specific queries stay unambiguous.
• Hybrid Search Engine: A single Pinecone sparse-dense index holding gemini-embedding-2 embeddings (768-dim) alongside BM25 sparse vectors, fused by a tunable alpha (0.0 = pure keyword → 1.0 = pure semantic).
• Conversational follow-ups: a follow-up like "and when does it lock?" is rewritten as a standalone question before retrieval. When there is no history, the question is used as-is. History is read server-side from Neon.
• Answer Generation: gemini-3.5-flash with medium thinking. Google's API currently routes this model ID to Gemini 3.6 Flash. Answers and conversational query rewriting use this model; field extraction retains the separate flash-lite model. Gemini 3 requests use thinking levels instead of the legacy token budget and omit deprecated sampling settings.
• Two decision paths: page classification and boundary detection use TypeSafe Jev (closed-set choice and yes/no probability). Gemini flash-lite still handles per-document context lines and review field extraction; answers and query rewriting stay on flash.
• Automatic file review: once a packet is ingested, the application's claims (income, employer, balances, declarations, loan terms) are checked against the pay slips, bank statement, Loan Estimate and title report. Each check shows both sides' values with their pages, in a panel beside the chat (a drawer on narrow screens) so the officer can ask about a flag without leaving it. See Automatic file review.
• Use your own provider: Settings lets a user choose Gemini (API key) or Cloudflare Workers AI (account ID and API token). Credentials are validated, encrypted with Fernet using a key derived from JWT_SECRET, and saved per account in Postgres. The selected provider survives logout/login; Settings shows only the saved key's last four characters. Rotating JWT_SECRET requires saving credentials again. Question rewriting and answers use the selected provider without spending daily credits; uploads and retrieval embeddings still use the app's keys. Without own credentials, the app's daily cap applies.
• Flag decisions with an audit trail: every mismatch or review flag can be accepted (a note is required) or confirmed as an issue. Decisions are append-only and record who decided and when. The sidebar shows findings separately from reviewed progress; confirming an issue does not mark it fixed.
• Several PDFs per borrower: upload up to 20 files at once; they are merged in order into one PDF for the chat, then split into logical documents during ingestion.
• Grounded answers: source values retain their labels, document identity and page citations. Requested calculations and discrepancy explanations show their inputs and arithmetic. The prompt instructs the model not to mix borrowers, employers, pay periods or current/year-to-date earnings; these instructions reduce errors but do not guarantee correctness.
• Bounded document-aware retrieval: ordinary questions retrieve up to min(2 × search breadth, 24) hybrid candidates. One fetch of at most 36 IDs adds opening and nearby chunks from the top six hits' logical documents, with chat/document/type validation. Duplicate evidence is removed and whole chunks are packed into at most 18 chunks / 6,000 cl100k source-context tokens. This is structural linking, not verified entity relationships. Explicit whole-document summaries retain their separate full-text path.

---

🧾 Automatic file review

A loan officer's core check is stare and compare: the application holds the borrower's claims, the supporting documents are the evidence. The review does that comparison for every file, and the ingest worker runs it right after the document turns ready.

1. Classify and split the packet into documents (17 types, including Loan Application and Loan Estimate). Explicit Loan Estimate headings retain that type rather than being grouped under Lender Fee Sheet; older indexed chats need reprocessing to update their stored labels.
2. Extract a fixed set of fields per document type: first from Textract's FORMS label/value pairs, then one flash-lite call per document for anything missing and for lists (deposits, debits, liabilities, liens, easements).
3. Grounding guard: a model-extracted value is kept only if its exact text is found in that document, and its page is located in code, never taken from the model.
4. Compare with rules in plain Python (thresholds at the top of core/review.py), not with the LLM. Asking about any review flag follows the normal retrieval and LLM answer path; the model performs the answer's arithmetic from the evidence. An unexplained transfer remains a review item, not proof of undisclosed third-party funds or a false declaration.

Check	Compares
Income	stated monthly income vs pay slips (pay frequency from the slip or its period dates), 5% tolerance
Employer	application vs pay slip
Checking balance	stated vs statement ending balance, 10% tolerance
Payroll deposits	statement deposits vs pay-slip net pay
Large deposits	non-payroll deposits above 25% of monthly income vs the declaration
Loan amount, purchase price	application vs Loan Estimate
Stated debt payments	stated monthly payments vs statement debits
Liens, easements	title report; "payoff required" read from the report's own wording
Packet contents	Loan Application, Pay Slip and Bank Statement are noted when absent; comparisons needing absent documents are skipped, not called failed

Each check is mismatch, review, missing, match or info. Documents absent from the packet are informational, not a loan-completeness verdict. Selected files can be previewed before clicking Process document, without saving or ingesting them. The original uploaded/merged PDF is retained separately for an owner-only preview available during processing and afterward. Swipe left/right or use arrow buttons to view its pages; review evidence opens the cited page. Older chats uploaded before this feature have no original to show. A review that fails is stored as failed and never fails the upload: the document stays searchable. If a worker is killed mid-review, the job's lease brings it back and the review is rebuilt; a review left on running with no job able to finish it is marked failed ("re-upload to review") by the worker's periodic check, so the panel never spins forever.

Try it: samples/synthetic_borrower/whitfield_loan_packet.pdf is an 8-page synthetic packet with five logical documents and planted issues. whitfield_3doc_packet.pdf is a three-page alternative: loan application, loan estimate, then bank statement. answer_key.md covers the 8-page packet's review and 35 test questions. Regenerate both with python samples/synthetic_borrower/make_samples.py.

Citations and reliability

• Page numbers are 1-based everywhere a person or the model sees them. Pages are stored 0-based; citations, the answer prompt and the eval prompt add 1. Before this, answers cited one page early. Pages are pages of the merged file when several PDFs were uploaded.
• Declaration citations match the question and its answer together: the undisclosed-funds YES/NO must follow that declaration's wording. A bare NO elsewhere (such as Loan No.) is not accepted as its source.
• The grounding guard ignores table cell separators, so a value quoted from a table row (07/15 | DIRECT DEP … | +$2,845.31) still matches, and list values the model wraps as {"value": …} are unwrapped rather than dropped.
• With PyMuPDF, page-boundary detection reads table rows without their | separators: with them, the model called a pay slip and the next bank statement "the same document" every time.
• Provider calls retry 429s, 5xx and timeouts with full-jitter backoff behind a circuit breaker, and the worker stops claiming jobs while a provider is down (llm/llm_router.py, worker.py).

---

🏗️ Architecture

Six layers, read top to bottom. Each arrow is a hand-off between layers; the ingest and retrieval pipelines flow left-to-right within their own band and the bands stack one below the other, while the shared services (data, external AI) are reached once per layer rather than by every stage, so the flow stays legible. Observability is cross-cutting.

%%{init: {"flowchart": {"rankSpacing": 12, "nodeSpacing": 12, "padding": 4}, "themeVariables": {"fontSize": "12px"}}}%%
graph TD
    User(["👤 User"])

    subgraph CLIENT ["1 · Client layer — React / Vite"]
        Land["🛬 Landing page"]
        UI["💬 Chat UI · upload gate · live ingest stepper · streamed answers"]
    end

    subgraph APP ["2 · Application layer — FastAPI (main.py)"]
        Auth["🔐 Auth · bcrypt · access + refresh JWT · lockout · per-IP rate-limit"]
        REST["🗂️ Chat endpoints · POST /message → SSE · 202 async upload + polling"]
    end

    subgraph INGEST ["3 · Ingest pipeline — worker.py, from the job queue"]
        direction LR
        Ext["📄 Extract · Textract / PyMuPDF"] --> Split["🏷️ Classify + split · Jev"] --> Chunk["✂️ Chunk · tables atomic · contextual"] --> Emb["🧬 Embed · 768d"] --> Up["📤 BM25 fit + Pinecone upsert"]
    end

    subgraph DATA ["4 · Data layer — Neon Postgres + Pinecone"]
        PG[("🐘 Postgres · accounts · chats · messages · bm25_params")]
        Pine[("🌲 Pinecone · one namespace per user")]
    end

    subgraph QUERY ["5 · Retrieval + answer layer"]
        direction LR
        RW["📝 Rewrite follow-up → standalone"] --> Hyb["🔍 Hybrid query · α·dense + (1−α)·sparse"] --> Ans["🤖 gemini-2.5-flash · streamed, cited"]
    end

    subgraph EXT ["6 · External AI services"]
        Gem["☁️ Google Gemini · flash / flash-lite / embeddings"]
        Jev["☁️ TypeSafe Jev · page labels + boundaries"]
        Tex["☁️ AWS Textract · TABLES + FORMS"]
    end

    OBS["📈 Observability · Langfuse LLM traces"]

    User --> CLIENT
    CLIENT -->|HTTP + JWT| APP
    APP -->|identity · ownership| DATA
    APP -->|upload| INGEST
    INGEST -->|vectors + bm25_params| DATA
    DATA -->|hybrid search| QUERY
    INGEST -->|extract · classify · embed| EXT
    QUERY -->|rewrite · answer| EXT
    INGEST -.->|LLM traces| OBS
    QUERY -.->|LLM traces| OBS

Loading

---

🛠️ Setup & Execution

1. Requirements

• Node.js: For the React frontend.
• Python 3.10+: For the FastAPI backend.
• Neon (or any Postgres): Accounts, chat sessions and messages.
• Pinecone: Serverless index, dimension=768, metric=dotproduct.
• Google AI API key: gemini-embedding-2 embeddings and gemini-2.5-flash answers.
• AWS account (default extraction path): AWS Textract API (AmazonTextractFullAccess — or scoped to textract:AnalyzeDocument). Skip if you set EXTRACT_METHOD=pymupdf (local, no AWS calls).

2. Backend Setup

1. Navigate to the backend directory:

   cd backend

2. Install dependencies:

   pip install -r requirements.txt

3. Extraction Setup (AWS Textract)

PDF extraction runs on AWS Textract (TABLES + FORMS features), billed at $0.065 per page — about $3.25 for a 50-page packet. FORMS returns label/value pairs, which the file review uses before asking an LLM.

EXTRACT_METHOD=pymupdf reads the PDF's text layer locally, for free. It detects tables with PyMuPDF's find_tables() so they stay whole in chunking, but it cannot read scanned pages and returns no FORMS pairs. A PDF with no readable text on any page is rejected with a clear message; pages without text are listed in the file header ("No readable text on page 9. Not searched or reviewed.") instead of being indexed blank. Answers come from the Gemini API directly, so no self-hosted LLM server is required.

A. AWS

1. Create an IAM user with AmazonTextractFullAccess (or a scoped policy granting textract:AnalyzeDocument).
2. Generate an access key pair for that user.
3. Set the credentials in backend/.env (see section D).

B. Pinecone

Create a serverless index with dimension=768 and metric=dotproduct — dotproduct is required for sparse-dense hybrid queries, and the dimension must equal EMBED_DIM in llm/llm_router.py, which also drives the embedding call itself. The backend verifies both at startup and refuses to run on a mismatch, rather than failing minutes later at upsert.

C. Neon (Postgres)

Create a database and copy its pooled connection string (the -pooler host — PgBouncer multiplexes many client connections onto few backends, which a free-tier compute needs). Tables are prefixed drs_ so this schema can share a database with other projects.

Create them with either:

# from backend/
python -c "from db.database import engine, Base; import db.models; Base.metadata.create_all(engine)"
# ...or paste migrations.sql into the Neon SQL editor

D. Configure backend/.env

Copy backend/.env.example to backend/.env and fill in your credentials.

4. Running the Backend and the Ingest Worker

Two processes: the API serves requests, the worker runs ingestion.

# API
python -m uvicorn main:app --app-dir backend --port 8000

# Ingest worker — in a second terminal
python backend/worker.py

Or one process: set RUN_WORKER_IN_PROCESS=1 and only the API command is needed — the same worker runs on a background thread inside it, with the same queue, retries and fair ordering. On shutdown it stops claiming and gets 30s to finish; anything longer is picked up again through the job lease. What you give up is isolation: an API crash also pauses ingestion, and the two can't be scaled separately.

Uploads queue but never finish without the worker running. Ingestion is a row in drs_ingest_jobs, not a background task inside the API, so a deploy or crash of the API does not destroy work in flight — restarting the worker just returns its jobs to the queue. docker compose up starts both.

Run more than one worker if ingest is the bottleneck: the claim is a conditional UPDATE ... FOR UPDATE SKIP LOCKED, so a second worker takes different jobs rather than duplicating the first one's.

5. Running the Frontend

1. Navigate to the frontend directory:

   cd frontend

2. Install dependencies:

   npm install

3. Run Dev Server:

   npm run dev

4. Open http://localhost:5173, create an account, then start a chat and upload a PDF.

Point a build at a deployed backend with VITE_API_URL:

VITE_API_URL=https://your-backend.onrender.com npm run build

---

🔌 API

Chat and account endpoints require Authorization: Bearer <token>; signup, login, token refresh, / and /api/health do not.

Method	Path	Notes
POST	/api/auth/signup	→ { token, user_id, username }
POST	/api/auth/login	5 failures / 15 min locks the username
GET	/api/chats	Sidebar list, newest first
POST	/api/chats/new	Reuses an existing empty chat
GET	/api/chats/{id}	Chat + full message history, the file review, the latest decision per flag and the decision history
POST	/api/chats/{id}/document	202 — queues a job; the worker ingests it. One PDF or up to 20 under the same file field, merged in order
GET	/api/chats/{id}/status	Poll while processing
POST	/api/chats/{id}/message	Ask a question. Returns question_asked and question_searched so a rewritten follow-up is diagnosable
PATCH	/api/chats/{id}	Rename
DELETE	/api/chats/{id}	Drops the namespace and the rows. Flag decisions are kept (audit trail)
GET	/api/account/credits	{cap, used, remaining, provider, own_key} for today; questions using own credentials are not counted
GET	/api/account/api-keys	Selected provider and saved credential metadata (last four characters only)
PUT	/api/account/api-keys	{provider, key, account_id?} — validate, encrypt, save and select credentials
PUT	/api/account/provider	{provider: "FREE"|"GEMINI"|"CLOUDFLARE"} — select saved credentials or free credits
DELETE	/api/account/api-keys/{provider}	Remove gemini or cloudflare credentials; removing the active provider restores free credits
POST	/api/account/check-key	{provider: "GEMINI"|"CLOUDFLARE", key, account_id?} — validates own credentials; Cloudflare checks the answer model with a small completion
POST	/api/pdf/preview/pages/{page}	Authenticated selected-file preview: multipart file, PNG of a 1-based page with X-PDF-Pages; upload size limit applies, no persistence or ingestion
GET	/api/chats/{id}/pdf	Owner-only original PDF; unavailable for pre-feature uploads
GET	/api/chats/{id}/pdf/pages/{page}	Owner-only PNG of a 1-based page; includes X-PDF-Pages count
POST	/api/chats/{id}/review/decisions	{check_key, decision: accepted|confirmed, note}. Accept requires a note. 409 if the file has no finished review, 422 for an unknown flag

Request and response headers

Header	Direction	Notes
Idempotency-Key	request	Optional, on POST /document. Without it, the server derives a key from the chat id and uploaded PDF bytes, so retries of the same upload reuse its job. Supply a key to identify retries explicitly. The response carries duplicate: true when a job was reused.
X-Request-ID	both	Echoed on every response. Send one to have it used as the correlation id; otherwise the server generates it. The same id is stored on the job row and adopted by the worker, so one value follows an upload across both processes.
Retry-After	response	Sent with every 429, in seconds, derived from the limit that tripped.
X-Gemini-Key	request	Optional, on POST /message. Uses the user's Gemini key for rewrite and answer, without spending app credits.
X-Cloudflare-Account-ID, X-Cloudflare-API-Token	request	Optional pair, on POST /message. Uses the user's Cloudflare account for rewrite and answer, without spending app credits. Never send with X-Gemini-Key.

Chat lifecycle: awaiting_document → processing → ready | failed

A chat that belongs to another user returns 404, not 403 — a 403 would confirm the id exists.

---

📈 Performance & Evaluation

140-question evaluation

A 34-page mortgage packet with 22 logical documents and 142 chunks. Generator and judge: gemini-3.5-flash-lite. Hybrid retrieval: α=0.5; context metrics use six recorded evidence chunks per case.

Metric	Cases scored	Correct / passed	Rate	Mean score
Answer accuracy (assistant-reviewed)	120	109	90.8%	—
Ragas answer correctness	120	—	—	0.803
Context precision	120	—	—	0.758
Context recall	120	—	—	0.925
Faithfulness	120	—	—	0.962
PII response-boundary adherence	9	8	88.9%	—
Scope adherence	5	5	100%	—
Toxicity (non-toxic)	5	5	100%	—

Development-set results on a partially synthetic packet. PII checks assess responses, not cross-account isolation; one document-injection case is untested.

Answer-quality run cost: $0.174 generation + $0.827 judging = $1.001; safety judging excluded.

Gold dataset · Safety cases · Responses · Safety responses · Ragas scores · Safety judgments · Accuracy review · Evaluation summary

Latency and cost per query: 140 questions asked one at a time with gemini-3.5-flash-lite, excluding judging, HTTP transit and message saving.

Latency (s)				TTFT mean (s)	Cost per query ($)
p50	p95	p99	mean		p50	p95	p99	mean
4.5	7.4	26.9	5.4	5.2	0.0019	0.0034	0.0046	0.0021

Latency results (140 questions)

Where one query's time goes

"Who are the borrower and co-borrower on this application?", run 10 times with gemini-3.5-flash-lite. Average of runs 3–10:

Step	Time
Neon reads (chat, daily credits, history)	1.44s
Query rewrite (no history, skipped)	0.00s
Query embedding (Gemini)	0.53s
BM25 query encoding	0.00s
Pinecone hybrid search	0.26s
Linked-chunk fetch (Pinecone)	0.27s
Evidence bounding + prompt	0.00s
Generation (streamed)	1.63s
Total	4.16s

The first query took 11.4s while the Neon connection opened and the retriever loaded.

Load testing (k6)

# from backend/, with Render and any other worker on the same database stopped
python load_test_k6.py --render-suspended --levels 25,50,75,100,150

load_test_k6.py starts the API and three ingest workers locally (20 job slots each), creates test accounts, runs load_test_k6.js once per level (30s ramp, 180s hold), then deletes the accounts and everything they created. It refuses to start while real jobs are queued. Each virtual user logs in, creates a chat, uploads a PDF, polls until ingestion finishes, then lists chats, opens its chat and checks health every 5–10s. Workers replace ingestion with a 4s wait, so no LLM, embedding or Pinecone calls are made and no questions are asked. Thresholds: under 1% failed requests; p95 under 2s for health, 5s for chats, chat and status, 10s for upload.

Measured on a laptop against the remote Neon database with about 0.8 GB free RAM; these are not production capacity figures. Times in seconds, p50 / p95 / p99:

Users	Requests	Errors	Login			Chats			Upload			Upload → ready			Thresholds
			p50	p95	p99	p50	p95	p99	p50	p95	p99	p50	p95	p99
25	1,526	0%	3.8	5.0	5.1	1.2	1.6	3.3	4.5	4.6	5.0	6.9	9.6	9.7	met
50	3,017	0%	4.0	6.5	6.6	1.2	1.5	2.4	4.7	6.4	7.1	8.8	11.9	13.0	met
75	4,455	0%	4.5	5.7	7.1	1.2	2.3	2.6	7.0	9.4	9.9	9.9	26.8	29.0	met
100	5,566	0%	4.9	6.4	7.2	1.2	2.0	2.6	16.6	19.8	20.6	9.8	28.4	37.3	upload p95
150	5,897	0%	6.2	9.7	10.7	2.6	3.5	4.7	33.3	43.4	45.0	9.7	42.0	52.8	upload, status p95

All 400 uploads completed. Upload time is the limit above 75 users: each upload writes the PDF to the database twice (job payload and stored original) from a 12-thread pool, and queue waits grow once the 60 job slots fill. Full per-endpoint results are in backend/k6_results/.

---

📈 Observability

OpenTelemetry over OTLP exports provider-call spans to Langfuse when LANGFUSE_PUBLIC_KEY and LANGFUSE_SECRET_KEY are set. Each question has a chat-message trace with rewrite and answer calls nested beneath it, tagged with user and session. Evaluation scripts in backend/eval/ send one evaluation-case trace per model or prompt trial, with the question, answer, result and case id. Ingest calls run in raw thread pools and appear as standalone traces. Set LANGFUSE_HOST to select the Langfuse endpoint; OTEL_SERVICE_NAME is optional. Tracing failures are logged and never break a request.

Structured JSON logs with a correlation id remain available without tracing. Every request gets one (an inbound X-Request-ID wins), and it is stored on the job row. The API upload and worker ingest therefore share a request_id. Set LOG_FORMAT=text for local output. Cost and token usage are recorded per provider call in these logs, including ingest calls without a parent trace.

---

🐳 CI/CD & Docker

.github/workflows/ci.yml runs on every push and PR:

1. backend — ruff, compileall and core/review.py (file-review rules and grounding guard), all with no network.
2. frontend — npm ci, npm run lint, npm run build.
3. docker — build both images (no push), so a broken Dockerfile fails here, not at deploy.
4. deploy — only after all three pass, only on push to main: POSTs the Render deploy hooks (RENDER_DEPLOY_HOOK_* secrets), skipping gracefully if they're unset.

docker compose up --build runs the stack locally: three services — api and worker from the same backend/Dockerfile (python:3.12-slim, non-root, /api/health probe) with different commands, plus frontend/Dockerfile (Vite build → nginx). Extraction runs on AWS Textract, so the backend image needs no GPU/GL libraries.

Deploying needs the worker running somewhere. Either a second service — the same image with python worker.py as its command (a Render Background Worker, which needs no public URL; it talks to the API only through the database) — or RUN_WORKER_IN_PROCESS=1 on the API service, so one web service does both. Without either, uploads remain queued until a worker starts.

The backend runs with --forwarded-allow-ips * (in the Dockerfile CMD) so slowapi's per-IP rate limits key on the real client (X-Forwarded-For) behind a proxy/balancer rather than the proxy's own IP — otherwise every user shares one rate-limit bucket. On a non-Docker deploy, set FORWARDED_ALLOW_IPS=* in the service env instead (uvicorn reads it).

---

📂 Project Structure

document-retrieval-system/
├── backend/
│   ├── core/                        # Processing & retrieval engine
│   │   ├── document_store.py          # Orchestration + rehydrate()
│   │   ├── retriever.py               # Hybrid search, routing, namespaces
│   │   ├── pdf_processor.py           # AWS Textract integration (TABLES + FORMS)
│   │   ├── chunker.py                 # Structure-aware chunking
│   │   ├── document_classifier_jev.py # Active doc-type & boundary decisions (TypeSafe Jev)
│   │   ├── document_classifier_gemini.py # Previous Gemini implementation (inactive)
│   │   ├── query_rewriter.py          # Follow-up → standalone question
│   │   ├── answer_generator.py        # Grounded answer prompt
│   │   ├── review.py                  # Automatic file review: fields, grounding guard, rules
│   │   └── models.py                  # Core dataclasses
│   ├── db/                          # Neon / Postgres
│   │   ├── database.py                # Engine + session factory
│   │   └── models.py                  # accounts, chat_sessions (+ review), messages, review_decisions
│   ├── llm/
│   │   └── llm_router.py              # Gemini answers + embeddings
│   ├── eval/                        # Measurement harnesses
│   │   ├── loan_packet_questions.json # 120 questions with gold answers
│   │   ├── loan_packet_generate.py    # Six-context answer generation, batches of 25
│   │   ├── loan_packet_judge.py       # Four Ragas metrics using the same six contexts
│   │   ├── safety_questions.json      # 20 safety cases
│   │   ├── safety_generate.py         # Six-context safety responses
│   │   ├── safety_judge.py            # Response safety checks
│   │   ├── query_latency.py           # 140-question latency and cost
│   │   ├── model_sweep.py             # Generated questions, no judge (trust this)
│   │   ├── model_eval.py              # flash vs flash-lite, LLM-judged
│   │   └── prompt_eval.py             # Prompt-change A/B
│   ├── auth.py                      # JWT + bcrypt + refresh tokens
│   ├── observability.py             # OpenTelemetry → Langfuse
│   ├── main.py                      # API entry point
│   ├── worker.py                    # ingest worker — claims and runs queued jobs
│   ├── job_queue.py                 # the queue: enqueue / claim / finish / reclaim
│   ├── logging_setup.py             # JSON logs + correlation id
│   ├── load_test_k6.py              # local k6 runner: accounts, API, stub workers, cleanup
│   ├── load_test_k6.js              # k6 user flow: login, upload, poll, browse
│   ├── k6_results/                  # load-test results (JSON)
│   ├── Dockerfile                   # python:3.12-slim + uvicorn
│   ├── migrations.sql               # Schema + housekeeping queries
│   ├── requirements.txt
│   └── .env                         # Keys, DB URL, provider + queue settings
├── frontend/
│   ├── src/
│   │   ├── api.js                   # API client, owns the JWT
│   │   ├── Landing.jsx              # Animated landing page
│   │   ├── Login.jsx                # Login / signup
│   │   ├── ChatPanel.jsx            # Upload gate, live ingest stepper, messages, review panel
│   │   ├── App.jsx                  # Shell, auth gate, chat rail
│   │   └── App.css                  # Design tokens & styles
│   ├── Dockerfile                   # Vite build → nginx
│   └── nginx.conf                   # SPA routing
├── .github/workflows/ci.yml         # lint · build · docker · gated deploy
├── docker-compose.yml               # local api + frontend stack
├── ruff.toml
├── samples/synthetic_borrower/      # Synthetic loan packet + answer key for the review
├── results/
│   └── current/                     # Current 140-question quality, safety and latency results
└── README.md

---

📜 License

MIT License