Project

PolicyTrace

A Document AI workflow for UK motor insurance PDFs, with structured extraction, provenance, conflict resolution, and human review.

PolicyTrace Document AI AI Engineering Production AI Human Review

Project tag All projects

PolicyTrace review dashboard showing highlighted PDF evidence beside a structured Golden Record.

Split-screen review: source PDF evidence on the left, typed Golden Record fields and review controls on the right.

The point is not extraction. The point is reviewable extraction.

PolicyTrace turns a messy insurance document pack into a typed Golden Record, then keeps the evidence trail close enough for a reviewer to inspect, verify, flag, or override individual fields.

Every source has a role

The Schedule, Certificate, Statement of Fact, and Policy Booklet are classified before extraction so the system can treat them differently.

Fields are merged with authority rules

The arbiter decides which document wins for each field and surfaces conflicts instead of silently hiding disagreement.

Evidence remains inspectable

Verbatim field citations and Docling geometry are used to connect extracted values back to PDF locations.

System Flow

From policy pack to Golden Record.

The model is one step in the workflow. Parsing, masking, validation, arbitration, provenance, and review are separate control points.

PDF Pack

Schedule, Certificate, Statement of Fact, and Policy Booklet enter as a document set.

Docling Parse

Text and layout are converted while preserving enough geometry for later field matching.

PII Mask

Presidio masks sensitive entities before model calls while dates remain available for extraction.

Classify

A fast classifier routes each document to the right extraction prompt and fallback path.

Extract

Groq, Instructor, and Pydantic produce typed partial records and verbatim field citations.

Arbitrate

The PolicyArbiter applies hierarchy-of-truth rules and records cross-document conflicts.

Provenance

Canonical values and source quotes are matched back to PDF text locations.

Review UI

FastAPI and React expose the Golden Record, source PDFs, highlights, and review state.

Engineering Decisions

What makes this more than PDF-to-JSON.

These are the reusable patterns the project demonstrates. They matter more than any single prompt.

arbiter.py A

Hierarchy of truth

The Schedule wins for vehicle, premium, excess, and driver-risk fields. The Certificate wins for legal driving permissions and class of use.

schema.py P

Typed Golden Record

Pydantic keeps downstream JSON predictable while optional fields allow partial per-document extraction before the final merge.

provenance.py E

Evidence matching

The LLM supplies verbatim citations. A separate matcher connects those phrases to Docling PDF geometry, reducing geometry hallucination risk.

privacy.py S

Selective masking

PII is masked before extraction, but date masking stays off because policy dates are core extraction data.

prompts.yaml V

Versioned prompts

Extraction prompts are configured and versioned outside code, so changes can be audited and improved without rewriting the pipeline.

api.py R

Review state

The API stores verify, reject, and override actions so human review becomes part of the workflow, not a screenshot afterthought.

Project Series

Core chapters first. Implementation notes second.

The first column is the PolicyTrace system design document: architecture, truth, evidence, conflict, and review. The second column is supporting implementation work: deployment, hardening, evaluation, and prompt design.

Core project chapters

Implementation notes

Evidence Trail

Canonical value, source quote, PDF location.

PolicyTrace separates the value used by the system from the phrase used to find the source. That matters when a date, amount, or name is normalized.

Field policy_header.period_of_cover.start_date

Canonical 2026-04-15T00:00:00

Citation 15/04/2026 at 00:00 hours

Location Certificate of Motor Insurance - Demo.pdf, page 1

Certificate of Motor Insurance

Policy number NBM-DEMO-0427

Insurer Northbridge Mutual Motor Insurance Ltd

Effective from 15/04/2026 at 00:00 hours

Expires 14/04/2027 at 23:59 hours

Registration ZX24 DEM

System Shape

A reference architecture for document-heavy AI workflows.

The same shape can later be reused for claims, compliance, onboarding, lending, audits, and other multi-document workflows.

Input control

PDF upload, file type checks, safe filenames, per-session storage, and page caps for long documents.

Model control

Classification and extraction are separated, with specialist prompts and structured validation retries.

Domain control

The arbiter applies insurance-specific source authority rules instead of trusting whichever model answer arrives last.

Review control

Reviewers see extracted fields, confidence, source snippets, PDF highlights, and override actions in one place.

Honest Limits

Useful demo, not a finished insurance production system.

The project is intentionally honest about the gap between a public demo and a controlled production workflow.

The public demo should use synthetic or properly redacted documents only.

Extraction is synchronous and can take roughly 30 to 90 seconds depending on document size and model latency.

Provenance is reviewer evidence, not a legal-grade guarantee.

Production use would need authentication, durable storage, audit logs, monitoring, retention policy, and a larger evaluation set.

Explore The Project

Try the workflow, then inspect how it is built.

The demo shows the review experience. The repository shows the architecture, source decisions, and implementation details behind it.

Try demo GitHub Architecture