Part of the PolicyTrace series

Documents System note

Try PolicyTrace yourself: a step-by-step guide

Run PolicyTrace in one click on Hugging Face, or locally in about eight minutes. Then check it the way a sceptic would, and break it on purpose.

Teja Sagiraju · Updated · 4 min read
The short version
• Two ways to try PolicyTrace: the live demo in one click, or a local install I timed on a fresh clone (about eight minutes, mostly downloading).
• Use the synthetic demo pack, or documents you've made up. Never real policies.
• Then check it the way a sceptic would: the expected values, the highlights, the conflicts, the review file, and what happens when you break it.

PolicyTrace reads a UK car insurance policy pack and returns one record of the facts a claims handler needs, with every value linked to the line in the PDF it came from. This is the practical post: how to run it, what to check, and what to do when something goes wrong. I ran the local install on a fresh clone before writing this, so the commands and timings are real.

Option 1: the live demo, no install

  1. Open PolicyTrace on Hugging Face. If it has been idle, it can take a minute or two to wake up.
  2. Click Load demo pack. This loads four synthetic PDFs: a Schedule, a Certificate of Motor Insurance, a Statement of Fact and a policy booklet.
  3. Click Extract & review. It usually takes two to three minutes on Groq's free tier, mostly waiting on rate limits.
  4. You get the review screen: the source PDF on one side and the extracted record on the other.
PolicyTrace review screen: the source PDF on the left with a highlighted line, and the extracted record on the right with verify, override and flag buttons for each field
The review screen after a run on the demo pack.

Option 2: run it on your own machine

You'll need: Python 3.11 or newer (I tested 3.13), Node.js 20 or newer, git, a free Groq API key, and about 8 GB of free disk space. Most of that is PyTorch, which Docling, the PDF parser, needs for reading page layouts.

1. Get the code.

git clone https://github.com/AItoolstack/ai-policytrace.git
cd ai-policytrace

2. Create a virtual environment and install. This took about eight minutes on my run.

python -m venv .venv
# Windows PowerShell
.venv\Scripts\Activate.ps1
# macOS / Linux
source .venv/bin/activate

pip install -r requirements-dev.txt
python -m spacy download en_core_web_lg

The spaCy model is about 400 MB. It's what the masking step uses to find names; en_core_web_sm also works if you're short of space, with weaker name detection.

3. Add your Groq key. Copy the example file and put your key in it. The .env file is ignored by git, so it won't be committed.

# Windows PowerShell
Copy-Item .env.example .env
# macOS / Linux
cp .env.example .env

# then edit .env:
GROQ_API_KEY="your_key_here"

4. Run the tests. They don't call any model, so they take a few seconds. You should see 32 passed.

pytest -q

5. Start the API.

uvicorn api:app --app-dir src --port 8000

Open http://localhost:8000/api/health. You should see {"status":"ok","version":"1.0.0"}.

6. Start the review screen in a second terminal.

cd ui
npm ci
npm run dev

Open http://localhost:5173, click Load demo pack, then Extract & review. The first extraction also downloads Docling's layout model, so it's slower than the rest.

Other ways to run it. docker build -t policytrace . then docker run --rm -p 7860:7860 --env-file .env policytrace builds the same image as the live demo and serves it at http://localhost:7860. If you only want the JSON, there's a command-line mode:

python src/main.py --input sample_data/policytrace_demo_pack --output output/golden_record.json

What to check

  1. The basics are right. manifest.json, in the demo pack folder, lists the expected policy number (NBM-DEMO-0427), registration (ZX24 DEM) and insurer. Check the record agrees.
  2. Click through the evidence. Click a few fields and see where the highlight lands. Most land on the value, and driver fields should land on their own row in the driver table; here's why that once went wrong. Look for fields marked "no source location · check manually" too.
  3. Read the conflicts. The conflict panel lists every field where the Schedule and the Certificate disagree, with both values and which one won. On the demo pack you should see none. Step 7 shows how to make a real one.
  4. Act as the reviewer. Verify one field, override another with a new value and flag a third. Then open review_state.json in the session folder under output/sessions/ and see exactly what was stored.
  5. Break it on purpose. Upload only the Certificate, or only the policy booklet, and see how the record and the logs change when a document is missing.
  6. See what was masked. Set debug: enabled: true in config/settings.yaml, restart the API and run again. The session folder gets a debug/ folder with the text before and after masking. It's off by default because that folder holds the unmasked text; here's why that matters.
  7. Make your own pack. Edit the POLICY values at the top of generate_synthetic_policy_pack.py in the scripts folder, for example an excess or a driver's date of birth, and run the script. It overwrites the demo pack (git can restore it). Run the extraction again and check the record follows your changes.

If something goes wrong

  • "Internal Server Error" straight after Extract. Look at the terminal running the API. "No Schedule or Certificate extracted. Check GROQ_API_KEY" usually means the key is missing or wrong in .env. Restart the API after fixing it.
  • A rate-limit error from Groq. The free tier has per-minute token limits. Wait a minute and try again, or upload fewer documents.
  • PowerShell won't run Activate.ps1. Use .venv\Scripts\activate.bat from Command Prompt instead.
  • The screen loads but nothing happens. Check the API is still running on port 8000; the UI on port 5173 sends its requests there.
  • Anything else. Open an issue on GitHub with the command you ran and the error, so the answer is public for the next person.

Limits

The demo pack is synthetic and one layout. A clean run on it shows the pipeline works, not that it would work on your insurer's documents. And timings depend on Groq's free tier on the day.

Go further
Code: PolicyTrace on GitHub
Demo: PolicyTrace on Hugging Face
Related: Why PolicyTrace exists: the claims problem · What happens to a PDF inside PolicyTrace

If you ran it, what broke first? Tell me in the comments.

First published 17 Oct 2026. Updated 25 Oct 2026: run time and spaCy notes now match the current code; a screenshot of the review screen added.

Written by Teja Sagiraju

AI engineer with 14 years across machine learning and applied AI: prompt design, document AI, LLM workflows and the review loops that make them safe to ship. I build real systems and write up what breaks along the way.

Connect

Get the next note by email

One short email when something ships or breaks. Free.