Series
A UK motor policy pack in, one checked record out. Notes from building and running it, newest first.
A claims handler needs about a dozen facts from four or five PDFs before a motor claim can move. What PolicyTrace changes for that first step, what it can't do yet, and how an insurer could test it.
Run PolicyTrace in one click on Hugging Face, or locally in about eight minutes. Then check it the way a sceptic would, and break it on purpose.
Masking personal data before the model sees it isn't enough if the same container keeps the unmasked text on disk. How I found it, the fix, and what masking still misses.
Unit tests protect the code around a model. They can't tell you whether a new model still gets the answers right, and that's the change most likely to break things.
A highlight can quote the right words from the wrong person. A text-match score can't tell the difference; the matcher has to know whose row it is.
The model's job in PolicyTrace is narrow: read one masked document and fill in a form. Everything that decides whether you can trust the result happens after it.
My first run reported four conflicts between two policy documents. Two were double spaces, and two led to a real merge bug that a quiet merge would have hidden.
Reads a UK motor policy pack and gives a claims handler one checked record, with every value traced to the line in the PDF it came from.