Semantic caching, explained: when an AI can reuse an answer
A semantic cache reuses an AI answer when a new question is close in meaning to an old one. The real test is whether that answer is still right for this customer, today.
Topic
Model choice and cost
Small or large models, cost and latency: which model does the work, and what tells you when to call a bigger one.
I didn’t need the expensive model. I needed to know when to call it.
On a £100.01 refund, a cheap AI asked Finance for a ticket twice, and once just said the job was done. I tested two ways to tell when an answer needs a second opinion.