"When an agent gives a customer the wrong price, it's not a retrieval miss. It's a compliance problem."
https://www.loom.com/share/02ca5ddc2bd94152805ce8973787d1bb
The knowledge layer for AutoCLM, an AI-native customer lifecycle management product built for telcos. Every agent conversation about plans, pricing, and offers ran through this system before any response reached a customer.
The job wasn't just retrieval. It was verified retrieval. There's a difference.
Built at Boring Workflows during a 4-month internship.
LLMs hallucinate on structured data. Prices, device specs, offer terms, plan conditions. Exactly the things a telecom customer asks about. And in a regulated industry, a confidently wrong answer is a compliance problem, not just a UX problem.
Standard RAG doesn't fully solve this. It has two failure modes that kept coming up:
Vocabulary mismatch. A customer asks about a plan in their own words. The document that holds the right answer was written by a marketing team with different vocabulary. Dense retrieval misses it semantically. Sparse retrieval misses it the other way. Neither alone is reliable enough.
Stale content reaching live agents. Marketing updates a plan document. Compliance hasn't reviewed it yet. If retrieval doesn't know the difference, an agent quotes the old price to a customer. That's the kind of thing that gets escalated.
Both were unacceptable. So both needed a structural fix, not just a configuration tweak.
Every query runs two searches in parallel. Dense vector search through Qdrant handles semantic similarity. BM25 sparse search handles keyword matching. Results from both are fused using Reciprocal Rank Fusion, which ranks candidates by their combined position across both result lists rather than raw score.
The point isn't that hybrid is always better. It's that dense and sparse fail in opposite directions, and RRF smooths out the cases where one method misses what the other catches.