Contact-centre agent assist

The engagement

A UK clearing bank, four million commercial customers. Agents took calls with a second window open, hunting through procedure documents. The answer existed; finding it took ninety seconds; the customer sat on hold.

We built a retrieval assistant on Bedrock, wired into Amazon Connect. It surfaced answers to the agent, never directly to the customer. Average hold times fell by about a third.

Retrieval was younger then. A lot of what we did by instinct is now a discipline with a literature, an evaluation methodology, and a set of known failure modes. This is the 2026 version.

What I'd change

Then	Now	Why
Managed knowledge base, opaque retrieval	Hybrid: BM25 + dense, cross-encoder rerank	Bank procedures are full of near-identical language. Dense retrieval alone cheerfully confuses commercial overdraft policy with personal. Lexical matching catches what embeddings blur.
Fixed-size chunking	Section-aware, heading path prepended	A chunk saying "do X" without its parent condition "if the customer is a sole trader" is worse than no answer. It is a confidently wrong answer.
Trust the model to behave	Grounded generation with a refusal threshold	Below a retrieval-confidence floor, the system says "no confident answer." Coverage drops; trust doesn't.
Eyeball a few outputs	Golden set, recall@k, groundedness, CI gate	The question I could not answer well in 2023: how did you know it was working?

The refusal is the product

Worth stating plainly, because it inverts what most people optimise for.

In a consumer chatbot, a wrong answer is an annoyance. In a bank contact centre, an agent repeating a confidently-wrong statement about overdraft fees to a customer is a regulatory event — a mis-selling exposure, a complaint, potentially a redress exercise.

So the system is tuned for precision at the cost of coverage. If retrieval confidence is below threshold, it returns nothing and the agent does what they did before. A system that answers 70% of questions correctly and stays silent for the rest is worth vastly more than one that answers 95% with a 5% hallucination rate — and the second one is easier to build, which is why so many people build it.

The latency budget

Agent assist lives or dies here. If the answer lands after the agent has started improvising, the system is decoration.

Stage	Budget	Measured (p99)
Streaming STT	400 ms	___
Query embedding	30 ms	___
Hybrid retrieval	60 ms	___
Cross-encoder rerank	120 ms	___
First token	400 ms	___
End to end	~1.5 s	___

The rerank is the interesting trade. It measurably improves precision and it costs over a hundred milliseconds. Whether that's worth it is an empirical question, and I intend to answer it with numbers rather than taste.

Retrieval quality

Method: a synthetic corpus of banking procedure documents, and a hundred-question golden set with known-correct source passages. Ground truth by construction, so the metrics mean something.

Metric	Naive vector baseline	This build
Recall@10	___	___
MRR	___	___
Groundedness	___	___
Refusal precision	___	___
Hallucination rate	___	___

Cost

	Local (vLLM, quantised)	Frontier API
Cost per 1,000 calls	___	___
p99 latency	___	___
Quality delta	—	___

Decisions I'd defend in an architecture review

1. Hybrid retrieval, not pure vector. Near-duplicate procedural language defeats embeddings. ADR-001.
2. Section-aware chunking with heading path. A conditional clause without its condition is a liability. ADR-002.
3. Refuse below threshold. Precision over coverage, deliberately. ADR-003.
4. Every sentence cites a retrieved passage, or it isn't emitted. ADR-004.
5. No fine-tuning. Procedures change weekly; retraining is a treadmill, retrieval is a database write. ADR-005.

What I got wrong