
The story that should outlive this session is about a question that worked perfectly and cost 1.3 million tokens. The answer was good, it passed the team's question-and-answer checks and acceptance testing, and the tool calls were sensible — yet a single query consumed enough to have destroyed throughput at production scale. That is a category of defect traditional software mostly does not have, because the cost is incurred inside a reasoning process with no fixed shape. The team extended their agent framework for per-turn reasoning visibility and then bounded how much data the agent could pull, a sequence that matters because the guardrail could not have been written before the behaviour was observed. Their architecture separates agent logic from data logic so each side scales independently, and their closing advice is to find the simplest pattern that delivers the benefit and analyse with real data afterwards.
