How we kept LeadVector's inference costs predictable

LeadVector does account research with agents, and research is expensive if you let it be. Early on, cost scaled almost linearly with usage, which is the wrong shape for a product. This is a short account of the decisions that flattened that curve without hurting quality.

Route before you reach for the biggest model

Not every step needs the most capable model. We route by task: cheap models handle extraction and formatting, stronger models handle reasoning that actually needs them. Most of the volume runs on the cheaper path, and the expensive path is reserved for where it earns its cost.

Cache the work, not just the answer

Account research repeats. The same company gets looked at more than once, and much of the intermediate work can be reused. We cache at the level of research steps, not just final answers, so a second look at an account is mostly free.

Let evaluation catch regressions

Cost work is dangerous without evaluation, because it is easy to make something cheaper and quietly worse. Every routing or caching change runs against our evaluation set first, so we can see whether we saved money at the expense of quality.

What we would tell our past selves

1. 01Put cost controls in the first architecture decision, not the last.
2. 02Route by task; reserve the expensive model for reasoning that needs it.
3. 03Cache intermediate work, not only final outputs.
4. 04Gate every cost change behind evaluation.
5. 05Track cost per job so the trend is visible before it is a problem.