AI that holds in production.

Because the failure is almost never where teams look. When a RAG system hallucinates, the instinct is to tune prompts or swap models, layers 4 and 5 of the stack. In my experience the root cause usually sits lower: documents chunked with no regard for meaning, embeddings that flatten domain language, no evaluation harness to catch drift. The fix is representation and retrieval work. Model the knowledge properly, measure retrieval quality on your own data, then tune what the numbers say is broken.

Different tools for different problems. RAG injects knowledge at question time, so it wins when facts change often, when answers must cite sources, or when data is private and per-customer. Fine-tuning changes how a model behaves, so it wins for tone, format discipline, and narrow repeated tasks. Most production systems I design use retrieval for knowledge and reserve fine-tuning for behavior, if it appears at all. Start with retrieval: it is cheaper to iterate, easier to evaluate, and every answer keeps an audit trail of where it came from.

No. Long context and retrieval solve different problems. A million-token window still cannot hold a corpus that changes daily, enforce per-user permissions, or tell you which document an answer came from. And stuffed context is priced per token, on every single call. Retrieval keeps the model grounded in a governed, current knowledge base at a fraction of the cost. What long context does change is chunking strategy: you can afford bigger, more coherent passages. That is an architecture decision, not the death of retrieval.

Measure first. Build a retrieval evaluation set from real user questions, score the system you have, and let the numbers pick the lever. The levers, in the order I usually reach for them: chunking that follows document structure instead of fixed sizes, hybrid retrieval that combines keyword and vector search, re-ranking on the candidate set, and richer representation, metadata and knowledge graphs, when entities and relationships matter. Most teams skip the measuring and tune blind. The measuring is the fix: each lever either earns its complexity on your data or it does not.

With an evaluation pipeline built before the application is finished, not after. I start from a golden set: real questions with expert-approved answers, including the ugly edge cases. Retrieval gets measured on its own, does the right context surface, before generation is judged, is the answer faithful to that context. On top of that: automated scoring for regressions, human review where judgment matters, and observability in production so every answer can be traced back through the pipeline. If a system cannot be measured, it does not ship.

Because a demo only has to work once, and production has to work every time. The pattern repeats in every failed PoC I have audited: the data layer was improvised, nothing was evaluated against a quality bar, nobody priced tokens at scale, and there was no observability for when answers go wrong. Closing the gap means treating the PoC as evidence, not as version one. Re-architect on solid data foundations, build the measuring in from day one, then scale deliberately. Mapping that path is exactly what my LLM production-readiness audit delivers.

It depends on the shape of the engagement, so I keep the shapes explicit. Advisory and fractional-CTO work runs on a monthly retainer sized by cadence. An architecture sprint or production-readiness audit is fixed-scope, with the price agreed before we start, so there is no meter running. Due diligence is a defined deliverable. What I do not do is open-ended hourly billing: it rewards slowness and you cannot budget around it. Call me, describe the problem, and you will leave the first conversation knowing the engagement shape and what it costs.