Pick an LLM provider you can walk away from
Notes on designing AI features for a regulated SaaS where auditability matters more than the leaderboard.
Models change every few months. Your compliance obligations don’t. If your AI architecture assumes one provider forever, you’ve built yourself a migration project.
Three things worth designing in from day one
- A thin model interface. Prompts, tools and evaluation live in your code, not in a vendor console.
- An audit trail. Every model call logged with input, output, model version and the decision it fed into.
- Evals you own. A fixed test set you run against any candidate model, so switching is a measurement, not a leap of faith.
Auditability is a feature
In KYC and AML, “the model said so” is not an explanation. Design for the person who will later have to reconstruct why a decision was made.