Key takeaways
- ✓The best model is the one that does your task reliably at a reasonable cost
- ✓Use different models for different steps, and test them on your own documents
- ✓Build so you can switch models without rebuilding the software
A lot of noise
New AI models appear every few weeks, each with benchmarks that show it beats the others. For developers that race is interesting. For a company that wants AI to read purchase orders or answer questions from project files, it is mostly noise.
The question is not which model is the smartest, but which model does your specific task reliably, fast enough and at a cost that makes sense. Those are different questions with different answers.
Large and small models
Large models are good at complex reasoning, long documents and tasks where the instructions are vague. They are slower and cost more per task. Small models are fast and cheap, and for well defined tasks, such as recognising a customer in a mail or pulling the order lines from a PDF, they are often just as good.
In a well built system, different steps use different models. A small model sorts incoming mail, a larger one drafts the difficult reply. That keeps the cost low without giving up quality where it matters.
Speed matters more than people expect. A colleague waits for an answer in a chat, while a mail that gets sorted in the background can take a few seconds longer. Matching the model to the moment keeps the experience fast where people notice it.
Open models and running them yourself
Next to the models you use through a provider, there are open models you can run on your own servers or in your own cloud environment. That gives more control over where data goes. It also means you carry the cost of the hardware and the maintenance, and the most capable models are usually not the open ones.
For most SMEs a hosted model with the right agreements is the sensible choice. Running a model yourself becomes interesting when data may not leave your environment at all, or when volumes are so high that it pays off.
Processing in Europe
The large cloud providers offer AI models with processing in European data centres, and there are European model providers as well. What matters is the setup: in which region the data is processed, whether it is stored, and whether it is used for training. That is decided by the provider, the contract and the settings, not by the name of the model.
What it costs
Model usage is paid per amount of text processed. For typical SME tasks, such as reading a mail or a document of a few pages, that comes down to cents or fractions of a cent per task. The real cost of an AI project is in the work around the model: connecting it to your systems, handling exceptions and building in approvals.
That is also why chasing the cheapest model rarely pays off. A model that is a bit more expensive but makes fewer mistakes saves more in correction time than it costs.
Test on your own work
Benchmarks are measured on general tasks. Your documents are not general. The only reliable way to choose is to test a few models on a set of your own mails and documents, including the difficult ones, and measure how often they get it right. That takes days, not months, and it settles discussions quickly.
Measure the things that matter to you: how often the answer is right, how often a person has to correct it, how long it takes and what it costs. Write those numbers down. They become the baseline for every later decision about models.
Models and your data
A question we hear often is whether a model learns from your data. With business agreements from the large providers, your data is not used to train their models, and it is processed under a data processing agreement. With free consumer tools that is not always the case. That difference matters far more than which model you pick.
Inside your own system, the model does not need to see everything. A well built system only sends the model what it needs for the task at hand, within the access rights of the person who asked. That limits what can go wrong, whatever model is behind it.
When to change models
Change models when a test on your own documents shows a clear gain in quality or cost, not because a new model is in the news. Keep a fixed set of test cases from your own work, so you can compare a new model against the current one in an afternoon. If it is better, switch. If not, you have lost nothing.
Do not get locked in
The model that is best today will not be best in a year. Build the software so the model is a part you can replace: the business logic, your data and your rules live in your own system, and the model is called through a layer that can point to another provider. Then a better or cheaper model is an update, not a new project.
Let the task decide, not the hype.
Tell us how the work runs today.
