To be honest, I often wonder how many people actually test these models themselves. Many reports are based primarily on benchmarks or figures provided by the developers. As CTO of an AI portal, I’m fortunate to have the opportunity to try out new models myself. For example, at Localign, we had various models take the Dutch HAVO final exams, using the same exams and official answer keys. This revealed clear differences between subjects, models, and costs.
Some models are demonstrably better at their core. They reason more consistently, follow instructions more accurately, or process more information. However, their performance in an AI portal is determined at least as much by the context they’re given. What information does the model receive? What instructions are provided? What tools can it use?
A good model, therefore, does not necessarily make for a good AI assistant.
An AI interface is much more than just a model
This becomes clear when you use a model directly via an API. You send a question and receive an answer, but you don’t automatically get the same experience as you would within the ChatGPT or Claude interface. There’s a complete agentic layer surrounding such products. It determines how much context is included, whether a search should be performed on the internet, which documents are relevant, which tools are available, and which model best suits the task. This layer can also verify which data may be used and which actions require additional consent.
As a result, two AI portals can use the same underlying model and still perform very differently. The difference lies not only in the model itself, but primarily in how it is controlled. An API connection to a language model can be built relatively quickly these days. Developing an AI portal that understands what someone is actually trying to achieve, gathers the right information, and acts on it reliably is much more complicated. This requires an agentic layer between the user, models, information, and possible actions.
From a Single Question to Multiple Actions
Consider a task that seems simple at first glance:
“Summarize the key developments regarding Project X and draft an email to the team.”
A simple chatbot sends this task directly to a model. A good AI assistant must first determine which project is being referred to, which documents and emails are relevant, and whether the user has access to that information. The AI portal must also determine whether the data may be processed within the chosen workflow and which steps need to be traceable afterward.
Only when sufficient context is available can the assistant draft a reliable summary and an email. A single question thus leads to multiple actions: interpreting, searching, retrieving, verifying, and writing. Some steps depend on previous results, while others can be performed simultaneously.
The model remains important, but within such a process, it is only one component of a broader architecture. The agentic layer determines which steps are necessary, what information the model receives, and when enough is known to provide an answer.
At Localign, we’re building precisely that layer. A query isn’t automatically sent directly to a single model. The AI portal determines which models, sources, instructions, checks, and actions are needed. That’s exactly where the toughest design choices arise for me as CTO.
The Daily Trade-off: Speed, Cost, and Quality
My biggest challenge is constantly balancing response time, cost, and quality, within fixed constraints regarding privacy, security, and accountability. The major players have set clear expectations: users want to see an assistant get to work almost immediately and receive a usable result quickly. Localign must also meet that standard. A meticulous system that remains idle for too long quickly feels like it isn’t working.
However, speed isn’t simply a matter of choosing a faster model. To provide an answer, the AI portal must first classify the question, retrieve instructions, consult sources, verify permissions, invoke tools, and combine results. Each step costs time and money.
A concrete example is the way you give instructions to an assistant. You can create a single, comprehensive system prompt that specifies how it should write emails, create charts, structure presentations, and analyze documents. That way, all instructions are always available.
But for a simple question about a document, the rules for PowerPoint aren’t necessary. Yet the model still has to process them every time. That increases the prompt size, raises costs, and can slow down processing.
You can solve this with a router that first determines which skills are needed. Only the relevant instructions are then loaded. This keeps prompts smaller and more targeted. But the router itself is an additional decision step, with extra latency and costs. Moreover, it might miss a skill that turns out to be necessary later on.
The same trade-off arises when consulting sources. You can first determine exactly which sources are needed and then search them. This prevents waste, but takes time. You can also start multiple likely avenues in parallel. For example, while the system searches documents, it can already retrieve emails.
That can be much faster, but you may end up paying for actions that later turn out to be unnecessary. You’re essentially buying speed by opening multiple doors at once before you know which one holds the right information.
The system can even start formulating conclusions before all the results are in. If the initial information confirms the suspected direction, this saves time. If later information changes the conclusion, part of the work must be redone.
Virtually every optimization shifts the problem. Smaller prompts are cheaper, but may lack sufficient context. Working in parallel reduces wait time but increases the risk of unnecessary actions. Determining exactly what is needed first prevents waste but, in turn, adds latency.
Not all latency should be eliminated through optimization. Checks on authorizations and personal data, recording the sources used, and requesting permission for certain actions take time, but are necessary conditions for responsible use.
Ultimately, quality remains the deciding factor. A fast and inexpensive assistant is of little value if it uses the wrong source or draws conclusions too soon. Conversely, a theoretically perfect answer that takes a minute to appear is equally unusable for many everyday tasks.
Not every question deserves the same process
The solution, therefore, is not to route every question through the same extensive process. A request to rewrite a single sentence should be handled almost immediately. This requires no searches, specialized tools, or extensive checks.
A request to compare multiple confidential documents and derive a well-founded recommendation from them, on the other hand, does justify more context, additional checks, and a longer processing time. Furthermore, as soon as an assistant also performs actions—such as sending an email or modifying data—authorizations, consent, and logging come into play.
At Localign, we therefore strive to tailor an appropriate workflow for each assignment. Simple questions should remain simple. Complex assignments may require more work, but only when that is likely to contribute to the outcome.
Sometimes a small, fast model can determine which workflow is needed, after which a more powerful model performs the analysis. For another query, that intermediate step is unnecessary. Some actions can be safely initiated in parallel, while for other tasks it’s wiser to first gain more certainty.
The ideal path is therefore not predetermined. The AI portal must determine during execution which intelligence, information, and controls are actually needed. For me, that is the essence of a good AI assistant: not just in what a model can do, but in the choices the agentic layer makes regarding context, tools, and timing.
Look Beyond the Model’s Name
For organizations selecting an AI portal, the question of which model is used remains relevant. But just as important is how the portal determines what information is needed, what tools are available, how authorizations are respected, and what happens if a model or data source is unavailable.
The underlying model must also remain replaceable. New models will continue to emerge, and the model that tops a benchmark today may be surpassed in just a few months. That’s why we’re building Localign to be model-independent and European. Not because every European model is automatically better, but because organizations need to be able to maintain control over the entire chain: from infrastructure and data processing to routing and actions.
The next phase of AI will not be won solely by whoever develops the smartest model. It will be won by whoever can reliably connect models with the right information, authorizations, and daily tasks.
A good AI model can formulate a convincing answer. A good AI assistant first understands what actually needs to be done.
This blog post is a contribution from Localign. Localign is a European AI platform that enables organizations to use various AI models securely and in compliance with regulations from a single environment, while retaining control over their data. For more information, visit www.localign.com or stop by the Localign booth at Data Expo.
