The technology exists, the possibilities are endless and stories about ChatGPT, Copilot and other AI tools are now reaching everyone. Yet many companies struggle to move from experimentation to practical application.
Machine learning and generative AI
Generative AI is artificial intelligence that can create new content based on existing data. Unlike traditional AI systems that recognize or classify patterns, generative models can write text, create images or generate code.
Large Language Models (LLMs) are at the heart of many generative AI applications. These systems are trained on massive amounts of text and can understand and produce human language. Think ChatGPT, Google Gemini or Microsoft Copilot. They "understand" context, can reason and formulate answers in natural language.
The main difference from traditional machine learning? With traditional models, you train a specific system for one task, such as predicting machine failures. With LLMs, you get a versatile system that can handle different tasks: answering questions, summarizing texts, writing code or analyzing documents.
Applications of generative AI
The power of generative AI becomes especially apparent when processing unstructured data. Whereas traditional systems struggle with variation in language and format, LLMs are actually stellar at understanding context and meaning.
- Knowledge management is an obvious application. Instead of spending 20 minutes searching for information about a specific machine setup, you can ask a question of a system trained on all of your company's technical manuals, reports and troubleshooting documents. The answer comes within seconds, complete with source citations so you can verify that the information is correct.
- In quality control , generative AI can analyze and summarize reports. An LLM can detect patterns in thousands of inspection reports, spot discrepancies and make recommendations. Not to replace human expertise, but to support it with faster analysis of large data volumes.
- Semantic search offers even more possibilities. Traditional search systems find only documents containing specific words. Semantic search understands meaning. If someone searches for "screen doesn't work," the system also finds documents about "no power" or "display stays black." It understands that these are related problems. For troubleshooting and knowledge sharing, this is a huge step forward.
- LLMs are also invaluable in software development . They can generate code, detect bugs and write documentation. Developers use them as intelligent assistants who are available 24/7 and provide immediate answers to technical questions.
Success of generative AI starts with the data
Be aware that generative AI works on the basis of "garbage in, magic out." If you want to deploy generative AI successfully, it starts with your data. And not just its quality, but especially its organization and accessibility.
For many organizations, internal chatbots are interesting. These answer questions about business processes (Retrieval Augmented Generation or RAG). For such a RAG to work properly, the system must have access to all relevant documents. But these are often scattered around: manuals on a SharePoint, procedures in different folders, specifications in old Excel files. Before you can train an LLM, you need to know where your data is and how to make it accessible.
It gets more complex with permissions. In a properly functioning RAG system, an employee should have access to all the information he or she is entitled to, but no more than that. For example, many financial documents are protected from most users and operators will not be able to see all HR documents. Implementing such a system forces you to rethink your entire information architecture.
Privacy and security are also important concerns. What data can go to external APIs from OpenAI or Google? What information should stay within your own systems? And how do you prevent sensitive business information from accidentally ending up in the wrong answer?
Implementing LLMs, what to watch out for?
Costs are often not too bad. An API call to ChatGPT or Google costs a few cents. Compared to the cost of hours spent searching for information, the direct cost of the LLM itself is negligible. Unless you make millions of calls a day.
Evaluation is trickier. How do you measure whether an LLM answers well? With traditional models you can calculate accuracy, with generative AI you often have to read and evaluate the output. Some companies have another LLM check the answers, others do sampling or have domain experts validate the results.
Scalability brings challenges. For 30 queries a day, you can manually check each answer. For thousands of questions a day, you have to think more statistically and accept that sometimes there are errors. The trick is to determine which errors are acceptable and which are not.
For critical applications, caution remains necessary. An error in a technical manual is annoying; an error in safety instructions can be dangerous. Classify your use cases by risk and adjust your controls accordingly.
Where do you start?
Start with a concrete use case, not the technology. Many companies think, "We have to do something with AI." Then problems often emerge that don't require an LLM at all. If you always extract the same information from fixed-format documents, pattern matching is faster and cheaper than an LLM.
LLMs shine in problems with unstructured data where you're looking for information you don't know exactly how it's described. Or don't know if it's there at all. Think of searching technical reports, analyzing complaints or finding similar problems in a database of failures.
Start small and iterate. Start with a limited data set and a specific query. Test with domain experts, adjust instructions based on their feedback, and gradually expand. The strength of LLMs lies precisely in this iterative approach: you can make adjustments quickly without retraining the entire system.
Generative AI offers enormous potential, but successful implementation requires more than just access to a language model. It starts with well-organized data, clear processes and realistic expectations. Companies that pay attention to these can make the most of the technology.
This blog post is a contribution from TMC, with their data science experts helping organizations improve their decision-making and predictive capabilities, for Data Expo readers. Find more inspiration at www.themembercompany.com or visit TMC during Data Expo at booth #5 & #40.
