Can KM improve the quality of AI responses in the enterprise?

More and more companies are experimenting with artificial intelligence tools connected to their internal knowledge bases: chatbots to answer employees’ questions, Copilot, Gemini, Mistral or ChatGPT to produce summaries on specific topics…

The results are often disappointing. Errors creep into answers (hallucinations), content is forgotten (silences), relationships such as “X is a customer” are ignored.

How can knowledge management methods and tools help improve results?

KM to develop expertise

KM (knowledge management) encompasses the people, culture, processes, technologies and governance needed to capture, manage, share and disseminate knowledge.

A Knowledge Management System in my Company – What for?

COVID-19 has had a significant impact on the loss of collective knowledge in organizations. Telecommuting has reduced informal exchanges in the corridors. The ensuing resignation of experts in particular has led to the disappearance of critical knowledge.

KM’s mission is to capture people’s knowledge, make it explicit in knowledge objects and link it together, so that it can be not only found, but also understood and exploited.

For example, someone looking for answers to a question can:

• Via the search engine, access a list of documents and names of experts in the field.
• Via one of the experts, access precise answers and join a community of practice on his or her advice.
• Via one of the messages posted on the community forum, follow a training module recommended by one of the members.
• Via the training module, access a reference article on the company’s internal encyclopedia.
• Via the article on the internal encyclopedia, access a reference document and video selected by the article’s author.

In this way, you can move from one knowledge object to another, explore collective knowledge at your own pace, and choose the ways to develop your expertise.

KM makes knowledge objects usable by humans and machines

To be exploitable, these knowledge objects must have undergone a KM process:

• Capture and explicitation: The knowledge in the experts’ heads is made explicit in various forms: written documents, video… remarkable deliverables are captured. These knowledge objects are formatted according to templates (standard document templates…), titles are explicit and standardized…, this to facilitate exploitation by humans (identification of subject, understanding of content…).
• Validation: Knowledge objects are validated by the organization (peers in a community, an expert…), so that future readers can have confidence in their veracity.
• Capitalisation: Knowledge objects are stored in structured form in knowledge bases. Metadata is added (type of equipment, field of application, technology used…), to facilitate exploitation by machines (indexing, search, AI…).
• Dissemination: Knowledge objects are made available through an omnichannel strategy (website, tablet/phone application, paper…), but also via search engines, generative AI applications…
• Update: Knowledge objects are regularly re-evaluated by their owner (community of practice, team…), to ensure they remain relevant.

It’s impossible to capture everything. It would be too costly and unnecessary. We therefore identify the critical knowledge within the organization on which to concentrate our efforts to capture and make explicit.

Definition o critical knowledge: (1) The loss or inadequate maintenance of this knowledge entails a risk of the organization’s inability to carry out its mission, and/or jeopardizes the safety of its facilities (usefulness) AND (2) This knowledge is not available on the market, and its mastery is obtained through practice over a long period of time (uniqueness) AND (3) There is a real risk of loss (vulnerability), a risk that needs to be re-evaluated regularly.

Knowledge objects are rarely stored in a single location; they are generally distributed across several internal knowledge bases, depending on the needs and functionalities requested by users: EDMS, KM portal, corporate wiki-encyclopedia, LMS, file servers, SharePoint sites, Teams, DAM, records management systems…

Generative AI in the enterprise doesn’t work very well

AI in the enterprise, and in particular generative AI, represents enormous potential for companies. For the past two years, their employees have been using generative AI tools for simple tasks: summarizing a document or a meeting, helping to write an e-mail, producing PowerPoint from a document…

Companies are experimenting with generative AI tools connected to their internal knowledge bases: chatbots to answer employees’ questions, Copilot, Gemini, Mistral or ChatGPT to produce summaries or presentations on business themes…

They realize, however, that what works well on the Internet – ChatGPT, Copilot, etc. – doesn’t work as well inside their organization. There are two reasons for this:

• They haven’t built data models specific to their business.
• The internal content ingested by these tools has not been as well validated and filtered as that from Internet sources. Indeed, tapping into reputable newspaper websites is far more reliable than tapping into the bulk of file servers and SharePoint sites.

The results are often disappointing. Errors creep into responses (hallucinations), important information is overlooked (silences), relationships such as “X is a customer” are ignored.

These errors are actually erroneous deductions, meaning that the model is wrong, has ingested incorrect information (incorrect source material) or has failed to connect concepts.

Companies can’t afford these mistakes. What would happen if an employee made a decision based on an AI that had hallucinated or forgotten a major piece of information?

KM to help AI make fewer mistakes

Knowledge management (KM) methods and tools help artificial intelligence (AI) in a number of ways:

• Taxonomy: Giving AI meaning and context by defining business terms and creating hierarchies, helping AI to understand and infer the correct information.
• Ontology: Map relationships within the organization, enabling AI to make inferences about expertise and knowledge based on these relationships.
• Content types: Standardize the format of knowledge, making it easier for AI to process and understand information.
• Governance: Guarantee the reliability and veracity of content used by AI, preventing the dissemination of obsolete or incorrect information.
• Knowledge capture: Facilitate the acquisition of tacit knowledge from experts, which can then be used to improve AI models and their accuracy.

Some examples of how KM can help AI:

• Taxonomy example: An organization uses a taxonomy to define and organize business terms. This helps AI to understand the context of terms such as “Oasis”, distinguishing between a brand of drink, a rock band, a stretch of water in a desert or a place of rest.
• Example of ontology: By mapping relations such as “Louis-Pierre works at Amallte” and “Amallte is a consultancy expert in knowledge management”, the AI can deduce that Louis-Pierre has expertise in knowledge management.
• Example of content types: Standardizing content formats, for example by placing summaries and keywords in a specific section, enables AI to process and prioritize information more efficiently, improving the accuracy of responses.
• Governance example: Ensuring that all content ingested by the AI is up to date and reliable, to prevent the AI from using obsolete or incorrect information, which is crucial to generating accurate and reliable results.
• Knowledge capture example: Capturing expert knowledge through structured interviews or collaborative workshops enables AI to integrate this valuable information into its models, improving its decision-making capabilities.

By implementing all these elements, KM strengthens the ability of generative AI to provide accurate, reliable and contextual information within an organization.

Louis-Pierre GUILLAUME, Founder and Managing Partner, AMALLTE

Thanks to  Zach Wahl for his webinar @ SIKM in November 2024

To know more

“Artificial Intelligence (AI) and machine learning (ML) technologies do not eliminate the need for establishing a sustainable KM program. Rather, they make foundational KM practices even more essential.” Gartner, 2023

The Transformative Power Of Generative AI And Knowledge Management, Forbes/Forrester, juillet 2024

A Knowledge Management System in my Company – What for?

Call to Action – Free diagnostic

Assess your knowledge management and optimize your generative AI tools.

Request a 45-minute appointment: https://amallte.com/diag