Knowledge Base Management Systems

Continuing with the GenAI series, I am excited to share how we revolutionised the knowledge management system (KMS) for a leading client in the manufacturing industry. R&D teams in manufacturing often face the tedious task of manually sifting through complex engineering documents and standard operating procedures to ensure compliance, uphold safety standards, and drive innovation. This manual process is not only time-consuming but also prone to errors. To address this, we collaborated with our client to automate their R&D function’s KMS using Generative AI (GenAI). By allowing precise querying of specific sections of documents, our solution sped up access to critical information, reducing search time from hours to mere seconds. Our Generative AI team processed over 110 R&D-related documents, leveraging Large Language Models (LLMs) to generate accurate responses to complex queries. Hosted on a leading cloud platform with an Angular-based UI, the solution delivered remarkable benefits, including: - Significant accuracy in generated answers - Faster and more accurate data search and summarisation - Enhanced decision-making with easier access to critical R&D information - Improved overall employee productivity By implementing GenAI for knowledge management, the client's R&D function was also able to improve its competitive edge by tracking and responding quickly to market trends and consumer behavior. With plans to scale the solution to process over 1,500 documents across multiple departments, the client is creating a centralised hub for all their information needs. Taking advantage of GenAI can revolutionize knowledge management by delivering the right information to the right person on demand and enabling strategic impact. #GenAI #ManufacturingInnovation #KnowledgeManagement #GenAIseries #GenAIcasestudy #Innovation #R&D #DigitalTransformation #AI #Deloitte

Focus on Knowledge Management NOW I have been working on the ServiceNow platform for over six years, and one common mistake organizations make is neglecting to mature their knowledge bases and articles. A poor ServiceNow knowledge base can make your entire platform feel bleak. It can be very frustrating when you want to introduce new capabilities, like the virtual agent, or improve your service catalog, but your knowledge bases lack sufficient articles. Organizations need to invest time in building a strong knowledge base before they can successfully develop more comprehensive IT Service Management workflows. Here are several ways organizations can build a strong knowledge base: 1. Conduct an audit of existing knowledge articles to identify which articles should be retired or updated. 2. Hire a dedicated Knowledge Manager responsible for updating existing knowledge articles and creating new ones. 3. Develop a knowledge management governance process for creating new articles to ensure consistency in formatting, a clear content strategy, and proper meta tagging. Create a knowledge article template for this purpose. 4. Establish a review and approval process involving the Knowledge Manager, subject matter experts, and key stakeholders. 5. Ensure that knowledge articles are appropriately linked within service catalog items, virtual agents, and other relevant ServiceNow portals.  6. Gather valuable feedback from end users to ensure that knowledge articles are useful and effectively address their requests and incidents. 7. Review the knowledge management data to identify which articles are viewed the most. This will help you understand how to improve other ITSM workflows related to your service catalog items and request forms. 8. Knowledge Management is not a one-time task; become comfortable with making continuous improvements. Listen to your end users, as they can help you make your knowledge bases better. How do you improve your knowledge articles? Comment below #ITSM #ServiceNow #KnowledgeManagement #ITIL

What are 𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹 𝗔𝘂𝗴𝗺𝗲𝗻𝘁𝗲𝗱 𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝗼𝗻 (𝗥𝗔𝗚) 𝗦𝘆𝘀𝘁𝗲𝗺𝘀? Here is an example of a simple (naive) RAG based Chatbot to query your Private Knowledge Base. (Advanced concepts covered in following weeks). First step is to store the knowledge of your internal documents in a format that is suitable for querying. We do so by embedding it using an embedding model: 𝟭: Split text corpus of the entire knowledge base into chunks - a chunk will represent a single piece of context available to be queried. Data of interest can be from multiple sources, e.g. Documentation in Confluence supplemented by PDF reports. 𝟮: Use the Embedding Model to transform each of the chunks into a vector embedding. 𝟯: Store all vector embeddings in a Vector Database. 𝟰: Save text that represents each of the embeddings separately together with the pointer to the embedding (we will need this later). Next we can start constructing the answer to a question/query of interest: 𝟱: Embed a question/query you want to ask using the same Embedding Model that was used to embed the knowledge base itself. 𝟲: Use the resulting Vector Embedding to run a query against the index in the Vector Database. Choose how many vectors you want to retrieve from the Vector Database - it will equal the amount of context you will be retrieving and eventually using for answering the query question. 𝟳: Vector DB performs an Approximate Nearest Neighbour (ANN) search for the provided vector embedding against the index and returns previously chosen amount of context vectors. The procedure returns vectors that are most similar in a given Embedding/Latent space. 𝟴: Map the returned Vector Embeddings to the text chunks that represent them. 𝟵: Pass a question together with the retrieved context text chunks to the LLM via prompt. Instruct the LLM to only use the provided context to answer the given question. This does not mean that no Prompt Engineering will be needed - you will want to ensure that the answers returned by LLM fall into expected boundaries, e.g. if there is no data in the retrieved context that could be used make sure that no made up answer is provided. To make it a real Chatbot - face the entire application with a Web UI that exposes a text input box to act as a chat interface. After running the provided question through steps 1. to 9. - return and display the generated answer. This is how most of the chatbots that are based on a single or multiple internal knowledge base sources are actually built nowadays. As described, the system is really just a naive RAG that is usually not fit for production grade applications. What are we missing? - Advanced Chunking/Retrieval. - Guardrails/Firewalls. - Evals. - Observability/Governance. - Scalability. More on this in the upcoming posts, so stay tuned in! #LLM #GenAI #LLMOps #MachineLearning

Knowledge Management is hands down the most important factor for scalable GenAI adoption. Here’s a breakdown of the key components: 𝗖𝗲𝗻𝘁𝗿𝗮𝗹 𝗞𝗻𝗼𝘄𝗹𝗲𝗱𝗴𝗲 𝗟𝗶𝗳𝗲𝗰𝘆𝗰𝗹𝗲: The knowledge lifecycle spans the entire knowledge management process, interacting with all other components. It acts as the main decision-making and routing mechanism. 𝗖𝗿𝗲𝗮𝘁𝗲: Documenting knowledge and guiding users on how to capture their experiences with knowledge (both positive and negative). 𝗢𝗿𝗴𝗮𝗻𝗶𝘀𝗲: Structuring content and organising it in a way that ensures ease of access and effective retrieval. 𝗜𝗺𝗽𝗿𝗼𝘃𝗲: Knowledge management relies on systems thinking. As systems evolve, knowledge must be continually improved. 𝗦𝗵𝗮𝗿𝗲: The way existing and new knowledge is presented to users determines its effectiveness. Every business must understand its knowledge-sharing practices—at its core, this is change management. 𝗥𝗲𝘂𝘀𝗲: Reducing redundant work is fundamental to knowledge management. Creating reusable knowledge leads to faster time-to-value for an organisation. For every instance of unsuccessful scaling of an AI solution, there is often a story of poor knowledge management. The more projects we complete at Insurgence, the clearer it becomes that effective and automated knowledge management is at the heart of successful AI adoption at scale. Yes, it’s not glamorous, but it drives progress for the initiatives that do capture attention. Step 1: Find great ideas for AI. Step 2: Build a mechanism to enable them to thrive throughout your organisation at scale. Mandatory component of Step 2: Knowledge Management. At Insurgence we're doing both. Feel free to reach out for a yarn on where AI could help out your team!