Knowledge-Enhanced Large Language Models

Large Language Models (LLMs), such as ChatGPT, are a new generation of AI models that are revolutionizing the way we interact with computers. Trained on massive amounts of text data, LLMs can be used to perform a variety of tasks, including question answering, machine translation, text generation, and even code generation. As a result, LLMs are beginning to be used in a variety of real-world applications. For example, they are being used to develop conversational assistants, also known as co-pilots, such as GitHub Copilot and Microsoft 365 Copilot. These co-pilots can help developers write code faster and more efficiently, and they can also be used to generate other creative content, such as marketing copy or blog posts.

While LLMs have demonstrated remarkable capabilities, they still face limitations in knowledge-intensive tasks. In scenarios where factually correct answers are critical, such as industrial automation or healthcare. These shortcomings stem from inherent challenges such as Hallucination, which refers to the tendency of LLMs to generate factually incorrect or misleading information. Updateability, the struggle to incorporate new knowledge or adapt to evolving information landscapes. Finally, provenance, the ability to trace the origin of information, remains a challenge for LLMs, making it difficult to assess the reliability and trustworthiness of their output.

Overview of the group.

To address the limitations of LLMs, we separate them into two components: a knowledge store and linguistic capabilities. This approach allows us to exploit the complementary strengths of each component, i.e., the knowledge store to provide factual accuracy and the LLM's linguistic capabilities generate creative and informative text. In the KELLM group, we develop approaches that combine Knowledge Graphs (KGs) and Large Language Models (LLMs), also known as Foundational Models (FMs), see Figure 1. KGs are structured sources of factual knowledge, including ontologies and taxonomies. We use KGs to provide LLMs with access to reliable and up-to-date business information, which reduces hallucination and improves the accuracy of LLM output. We preserve the linguistic capabilities of LLMs, such as their ability to understand and generate text. This combination results in a more flexible and adaptable LLM architecture that can incorporate business knowledge as needed. Making LLMs more accessible and useful to a wider range of businesses and organizations. 

Why Knowledge Graphs (KGs) as the foundation of our knowledge store?

The answer lies in their remarkable ability to drive agile knowledge integration, enabling organizations to seamlessly integrate heterogeneous data from disparate sources. KGs enable us to apply logical rules and reasoning to leverage the collective expertise of an organization's subject matter experts for improved reasoning and knowledge discovery. They also unlock the potential for improved search functionality and personalized recommendations, tailoring the user experience to individual interactions and preferences and ultimately increasing engagement. KGs play a key role in maintaining data governance and quality, enabling organizations to define and enforce data standards, relationships, and hierarchies. This, in turn, helps improve data accuracy and consistency, a cornerstone of informed decision making.

Technology offerings

Our primary focus is the development of state-of-the-art cognitive conversational assistants. These intelligent systems are carefully crafted through the harmonious fusion of Knowledge Graphs (KGs) and Large Language Models (LLMs). By combining the structured knowledge representation capabilities of KGs with the natural language understanding and generation capabilities of LLMs, we are at the forefront of creating conversational assistants that are not only adept at understanding context and providing accurate information, but also capable of engaging in dynamic, human-like conversations.

Figure 2 shows how this synergy between KGs and LLMs is at the core of our innovation, enabling us to redefine the way businesses and individuals interact with information and knowledge. Specifically, we help companies.

1. Knowledge graph construction

We offer comprehensive support to companies in their journey towards constructing their knowledge graph. Our approach is flexible, allowing businesses to build KGs at their own pace, following a pay-as-you-go model. This means that organizations can incrementally develop and refine their KGs in alignment with their evolving data needs and priorities. Our guidance ensures that the construction of KGs is not only efficient but also tailored to the specific requirements of each company, facilitating the seamless integration of heterogeneous data sources and the harnessing of valuable insights. 

2. Interfacing KGs and LLMs

We employ cutting-edge techniques, such as retrieval-augmented generation, to facilitate a robust and dynamic interface between these two essential components. Furthermore, our specialized Text-to-SPARQL components enable organizations to effortlessly translate natural language queries into structured queries, making it easier to extract precise and relevant information from KGs. This comprehensive approach empowers companies to unlock the full potential of their data by enhancing the interoperability of KGs and LLMs. 

3. Fine-tuning LLMs for domain specific tasks

We are also dedicated to helping companies fine-tune LLMs for domain-specific tasks. Our specialized approach tailors LLMs to the unique requirements of each business, ensuring they excel in tackling specific challenges within their industry or domain. By fine-tuning LLMs, organizations can leverage the power of state-of-the-art language models to enhance their data-driven decision-making, streamline processes, and achieve superior results in their specific fields. We are here to provide the guidance and expertise necessary to harness the full potential of LLMs in the pursuit of domain-specific excellence. 

4. Connecting tools, e.g., Conversational Forms

We also specialize in connecting various tools to LLMs, with a particular emphasis on conversational interfaces. We empower businesses to seamlessly integrate these advanced language models into their existing software ecosystem, enhancing user interactions and providing innovative conversational experiences. Whether it's chatbots, virtual assistants, or other conversational forms. 


Validation, analysis, and benchmarking LLM outputs using KGs

We assist organizations in ensuring the accuracy and reliability of LLM-generated content by validating it against the structured knowledge stored in KGs. Additionally, our analytical tools enable in-depth examination of LLM outputs, offering valuable insights and actionable data for decision-making. Through benchmarking, we provide a framework for companies to assess the performance of LLMs in various contexts, facilitating continuous improvement and fine-tuning to meet their specific needs and standards. 

6. KGs at each stage of the LLM lifecycle

We are also actively engaged in groundbreaking innovation in the construction of LLMs. Our unique approaches include a thorough examination of how KGs can be integrated into each stage of the LLM lifecycle. By leveraging KGs, we empower organizations to improve LLM construction, from model training to fine-tuning and deployment. This breakthrough approach not only ensures improved LLM performance, but also facilitates the exploration of new applications and opportunities in natural language understanding and generation, ultimately driving innovation and competitive advantage for our customers.


Do not hesitate to contact us for a live demo. 

BAföG Buddy is a specialized conversational assistant that is designed to provide initial advice for BAföG-related questions. While large language models such as ChatGPT can communicate, they often lack the certainty of correctness. Our chatbot, supported by an extensive knowledge base extracted from PDF documents, operates with a precise understanding of BAföG regulations and provides you with accurate answers and guidance. We use our resources, including knowledgeable staff and documented expertise, here at Fraunhofer FIT to meticulously research and deliver customized processes and tools that optimize the chatbot for your unique case, ensuring accurate and reliable assistance about BAföG.

Our Talk with the LLM Ecosystem demonstrator enables users to seamlessly inquire about various Large Language Model (LLM) models, their specific characteristics, the companies responsible for their development, and the licensing details governing their use. This valuable information is meticulously stored in a Knowledge Graph for easy access and accurate responses to user queries. By harnessing the power of this Knowledge Graph, our demonstrator enables users to navigate the complex LLM landscape, providing them with essential insights and knowledge to make informed decisions and realize the full potential of these models.