The Good Tech Companies - Enhancing RAG with Knowledge Graphs: Integrating Llama 3.1, NVIDIA NIM, and LangChain for Dynamic AI

Episode Date: October 22, 2024

This story was originally published on HackerNoon at: https://hackernoon.com/enhancing-rag-with-knowledge-graphs-integrating-llama-31-nvidia-nim-and-langchain-for-dynamic-ai. Use Llama 3.1 native function-calling capabilities to retrieve structured data from a knowledge graph to power your RAG applications. Check more stories related to data-science at: https://hackernoon.com/c/data-science. You can also check exclusive content about #knowledge-graph, #retrieval-augmented-generation, #llama-3.1, #nvidia-nim, #neo4j, #langchain, #graph-based-llm-agent, #good-company, and more. This story was written by: @neo4j. Learn more about this writer by checking @neo4j's about page, and for more stories, please visit hackernoon.com. This article demonstrates the use of Llama 3.1, NVIDIA NIM, and LangChain to create a knowledge graph-based agent for retrieval-augmented generation (RAG), leveraging structured data and dynamic query generation to improve information retrieval and response accuracy.

This audio is presented by Hacker Noon, where anyone can learn anything about any technology. Enhancing RAG with Knowledge Graphs, Integrating LAMA 3.1, NVIDIA NIM, and Langchain for Dynamic AI, by Neo4j. While most people focus on retrieval augmented generation RAG over unstructured text, such as company documents or documentation, I am pretty bullish on retrieval systems over structured information, particularly knowledge graphs. There has been a lot of excitement about GraphRag, specifically Microsoft's implementation. However, in their implementation, the input data is unstructured text in the form of documents, which is transformed into a knowledge graph using a large language model, LLM. In this blog post, we will show how to implement a

retriever over a knowledge graph containing structured information from FDA Adverse Event Reporting System, FAERS, which offers information about drug adverse events. If you have ever tinkered with knowledge graphs in retrieval, your first thought might be to use Sean LLM to generate database queries to retrieve relevant information from a knowledge graph to answer a given question however database query generation using llms is still evolving and may not yet offer the most consistent or robust solution so what are the viable alternatives at the moment in my opinion the best current solution is dynamic query generation rather than relying entirely on an LLM to generate the complete query, this method employs a logic layer that deterministically generates a database query

from predefined input parameters. This solution can be implemented using an LLM with function calling support. The advantage of using a function calling feature leas in the ability to define to an LLM how it should prepare a structured input to a function. This approach ensures that the query generation process is controlled and consistent while allowing for user input flexibility. The image illustrates a process of understanding a user's question to retrieve specific information. The flow involves three main steps. One, a user asks a question about common side effects of the drug Lyrica for people under 35 years old. Backslash dot 2. The LLM decides which function to call and the parameters needed. In this example, it chose a function named side underscore effects with parameters including the

drug Lyrica and a maximum age of 35. Backslash dot 3. The identified function and parameters are used to deterministically and dynamically generate a database query, cipher, statement to retrieve relevant information. Function calling support is vital for advanced LLM use cases, such as allowing LLMs to use multiple retrievers based on user intent or building multi-agent flows. I have written some articles using commercial LLMs with native function calling support. However, we will use recently released LLAMA3.1, a superior open-source LLM with native function calling support. The code is available on GitHub. Setting up the knowledge graph. We will use Neo4j, which is a native graph database, to store the adverse event information. You can

set up a free cloud sandbox project that comes with pre-populated FAERS by following this link. The instantiated database instance has a graph with the following schema. The schema centers on the case node, which links various aspects of a drug safety report, including the drugs involved, reactions experienced, outcomes, and therapies prescribed. Each drug is characterized by whether it is primary, secondary, concomitant, or interacting. Cases are also associated with information about the manufacturer, the age group of the patient, and the source of the report. This schema allows for tracking and analyzing the relationships between drugs, their reactions, and outcomes in a structured manner. We'll start by creating a connection to the database by instantiating

any O4J graph object, setting up the LLM environment. There are many options to host open-source LLMs like the LLAM A3. 1. We will use the NVIDIA API Catalog, which provides NVIDIA NIM inference microservices and supports function calling for LAMA3. 1. Models When you create an account, you get 1000 tokens, which is more than enough to follow along. You'll need to create an API key and copy it to the notebook we'll use the LLAMA3. 1 to 70B because the 8B version has some hiccups with optional parameters in function definitions. The nice thing about NVIDIA NIM microservices is that you can easily host them locally if you have

security or other concerns, so it's easily swappable, and you only need to add a URL parameter to the LLM configuration tool definition. We'll configure a single tool with four optional parameters. We'll construct a corresponding cipher statement based on those parameters to retrieve the relevant information from the knowledge graph. Our tool will be able to identify the most frequent side effects based on input drug, age, and the drug manufacturer. The get underscore side underscore effects function is designed to retrieve common side effects of drugs from a knowledge graph using specified search criteria. It accepts optional parameters for drug name, patient age range, and drug manufacturer to customize the

search. Each parameter has a description passed to an LLM along with the function description, enabling the LLM to understand how to use them. The function then constructs a dynamic cipher query based on the provided inputs, executes this query against the knowledge graph, and returns the resulting side effects data. Let's test the function R tool first mapped the Lyrica drug mentioned in the question to Lyrica, Lyrica crossing values in the knowledge graph, then executed a corresponding cipher statement to find the most frequent side effects. Graph-based LLM agent. The only thing left to do is configure an LLM agent that can use the defined tool to answer questions about the drug's side effects.

The image depicts a user interacting with a LLM3. One agent to inquire about drug side effects. The agent accesses a side effects tool that retrieves information from a knowledge graph to provide the user with the relevant data. We'll start by defining the prompt template. The prompt template includes the system message, optional chat history, and user input. The agent underscore scratchpad is reserved for the LLM as it sometimes needs multiple steps to answer the question, like executing and retrieving information from tools. The langchain library makes it straightforward to add tools to the LLM by using the bind underscore tools method the agent processes input through transformations and handlers that format the chat history. Apply the LLM with the bound tools and parse the output.

Finally, the agent is set up with an executor that manages the execution flow, specifies input and output types, and includes verbosity settings for detailed logging during execution. Let's test the agent results. The LLM identified that it needs to use the get underscore side underscore effects function with appropriate arguments. The function then dynamically generates a cipher statement, fetches the relevant information, and returns it to the LLM to generate the final answer. Summary, function calling capabilities are a powerful addition to open-source models like LAMA3. 1. Enabling more structured and controlled interactions with external data sources and tools.

Beyond just querying unstructured documents, graph-based agents offer exciting possibilities for interacting with knowledge graphs and structured data. The ease of hosting these models using platforms like NVIDIA and IM microservices makes them increasingly accessible. As always, the code is available on GitHub. To learn more about this topic, join us at Nodes 2024 on November 7, our free virtual developer conference on intelligent apps, knowledge graphs, and AI. Register now. Thank you for listening to this Hackernoon story, read by Artificial Intelligence. Visit hackernoon.com to read, write, learn and publish.