The Good Tech Companies - Building a Hybrid RAG Agent with Neo4j Graphs and Milvus Vector Search

Episode Date: November 5, 2024

This story was originally published on HackerNoon at: https://hackernoon.com/building-a-hybrid-rag-agent-with-neo4j-graphs-and-milvus-vector-search. Build a Graph-RAG agent using Neo4j and Milvus to combine graph and vector search, delivering accurate, context-rich answers to complex queries. Check more stories related to machine-learning at: https://hackernoon.com/c/machine-learning. You can also check exclusive content about #vector-search, #graph-rag, #hybrid-rag, #milvus, #langchain, #llm, #knowledge-graph, #good-company, and more. This story was written by: @neo4j. Learn more about this writer by checking @neo4j's about page, and for more stories, please visit hackernoon.com. This guide explores building a hybrid RAG agent using Neo4j and Milvus to leverage both graph and vector databases, enhancing response accuracy and relevance through combined search methods and self-correction mechanisms.

This audio is presented by Hacker Noon, where anyone can learn anything about any technology. Building a Hybrid RAG Agent with Neo4j Graphs and Milvus Vector Search By Neo4j, this blog post details how to build a graph RAG agent using the Neo4j Graph Database and Milvus Vector Database. This agent combines the power of graph databases and vector search to provide accurate and relevant answers to user queries. In this example, we will use LangGraph LAMA3.18b with OLAMA AND GPT-40. Traditional retrieval augmented generation RAG systems rely solely on vector databases to retrieve relevant documents. Our approach goes further by incorporating Neo4j to capture relationships between entities and concepts, offering a more nuanced understanding of the information.

We want to create a more robust and informative RAG system by combining these two techniques. Building the RAG agent, our agent follows three key concepts, routing, fallback mechanisms, and self-correction. These principles are implemented through a series of lang graph components routing. A dedicated routing mechanism decides whether to use the vector database, the knowledge graph, or a combination of both based on the query. Fallback. In situations where the initial retrieval is insufficient, the agent falls back to a web search using Tavili. Self-correction, the agent evaluates its own answers and attempts to correct hallucinations or inaccuracies. We then have other components,

such as retrieval, we use Milvus, an open-source and high-performance vector database, to store and retrieve document chunks based on semantic similarity to the user's query. Graph enhancement, Neo4j is used to construct a knowledge graph from the retrieved documents, enriching the context with relationships and entities. LLM's integration, LAMA3-18B, a local LLM, is used for generating answers and evaluating the relevance and accuracy of retrieved information, while GPT-40 is used to generate cipher, the query language used by Neo4j. The Graph RAG Architecture The architecture of our Graph RAG agent can be visualized as a workflow with several

interconnected nodes question routing. The agent first analyzes the question to determine the best retrieval strategy, vector search, graph search, or both. Retrieval Based on the routing decision, relevant documents are retrieved from Milvus, or information is extracted from the Neo4j graph. Generation. The LLM generates an answer using the retrieved context. Evaluation. The agent evaluates the generated answer for relevance, accuracy, and potential hallucinations. Refinement. If needed, if the answer is deemed unsatisfactory, the agent may refine its search or attempt to correct errors. Examples of agents. To showcase the

capabilities of our LLM agents, let's look into two different components, and while the full code is available at the bottom of this post, these snippets will provide a better understanding of how these agents work within the langchain framework. Graph generation this component is designed to improve the question answering process by using the capabilities of a Neo4j. It answers questions by leveraging the knowledge embedded within the Neo4j graph database. Here is how it works 1. Allows the LLM to interact with the Neo4j graph database. It uses the LLM in two ways. This instance of the LLM to interact with the Neo4j graph database. It uses the LLM in two ways. This instance of the LLM is responsible for generating cipher queries to extract relevant information from the graph based on the user's question. Validation. Make sure the cipher queries are validated to ensure

that they are syntactically correct. Backslash dot dot. Two. Context retrieval. The validated queries are executed on the Neo4j graph to retrieve the necessary context. Backslash dot. 3. Answer generation. The language model uses the retrieved context to generate an answer to the user's question. This component enables the RAG system to tap into Neo4j, which can help provide more comprehensive and accurate answers. Composite agent, graph and vector magic wand. This is where the magic happens. Our agent can combine results from Milvus and Neo4j, allowing for a better understanding of

the information and leading to more accurate and nuanced answers. Here is how it works. 1. Prompts. We define a prompt that instructs the LLM to use the context from both Milvus and Neo4j to answer the question. 2. Retrieval. The agent retrieves relevant information from Milvus using vector search and Neo4j using graph generation. 3. Answer generation. LLAMA318B processes the prompt and generates a concise answer, leveraging the combined knowledge from the vector and graph databases with the composite chain. Let's have a look at the results of our search, combining the strengths of graph and vector databases to enhance our research paper discovery. We begin with our graph search using Neo4j. The graph search excels at finding relationships and metadata.

It can quickly identify papers based on titles, authors, or predefined categories, providing a structured view of the data. Next, we turn to our vector search for a different perspective. Vector search is really good in understanding context and semantic similarity. It can uncover papers that are conceptually related to the query, even if they don't explicitly contain the search terms. Finally, we combine both search methods. This is a crucial part of our RAG agent, making it possible to use both vector and graph databases. By integrating graph and vector searches, we leverage the strengths of both approaches. The graph search provides precision and navigates structured relationships,

while the vector search adds depth through semantic understanding. This combined method offers several advantages. structured relationships, while the vector search adds depth through semantic understanding. This combined method offers several advantages. 1. Improved recall. It finds relevant papers that might be missed by either method alone. 2. Enhanced context. It provides a more nuanced understanding of how papers relate to each other. 3. Flexibility. It can adapt to different types of queries, from specific keyword searches to broader conceptual explorations. Summing it up, in this blog post, we have shown how to build a graph RAG agent using Neo4j and Melvus. By combining the strengths of graph databases and vector search, this agent provides accurate and relevant answers to user queries. The architecture of our RAG agent, with its

dedicated routing, fallback mechanisms, and self-correction capabilities, makes it robust and reliable. The examples of the graph generation and composite agent components demonstrate how this agent can tap into both vector and graph databases to provide comprehensive and nuanced answers. We hope this guide has been helpful and inspires you to check out the possibilities of combining graph databases and vector search in your own projects. The current code is available on GitHub. To learn more about this topic, join us at Nodes 2024 on November 7, our free virtual developer conference on intelligent apps, knowledge graphs, and AI.

Register now. Thank you for listening to this Hackernoon story, read by Artificial Intelligence. Visit hackernoon.com to read, write, learn and publish. Register now.