RAG - Async Pipelines, MCP

What is Synchronous? Everything goes sequentially. Example: P1, P2, and P3 are the three processes. Synchronous means P2 will start when P1 starts and finishes its job. P3 will start when P2 starts and finishes its job. It is not required all the time. During retrieval, in the case of hybrid search, first we will do vector search, then text search, and then any work related to semantic caching / context search. The goal here is to give more relevant context to the LLM. Suppose: Vector search takes 10 seconds Text search takes 5 seconds Semantic caching / context search takes 2 seconds In the case of a sequential process, we need to wait 17 seconds to get the context. So, is there any way we can reduce the waiting time? The answer is Async Pipelines. Here, we will start 3 threads at the same time. One thread will do vector search. Another thread will do text search. The third thread will do semantic caching / context search. Their finishing times may be different. At most, we have to wait only 10 seconds to get the context. Through this async pipeline, we are reducing latency. Context Switching A single thread switches between multiple tasks when needed. It is also called parallel processing. Multithreading Each task will be allotted a separate thread. So, it can finish on its own time. These threads are limited to the available CPU cores. We can see the real difference in latency when we do this at the production level. MCP - Model Context Protocol If we ask the question "What happened today?", an LLM cannot answer. If we ask the same question to ChatGPT or Gemini, it gives a response. How? Because of tool calling, which means attaching some tools or functionality to the LLM. The result of the tool will be fed into the LLM. The application developer will be responsible for writing the code for the tool/functionality. Example Suppose the idea of the application is to fetch weather details. Every developer may write their own code to fetch the weather details. At the end, all users/developers will get more or less the same result. This is not the right approach. So, here comes the concept of MCP. The responsible body/owner that provides the weather details will write the common tool or protocol (a set of rules). The consumer/developer will use the common tool/protocol to fetch the result. Here, the developer does not need to write their own code. The common tool/protocol is called MCP. MCP will have functions and their descriptions. The LLM will decide which function to call. We can call the tool either by: 1. StdIO (Standard Input/Output) MCP is within the same machine. 2. HTTP We can also use the HTTP method to call the MCP, such as calling an API. However, we are introducing latency when using HTTP. What is the Relation Between MCP and RAG? Create an MCP for a RAG system. For example, suppose we have built a RAG system using documents that we gathered. Since you have the database and RAG system, you are the only person who can invoke the RAG system directly. Instead of keeping the RAG functionality restricted to your application, we can expose the RAG functionality through MCP. We can segregate the functionality according to different responsibilities or divisions. For example: RAG system → Responsible for retrieving relevant information from the documents MCP → Exposes the RAG functionality as a common tool LLM/Application → Can invoke the MCP tool when it needs information Now, anyone who has access to the MCP can use the RAG functionality without directly accessing the underlying database or writing their own retrieval code. In this way, MCP acts as a common interface between the RAG system and different applications or LLMs. The RAG system continues to handle document retrieval, while MCP provides a standardized way for other applications or agents to access that functionality.