The RAG vs MCP framing mistake
The question we hear most ("should we use MCP or RAG?") contains a category error. MCP is a protocol. RAG is a pattern. Asking which to pick is like asking whether to use HTTP or a database. Different layers, and most production systems end up running both.
The dates make the point. Anthropic open-sourced MCP on November 25, 2024 [1]. RAG is older: it comes from a May 22, 2020 arXiv paper, "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks" [2], by Patrick Lewis and co-authors at Facebook AI Research (FAIR), University College London and New York University, published at NeurIPS 2020 [3]. MCP is four and a half years younger. It is not a successor to RAG. It is a different kind of thing.
What MCP actually does (revision 2026-07-28)
MCP standardizes how an AI application talks to the servers that hold tools and context, using JSON-RPC 2.0 messages. Servers offer three primitives: Prompts (user-controlled), Resources (application-controlled) and Tools (model-controlled) [17]. On the client side, the current revision, 2026-07-28, lists one feature, Elicitation, which lets a server ask the user for more input. The same revision made the protocol stateless: every request is self-contained [4]. Roots and Sampling still work but are deprecated, and can be removed in the first revision released on or after July 28, 2027 [5].
MCP does not say where your data lives or how to retrieve it. It says how to ask, so any MCP client can use any MCP server without a custom integration. Anthropic's November 2024 release shipped the specification and SDKs, local MCP server support in the Claude Desktop apps, and pre-built servers for Google Drive, Slack, GitHub, Git, Postgres and Puppeteer [1]. On December 9, 2025, Anthropic donated MCP to the Agentic AI Foundation, a directed fund under the Linux Foundation co-founded by Anthropic, Block and OpenAI [6]. Our MCP pillar covers the governance arc.
What RAG actually does
RAG is a pattern, not a spec. Three steps. Retrieve candidate passages. Pass them to a model as context. Generate an answer grounded in them. The 2020 paper paired a BART generator with a Dense Passage Retriever (DPR) over a December 2018 Wikipedia dump, split into 21 million 100-word chunks and indexed with FAISS [3]. The industry kept the shape and swapped every part.
In production the retriever can be vector search (FAISS, Pinecone, Weaviate, pgvector), keyword search (Elasticsearch, OpenSearch), a knowledge graph, or a hybrid with a re-ranker on top. Chunking, embeddings, re-ranking and evals decide quality, and permission filtering decides who sees what; our Enterprise RAG pillar covers them chapter by chapter.
Behind MCP, a RAG server often stops at retrieval and leaves generation to the client's model. Azure AI Search's generally available API version, 2026-04-01, works this way: retrieval is extractive and the MCP connection returns grounding data only [7].
How MCP and RAG fit together in production
In a realistic enterprise deployment the agent uses MCP to reach a set of servers, often through an MCP gateway, and one of those servers is a RAG implementation over your corpus. The agent does not know it is calling RAG; it is calling a tool that returns relevant context. The RAG server does not know the agent's plan; it received a query and returned passages.
This is no longer a whiteboard pattern. Microsoft documents that each Azure AI Search knowledge base is a standalone MCP server exposing a knowledge_base_retrieve tool [7]. AWS Labs publishes an Amazon Bedrock Knowledge Base Retrieval MCP server [8]. Glean enables a remote MCP server by default for eligible tenants [9], and every Pinecone Assistant gets a dedicated MCP server with a context tool [10].
Two details trip up planning. The tool shape varies: Azure exposes one retrieval tool, while the Bedrock server exposes three (list, query and agentic query), and its setup requires at least one knowledge base tagged mcp-multirag-kb=true [8]. And built in is not the same as switched on: Weaviate's MCP server ships inside the database from v1.38 but is disabled by default when self-hosted [11].
The separation is the point of a protocol. Swap the retriever (a new vector store, hybrid search, GraphRAG) without touching the agent, or swap the agent without touching retrieval. For the practitioner version, with the integration arithmetic and the failure modes of picking only one, see how ASCENDING composes MCP and RAG in one stack (ASCENDING publishes this site).
When to use MCP, RAG or both
Start from the job. If success means "the answer cited the right document", RAG is the substantive architecture. If success means "the agent finished the task across three systems", MCP is. On client work we usually find both jobs on the same roadmap.
| Scenario | What you actually need |
|---|---|
| Single assistant that must answer from your private docs | RAG is the substantive architecture; MCP is optional until you add tools or clients |
| Agent that must call Jira, Slack and Snowflake in one plan | MCP for tool composability; add retrieval only if answers must be grounded |
| Assistant over 10,000 permission-controlled documents | RAG with a retriever that filters by the end user's identity; expose it via MCP once more than one client needs it |
| Regulated deployment that must show who retrieved what | Both: RAG with citation provenance, behind an MCP gateway that authenticates the user, authorizes each tool call and writes the audit record |
| Proof of concept: one model, one data source | RAG alone; MCP wrapping adds moving parts before you need them |
| Multi-client stack (Claude, Copilot, Cursor, your own chat) | MCP, with RAG inside one or more of the servers |
When not to use MCP: one application calling one retrieval service you own, with no second client in sight. A function call or plain API is simpler. When not to build your own RAG pipeline: when the source system already offers permission-aware search over MCP. Glean's remote MCP server, for example, is permission-aware, with each person using their own Glean access [9]. A second index over the same content mostly buys you a second set of permissions to keep in sync.
MCP vs RAG on deployment, identity, audit, transports and hosting
Five rows decide whether an MCP or RAG design survives production review: deployment model, identity and access control, audit logging, supported MCP transports and hosting. They fit a protocol and a pattern awkwardly, which is informative: wherever a cell says "not defined", the answer has to come from your implementation or your MCP gateway.
| Row | MCP (protocol) | RAG (pattern) |
|---|---|---|
| Deployment model | Client-server: an AI application connects through MCP clients to servers running as local subprocesses (stdio) or remote HTTP endpoints [12]. | A pipeline you build or buy (ingest, index, retrieve, optionally generate), shipped inside an application, as a search service, or wrapped as an MCP server. |
| Identity and access control | Authorization is optional. Over HTTP the spec's OAuth 2.1-based flow makes the server a resource server that publishes RFC 9728 metadata and rejects tokens issued for another audience [13]. The spec requires servers to implement proper access controls on tools [14]; with many servers, that policy usually moves to an MCP gateway in front of them. | Not defined by the pattern. Document-level filtering depends on the retriever and on whether the end user's identity reaches it [7] [8]. |
| Audit logging | Not defined by the protocol. The spec says clients should log tool usage for audit; uniform JSON-RPC messages make that easy at a client or gateway [14]. | Not defined by the pattern. A record of which passages went to which user exists only if your retriever or application writes one. |
| Supported MCP transports | stdio and Streamable HTTP in revision 2026-07-28, plus custom transports; Streamable HTTP replies with one JSON object or a request-scoped SSE stream [12]. The older HTTP+SSE transport has been deprecated since 2025-03-26 [5]. | None of its own. Reached through an application API or, once wrapped as an MCP server, over that server's transport. |
| Hosting | Not defined by the protocol. Servers run on a user's machine, in your cloud, or as vendor-hosted endpoints such as Glean's remote MCP server [9] or Pinecone Assistant's per-assistant server [10]. | Wherever the index and retriever run: a managed service (Azure AI Search, Amazon Bedrock Knowledge Bases, Elastic), a vector database you operate, or a library in your app. |
Governing retrieval once it is exposed over MCP
Once a RAG pipeline answers over MCP, the hard questions move. They stop being about embedding models and become: who is asking, may they call this tool, which documents may they see, and what record is kept. The spec leaves most of that to implementers; it says plainly that MCP cannot enforce its security principles at the protocol level [4].
An MCP gateway fills the gap: one endpoint in front of many servers that authenticates the user, decides per tool what that user may call, holds downstream credentials and keeps the record. Our guide to how regulated enterprises choose an MCP gateway covers selection, and the MCP gateway glossary entry has the short definition. For retrieval, five things decide whether governance holds:
- Pass the user, not only the service. Azure AI Search's query-time permission enforcement (in preview) returns unfiltered results for indexed sources when the user identity is omitted [7]. The Bedrock server cannot reach ACL-protected SharePoint, OneDrive or Confluence content without a
user_id[8]. Elastic's MCP endpoint over an API key is one shared identity; over OAuth 2.1 on Elastic Cloud Serverless, each user acts with their own permissions [15]. - Scope discovery to the caller. Revision 2026-07-28 lets
tools/listvary by the authorization on each request, so two users can see different retrieval tools behind one endpoint [14]. - Keep private results private. Revision 2026-07-28 puts cache hints on discovery, list results and
resources/read. A response markedcacheScope: "private", such as a permission-filteredtools/listor a user-dependentresources/read, may be reused only for the same authorization context; caches must not be shared across contexts [16]. Retrieval returned throughtools/callcarries no cache hint at all, so a gateway that caches retrieval results per tool instead of per user turns a cache hit into a data leak. - Name tools for aggregation. Two RAG servers that both expose
searchcollide behind one endpoint; the spec says aggregators should disambiguate, for example by prefixing tool names with a server identifier, and warns that a server's self-reported name is not guaranteed to be unique [14]. - Log the retrieval, not just the call. An auditor wants who asked, which tool, which documents came back and under which policy. The protocol writes none of that. Check what your gateway and your retriever each record.
Disclosure: ASCENDING, which publishes this site, builds Jarvis Registry. The Jarvis MCP gateway is customer-hosted in your AWS or Azure environment and exposes one Registry Endpoint that lists only the MCP servers the signed-in user may access, under RBAC scopes and resource ACLs that cover MCP servers and their tools, private by default until shared. It stores downstream OAuth credentials encrypted at rest, refreshes tokens, supports per-user credentials and keeps raw credentials from the client. ASCENDING describes Jarvis Knowledge Base as enterprise retrieval exposed through that gateway as a governed tool. On logging, Jarvis's published docs cover OpenTelemetry and Prometheus metrics for MCP activity, including latency, but do not specify per-call payload or policy-snapshot logging.
Frequently asked
-
When should I use MCP, RAG or both?
Pick RAG when the answer must cite your own documents; MCP when an agent has to act across systems or several AI clients share the same tools; both when agents need grounded answers and actions, which is the common enterprise case. In that setup retrieval is one MCP tool, the end user's identity reaches the retriever, and an MCP gateway enforces authorization and audit. One model over one source you own needs neither the protocol nor the gateway yet: plain RAG is simpler. -
What is the difference between MCP and RAG?
MCP is a protocol; RAG is a pattern. MCP (Model Context Protocol) standardizes how an AI application discovers and calls tools, resources and prompts on a server, using JSON-RPC 2.0 messages [4]. RAG (Retrieval-Augmented Generation) grounds a model's output in passages retrieved from a corpus [2]. MCP is how the agent talks to tools. A RAG pipeline is one tool it can talk to. -
Do I need MCP if I already have RAG?
Not while one application calls one retriever you own. You need MCP when a second tool or a second AI client arrives: Claude, Copilot, Cursor and your own chat would otherwise each need a custom integration to the same retrieval endpoint. At that point, wrap the existing endpoint as an MCP server instead of inventing a new calling convention, and decide whose identity it acts under, the end user's or one shared service account. -
Can a RAG system be an MCP server?
Yes, and vendors now ship it that way. Each Azure AI Search knowledge base is a standalone MCP server with a knowledge_base_retrieve tool [7], AWS Labs publishes a Bedrock Knowledge Base retrieval MCP server [8], and every Pinecone Assistant has its own MCP server [10]. Some expose one retrieval tool, others several. Pass the end user's identity through, or permission-aware retrieval may not filter. -
Is RAG obsolete now that MCP exists?
No. MCP carries tool calls and context; RAG retrieves and grounds. What MCP replaces is the custom API each client used to call your retriever. The index, the re-ranker and the evals stay where they are, and RAG is still how a model answers from documents it was not trained on.