Agentic Search vs Vector RAG for Code: Why Claude Code Uses grep, and When You Shouldn't

Some links on this page are affiliate links. They cost you nothing, and I only link tools I'd mention anyway.

Claude Code finds code by running searches like grep and glob and reading files in a loop, not by querying a vector index of your repository. That is a deliberate choice its creator has described, and it has a real cost: latency and tokens. This post separates what the primary sources say from what third-party blogs claim, runs a small keyword-versus-embedding comparison on a toy codebase, and gives a decision guide for your own project.

If you want the vector-RAG side first, read how RAG works and RAG pipeline architecture, stage by stage. For the general idea of letting a model drive retrieval, see agentic RAG.

What do the primary sources actually say?

I checked these on 2026-10-23.

Claude Code documentation. The page How Claude Code works describes an agentic loop of gathering context, taking action and verifying results. Its tool table lists, under Search: "Find files by pattern, search content with regex, explore codebases." A separate row, code intelligence (jump to definitions, find references), is described as requiring code intelligence plugins. The page does not mention a vector index or embeddings. Absence from a docs page is weak evidence on its own, which is why the next source matters.

Boris Cherny, Latent Space interview. I fetched the Latent Space page and asked a reader model for quotes on RAG. The page showed the date "May 7" with no year; surrounding references suggest 2025, but I could not confirm that from the page. These lines are as the fetch returned them, so check the transcript before quoting me:

• "And originally, we tried very, very early versions of Claude actually used RAG."
• "And there's a lot of complexity that comes with that because the code drifts out of sync."
• "And then there's security issues because this index has to live somewhere."
• "And eventually, we landed on just agentic search as the way to do stuff."
• "So essentially, at the cost of latency and tokens, you now have really awesome search without security downsides."

He also said the performance comparison was "internal vibes", which means there was no published benchmark. I did not find a controlled public comparison from Anthropic. Anyone telling you agentic search is proven better by a benchmark is going beyond what I could find.

Third-party coverage. I also saw OfficeChai's write-up and a SmartScope analysis via search snippets only, not full reads. The SmartScope piece argues the debate is about narrow RAG, and that semantic indexes still have roles for concept search, very large repositories and non-code knowledge. Some secondary sources give a specific month for when Claude Code dropped its index; I could not confirm that against a primary source, so I am not repeating it. I did not try to verify what other tools such as Cursor do, because I found no primary source for that either.

Why would grep beat an index for an agent?

Cherny's reasons, in his words above, are drift, security and quality. Expanded into engineering terms:

• Freshness. An index describes the repo as it was at embedding time. An agent working in a repo is editing it. Every edit makes some chunks stale. Grep reads the file system as it is now.
• Where the index lives. An index of proprietary code is a second copy somewhere, with its own access control, retention and breach surface. Local search needs none of that.
• The model is the query planner. A coding agent can search retry, see results, then search backoff and Retry-After, then open the file. That loop acts as query expansion plus a reranker, done by the model itself.
• Code is full of exact tokens. Function names, error strings, config keys. These are the rare-token queries where keyword search is strong and embeddings are comparatively weak, as the reranker and hybrid measurements showed on synthetic error codes.
• Less to operate. No chunker, embedding model, vector store, re-index job or version mismatch.

The costs are the ones Cherny named: latency and tokens. Every round is a tool call, and every result is read into the context window. See context engineering for why that budget matters.

What happens when I compare keyword and embedding search on a toy repo?

Run on 2026-10-23. Everything here is a toy: 7 Python files and 14 functions I wrote for the test, six questions, with the right file(s) chosen by me. Embeddings are local BAAI/bge-small-en-v1.5 via fastembed (pip install fastembed numpy); keyword search is real grep -rliE. There is no LLM and no agent. Where the keyword search needs a second round, the second pattern is one I wrote by hand to stand in for what an agent might try next. That is a simulation of agent behaviour, not a measurement of one.

The files: net/retry.py (backoff, parse_retry_after, call_with_retries), net/client.py (calls the retry helper), api/limits.py (request throttling), auth/session.py and jobs/cleanup.py (login tokens and their expiry cleanup), billing/invoice.py (tax and discounts), utils/text.py. The functions have short docstrings and some use words the questions do not.

import ast
import re
import subprocess
from pathlib import Path

import numpy as np
from fastembed import TextEmbedding

ROOT = Path("/tmp/r3a/toycode")
STOP = set("where is do we the a of how to can in on at and or for it its are from what who when by get".split())

# (question, file(s) that count as a correct answer, hand-written follow-up pattern an agent might try next)
TASKS = [
    ("where is parse_retry_after defined", {"net/retry.py"}, None),
    ("who calls call_with_retries", {"net/client.py", "net/retry.py"}, None),
    ("where do expired login tokens get removed", {"jobs/cleanup.py"}, "purge|expire|delete"),
    ("what limits how often a client can call us", {"api/limits.py"}, "throttl|rate|MAX_REQUESTS|window"),
    ("stop hammering a server that keeps failing", {"net/retry.py"}, "retry|backoff|attempt"),
    ("how is sales tax added to an order", {"billing/invoice.py"}, None),
]


def grep_files(pattern):
    out = subprocess.run(["grep", "-rliE", pattern, str(ROOT), "--include=*.py"], capture_output=True, text=True).stdout
    return [str(Path(p).relative_to(ROOT)) for p in out.split()]


def keyword_round(question):
    words = [w for w in re.findall(r"\w+", question.lower()) if w not in STOP]
    score = {}
    for w in words:
        for f in grep_files(w):
            score[f] = score.get(f, 0) + 1
    return sorted(score, key=lambda f: -score[f]), words


# function level chunks for embedding search
chunks = []
for f in sorted(ROOT.rglob("*.py")):
    src = f.read_text()
    for node in ast.walk(ast.parse(src)):
        if isinstance(node, (ast.FunctionDef,)):
            chunks.append((str(f.relative_to(ROOT)), node.name, ast.get_source_segment(src, node)))
emb = TextEmbedding("BAAI/bge-small-en-v1.5")
V = np.array(list(emb.embed([c[2] for c in chunks])))
V /= np.linalg.norm(V, axis=1, keepdims=True)


def embed_round(question):
    q = next(iter(emb.query_embed(question)))
    q = q / np.linalg.norm(q)
    order = np.argsort(-(V @ q))
    files = []
    for i in order:
        if chunks[i][0] not in files:
            files.append(chunks[i][0])
    return files, [chunks[i][1] for i in order[:3]]


print(f"{len(list(ROOT.rglob('*.py')))} files, {len(chunks)} function chunks (TOY codebase)\n")
for q, gold, follow in TASKS:
    kf, words = keyword_round(q)
    ef, fn = embed_round(q)
    hit = lambda lst, n: any(f in gold for f in lst[:n])
    print(f"Q: {q}   (answer: {sorted(gold)})")
    print(f"  keyword, one round {words}: top3={kf[:3]}  hit@3={hit(kf, 3)}")
    if follow:
        f2 = grep_files(follow)
        print(f"  keyword, 2nd round grep -E '{follow}': {f2}  hit={any(f in gold for f in f2)}")
    print(f"  embedding: top3 files={ef[:3]} top fns={fn}  hit@1={hit(ef, 1)} hit@3={hit(ef, 3)}")

Real output, trimmed to the questions that show something:

7 files, 14 function chunks (TOY codebase)

Q: where is parse_retry_after defined   (answer: ['net/retry.py'])
  keyword, one round ['parse_retry_after', 'defined']: top3=['net/retry.py']  hit@3=True
  embedding: top3 files=['net/retry.py', 'net/client.py', 'api/limits.py'] top fns=['parse_retry_after', 'call_with_retries', 'backoff_delay']  hit@1=True hit@3=True
Q: what limits how often a client can call us   (answer: ['api/limits.py'])
  keyword, one round ['limits', 'often', 'client', 'call', 'us']: top3=['net/client.py', 'api/limits.py', 'net/retry.py']  hit@3=True
  keyword, 2nd round grep -E 'throttl|rate|MAX_REQUESTS|window': ['net/retry.py', 'api/limits.py', 'billing/invoice.py']  hit=True
  embedding: top3 files=['api/limits.py', 'net/retry.py', 'auth/session.py'] top fns=['allow_request', 'call_with_retries', 'backoff_delay']  hit@1=True hit@3=True
Q: stop hammering a server that keeps failing   (answer: ['net/retry.py'])
  keyword, one round ['stop', 'hammering', 'server', 'that', 'keeps', 'failing']: top3=['auth/session.py']  hit@3=False
  keyword, 2nd round grep -E 'retry|backoff|attempt': ['net/client.py', 'net/retry.py']  hit=True
  embedding: top3 files=['net/retry.py', 'api/limits.py', 'jobs/cleanup.py'] top fns=['call_with_retries', 'parse_retry_after', 'allow_request']  hit@1=True hit@3=True

The other three questions (callers of call_with_retries, expired login tokens, sales tax) were found by both methods in the first round; the full run is the script above.

What the toy shows, and what it does not

• Top-1 file, first round: embeddings were right on 6 of 6. Keyword search put a correct file first on 4 of 6. The two misses ("what limits how often a client can call us" and "stop hammering a server") put net/client.py and auth/session.py first, driven by common words like client, call and that.
• Identifier questions are a tie. Both methods nailed parse_retry_after.
• Concept questions with a vocabulary gap ("hammering a server that keeps failing" for code that says retry and backoff) failed in the first keyword round. A second grep with synonyms found the file. That second round is exactly what an agent would be doing, and exactly where the extra latency and tokens go.
• It proves little about scale. Hit@3 over 7 files is nearly easy by chance, and 14 chunks cannot show index staleness, build time, or repeated scans of a million lines. Do not read this as "embeddings win". Read it as: embeddings save a round on vocabulary gaps, and an iterating searcher recovers them at a price.

Which should I use? A decision guide

Three practical moves that work with agentic search rather than against it:

1. Make the repo searchable by name. Descriptive file and function names are the agent's search surface. A CLAUDE.md that names key directories saves rounds; see Claude Code skills, hooks and subagents.
2. Give it a semantic hop through a tool. If you want embedding search, expose it as an MCP tool and let the agent decide when to call it. Building an MCP server shows the shape. I have not tested this setup, so measure it.
3. Keep retrieval and long context in balance. RAG vs long context vs compaction covers when to stuff files into context instead of searching at all.

How do I check it on my own repo?

Write 15 questions you or a teammate really asked about the code, with the file that answers each. Split them into identifier questions and concept questions. Run each through an agentic session and through an embedding search over function-level chunks, and count: found at all, found in the first round, tool calls and tokens used. If concept questions are rare, agentic search alone is likely enough. If they dominate and each costs five tool calls, an embedding hop is worth testing.

What I could not verify

• The original Boris Cherny post or a full transcript: the quotes came through a fetch tool's extraction of the Latent Space page, and the year was not shown.
• Any controlled benchmark comparing agentic search with vector RAG. Cherny described his own judgement as internal vibes.
• How other coding tools search. I did not find primary sources.
• Anything about Claude Code beyond what the docs page above states on 2026-10-23.