File vs Mem vs Claude Native Benchmark¶

Date: 2026-02-23 Method: 3-call timing (start → tool → elapsed) — measures true agent wall-clock cost including 2 MCP round-trips Corpus: dev/ directory (33 markdown files, ~135KB total)

Results¶

Test	file ms	mem ms	native ms	Winner	Notes
T1: Single read (hints.md ~7KB)	4046	4282	5179	file	file 6% faster than mem; native 28% slower
T2: Regex search (LogSpan, C=2)	5030	4099	6302	mem	mem 18% faster than file; native 54% slower than mem
T3: Batch read (8 arch docs)	4299	4405	9191	file	file ≈ mem; native 2.1× slower (8 parallel Reads)
T4: Section extract (3 sections)	4459	5524	6583	file	file wins; mem 24% slower; native required pre-step toc
T5: Pattern search (def \w+()	5454	4641	7487	mem	mem 15% faster than file; native 61% slower than mem

Raw Timings (ms)¶

Test	file	mem	native
T1	4046	4282	5179
T2	5030	4099	6302
T3	4299	4405	9191
T4	4459	5524	6583
T5	5454	4641	7487
Avg	4658	4590	6948

Observations¶

file vs mem — nearly identical overall¶

Average: file=4658ms, mem=4590ms — only 68ms difference (~1.5%)
Neither consistently dominates; each wins 2-3 tests depending on workload
file wins on single read (T1) and section extraction (T4)
mem wins on search tasks (T2, T5) and is roughly equal on batch read (T3)
The file/mem gap is within noise for most tasks — choice should be ergonomic, not performance-based

Where file and mem are closest¶

T3 (batch read): 4299ms vs 4405ms — only 106ms gap (2.5%). Both complete 8-file reads in a single call with identical MCP overhead.
T1 (single read): 4046ms vs 4282ms — 236ms gap (5.8%). Negligible in practice.

Where native falls furthest behind¶

T3 (batch read): native=9191ms vs file=4299ms — 2.1× slower. Even with 8 parallel Read calls in one turn, the per-tool overhead accumulates. This is the strongest case for using file/mem over native.
T5 (pattern search): native=7487ms vs mem=4641ms — 61% slower. The Grep tool itself is fast but carries the same MCP round-trip overhead.
T2 (regex search): native=6302ms vs mem=4099ms — 54% slower.

Ergonomic differences — T4 is most illustrative¶

file.slice_batch / mem.slice_batch: Select sections by heading name in a single call. No pre-knowledge of line numbers needed.
Native: Required a separate file.toc pre-step to discover line ranges, then 3 parallel Read calls with explicit offsets. More error-prone, requires two agent turns minimum.
This ergonomic gap (name vs line-number selection) is significant for agents — wrong line numbers produce incorrect extractions silently.

Native notes¶

The native Grep tool is backed by ripgrep and is genuinely fast at the filesystem level, but the MCP round-trip cost (~1.5-2s per call) makes it uncompetitive when multiple calls are needed.
For single-file targeted reads, native is reasonable but still 15-28% slower than file/mem due to round-trip overhead.
For batch and search operations, native consistently loses by a large margin.

mem-specific notes¶

All 33 docs were already loaded (duplicates skipped on write_batch). Read operations hit the SQLite cache, which is why mem is competitive with file despite the database overhead.
mem.grep returns structured, memory-centric output (memory name, match count, slice reference) that is more useful for navigation than raw line output.
mem.slice_batch accepts heading names (same as file.slice_batch) — no line number lookup needed.

Summary¶

file ≈ mem — essentially interchangeable on performance (~68ms avg difference)
Use file when working with the live filesystem (files may change, no setup needed)
Use mem when files are pre-loaded and you want search-oriented output or cross-memory grep
Avoid native for bulk/search ops — consistently 40-110% slower due to per-call MCP overhead
Native is acceptable for one-off targeted reads when the exact path and line range are already known