Same agent, same issues, three toolsets. Fewer input tokens is better.
Total input tokens across 12 tasks (median of repeated runs per side).
| Repo | Lang | grep | Lettuce | Graphify |
|---|---|---|---|---|
| flask #6044 | python | 88k | 112k | 119k |
| flask #6023 | python | 117k | 62k | 119k |
| zod #6027 | typescript | 160k | 90k | 212k |
| zod #5917 | typescript | 328k | 109k | 306k |
| express #7271 | javascript | 106k | 62k | 105k |
| express #7268 | javascript | 60k | 62k | 150k |
| gin #4622 | go | 103k | 77k | 104k |
| gin #4638 | go | 103k | 112k | 165k |
| ripgrep #3419 | rust | 78k | 95k | 237k |
| ripgrep #3263 | rust | 129k | 62k | 156k |
| sinatra #2138 | ruby | 171k | 62k | 184k |
| sinatra #2076 | ruby | 87k | 61k | 150k |
Same agent, three toolsets. For each of the 12 tasks (real GitHub issues across 6languages), the identical Claude Code agent (Claude Opus 4.8) located the relevant code three times: with plain grep + file reads, with Graphify's MCP, and with Lettuce's MCP. Same model, same task — only the navigation tool changes.
Latest versions, used as intended. Graphify is the current release (v0.8.39) with its own graph built per repo; Lettuce uses its graph + the exact two-tool prompt a real user gets. Each tool is given a prompt that tells the agent to use it first — so each is measured at its best, not handicapped.
Both really use their MCP. We verified from the transcripts that the Graphify arm calls Graphify's graph tools and the Lettuce arm calls understand — neither silently falls back to grep. Token counts come straight from the agent's own usage meter; the median over repeated runs is reported.
Run 3way-lang-20260615 · 6/14/2026.