🥬 Lettuce cuts the token bill

−42%

tokens, vs. an agent using grep + file reads

Measured, not modeled — a real Claude Code agent solving real GitHub issues, run with and without Lettuce.

0 of 16 supported languages at the 30-run target.

−42%

Fewer tokens

24,683,496 → 14,323,271

−55%

Faster

wall-clock time to an answer

129/160

Tasks won

across 32 repositories

Per-task results

flask #6044 multiple teardown errors are discarded on Python < 3.11+8%

grep+read

87,052

Lettuce

94,414

flask #6023 Config.from_file() silently ignores fewer OSError codes than from_pyfile()−47%

grep+read

117,229

Lettuce

61,854

flask #6056 Dependency Audit — check for outdated packages−47%

grep+read

65,134

Lettuce

34,200

flask #6057 Dead Code Audit — 65 potentially unused functions+81%

grep+read

52,327

Lettuce

94,831

flask #6025 FLASK_DEBUG env variable no longer works as of click 8.4.0+6%

grep+read

89,288

Lettuce

94,919

requests #7341 Documentation for ChunkedEncodingError is either incorrect or very misleading−37%

grep+read

149,719

Lettuce

94,276

requests #7399 documentation for requests.session.request(verify=...) duplicates requests.request() even where it is different−41%

grep+read

133,879

Lettuce

78,675

requests #6885 Unexpected `SSLEOFError` when Receiving Redirect (307) Response with Large Data Payload−60%

grep+read

154,348

Lettuce

62,028

requests #6900 Unable to request a private URL endpoint with custom SNI and self-signed CA when the proxy is set−66%

grep+read

186,442

Lettuce

63,146

requests #7456 typing.get_type_hints() fails on Response, PreparedRequest, and Request after 2.34.0−35%

grep+read

118,253

Lettuce

77,431

zod #6027 Bug: toJSONSchema `$ref` pointer not encoded when meta id contains `/` or `~`−7%

grep+read

160,019

Lettuce

148,661

zod #5917 The results vary depending on the position of the “optional” in the schema with preprocess since v4.4.0−73%

grep+read

340,981

Lettuce

91,526

zod #6019 `superRefine(async ...)` silently drops `ctx.addIssue` when `globalThis.Promise` is patched (Zone.js / Angular NgZone / `@opentelemetry/context-zone`)+27%

grep+read

70,643

Lettuce

89,676

zod #5933 redos checker test fails on Windows: recheck-jar path uses forward-slash regex on backslash paths−12%

grep+read

101,952

Lettuce

89,292

zod #6018 Error: `z.int` is not a function and Cannot read properties of undefined (reading 'datetime') when consuming zod@4 from a tsup-built shared package+30%

grep+read

70,064

Lettuce

90,885

date-fns #4193 Default exports removed from modularized imports+335%

grep+read

60,153

Lettuce

261,751

date-fns #4032 Vite/ESBuild fails to compile date-fns 4 under Node 24−75%

grep+read

250,535

Lettuce

62,562

date-fns #3996 Inconsistent Date Handling for Pre-1883 Dates in TZDate−46%

grep+read

144,879

Lettuce

78,187

date-fns #4068 [bug]: Build fails with date-fns type error when using Bun + Next.js 15 (Turbopack)−76%

grep+read

257,301

Lettuce

62,341

date-fns #4154 Attempted import error: 'millisecondsInSecond'−47%

grep+read

118,103

Lettuce

62,340

axios #6941 Axios v1.8.2 failing with `proxy: false` axios config−25%

grep+read

140,642

Lettuce

106,064

axios #6968 patchForm, postForm, putForm always give out Network Error on Android, but it worked just fine on IOS−24%

grep+read

139,501

Lettuce

106,356

axios #7487 typings bug (not type safety): class field type of AxiosHeaders => any−10%

grep+read

117,590

Lettuce

105,428

axios #6886 Using Next.js' patched fetch with Axios fetch adapter does not pass init options+23%

grep+read

85,365

Lettuce

105,238

axios #6878 onDownloadProgress occurred after request solved−58%

grep+read

206,615

Lettuce

87,226

express #7271 Bug: Nested API sidebar overlapping menu defaults to "Properties" when clicking individual Events/Methods−18%

grep+read

74,878

Lettuce

61,613

express #7268 Add util to print full paths of routes and methods registered across the application−49%

grep+read

120,527

Lettuce

61,609

express #6951 Standardize on error codes for errors express throws−16%

grep+read

73,309

Lettuce

61,530

express #6926 I am unable to set cookies in vercel app for my application. While my backend is deployed in render.−19%

grep+read

117,404

Lettuce

94,824

express #7274 How to inject Open Graph/Meta tags for Social Media before the sendFile function?−54%

grep+read

133,818

Lettuce

61,386

gin #4622 SaveUploadedFile: unexpected chmod on existing directories (breaks /tmp usage)−47%

grep+read

116,960

Lettuce

61,435

gin #4638 Unsafe type assertions in Hijack() and CloseNotify() cause panic with http.TimeoutHandler+25%

grep+read

102,700

Lettuce

127,934

gin #4644 SECURITY: Ghost Bits Vulnerability - Type conversion from rune to byte silently truncates high bits−69%

grep+read

202,340

Lettuce

62,202

gin #4688 AsciiJSON silently corrupts non-BMP characters (emoji) by emitting malformed \u escapes−33%

grep+read

117,161

Lettuce

77,976

gin #4641 Support Google AIP-136 `:verb` custom methods in route paths−43%

grep+read

139,319

Lettuce

79,075

cobra #2359 InitDefaultHelpFlag is not i18n aware−42%

grep+read

163,844

Lettuce

94,589

cobra #2362 Extending a Cobra CLI into an MCP Server and AI Agent — architecture, patterns, and the flag/schema gap+26%

grep+read

62,547

Lettuce

78,879

cobra #2414 Optionally list available commands in the unknown-command error−59%

grep+read

152,871

Lettuce

62,100

cobra #2376 bug: `ValidArgsFunction` receives partial input when argument contains colons in Bash−74%

grep+read

305,117

Lettuce

78,324

cobra #2385 [SUPPORT] Chaining comma-separated dynamic flag completions−74%

grep+read

238,314

Lettuce

61,889

ripgrep #3419 Nondeterminism in ignore::WalkBuilder parallel multi-root walk+2%

grep+read

77,673

Lettuce

78,917

ripgrep #3263 `foo/*/*` gitignore rule is not respected+29%

grep+read

60,921

Lettuce

78,395

ripgrep #3266 rg kicks out odd output from tmux-*.log files+115%

grep+read

60,677

Lettuce

130,538

ripgrep #3364 Unexpected output when running ripgrep with --multiline --pcre2 --json and a non-multiline regex+28%

grep+read

61,543

Lettuce

78,745

ripgrep #3277 Ripgrep config file doesn't work−44%

grep+read

168,500

Lettuce

94,698

fd #1936 [BUG] Artifact attestation mismatch+9%

grep+read

135,352

Lettuce

147,087

fd #1965 [BUG] fd to support light/dark terminal themes−0%

grep+read

78,359

Lettuce

78,312

fd #1919 [BUG] `--exclude` quirk, only seems to affect first `--search-path` in some cases (without glob)+23%

grep+read

105,512

Lettuce

129,776

fd #1985 [BUG] `fd` cannot handle arbitrary deep directories like GNU `find`−0%

grep+read

95,781

Lettuce

95,436

fd #1885 --override-ignore flag to selectively bypass .gitignore−49%

grep+read

121,188

Lettuce

61,705

gson #2689 JDK21 toJson failure−14%

grep+read

91,589

Lettuce

78,787

gson #2977 com.google.gson.JsonIOException: Failed making field 'java.time.LocalDateTime#date' accessible; either increase its visibility or write a custom TypeAdapter for its declaring type.−86%

grep+read

449,313

Lettuce

62,294

gson #2991 Prepare for final field mutation warning under Java 26+5%

grep+read

90,216

Lettuce

94,617

gson #2520 Request to enhance some tests performance to avoid random getDeclaredFields() results−32%

grep+read

90,836

Lettuce

62,035

gson #2638 Gson cannot serialize subclass who override a field−43%

grep+read

167,889

Lettuce

95,277

jsoup #2468 Requests silently bypass SOCKS proxy when using HttpClientExecutor (Java 11+ default)−32%

grep+read

92,035

Lettuce

62,170

jsoup #2486 XML parser does not support DOCTYPE with <!ENTITY> declarations−62%

grep+read

167,689

Lettuce

62,981

jsoup #2472 NodeTraversor revisits nodes created in head() causing infinite loop when mutating (since 1.21.2)−48%

grep+read

118,852

Lettuce

62,261

jsoup #2476 Cleaner may duplicate enforced attrs with preserve-case input−45%

grep+read

144,009

Lettuce

79,406

jsoup #2475 Cleaner may make input document URLs absolute; should have no side effects−88%

grep+read

518,076

Lettuce

62,246

sinatra #2138 sinatra-contrib gemspec should not reference files outside its root directory−73%

grep+read

231,910

Lettuce

61,746

sinatra #2076 Built content_type are malformed−49%

grep+read

119,480

Lettuce

61,355

sinatra #2080 `uri`/`url` helper can blow up with `ArgumentError - string contains null byte`+34%

grep+read

58,426

Lettuce

78,278

sinatra #2113 Empty PATH_INFO is (probably) invalid and will (probably) fail in Rack 3.2.−56%

grep+read

179,390

Lettuce

78,134

sinatra #2077 `NoMethodError` when using `content_type` params with `Integer` value−47%

grep+read

117,259

Lettuce

61,712

faraday #1143 The typheous Faraday Middleware emits a circular dep warning−44%

grep+read

172,749

Lettuce

97,294

faraday #1349 Faraday tries to modify frozen URI (FrozenError)−45%

grep+read

176,261

Lettuce

97,381

faraday #1629 KeyError: key not found: :status−81%

grep+read

430,417

Lettuce

80,738

faraday #718 Distinguish TimeoutErrors for open and read timeouts−14%

grep+read

112,078

Lettuce

96,100

faraday #1602 rspec testsuite error with ruby3.4.0dev+6%

grep+read

75,173

Lettuce

79,788

jq #3538 Inconsistent delpaths behavior with negative indices−69%

grep+read

200,575

Lettuce

61,810

jq #3482 Undefined behavior bug in modulo operator−30%

grep+read

88,208

Lettuce

61,848

jq #3483 UB in string parsing logic (`src/jv_parse.c`)−48%

grep+read

117,844

Lettuce

61,666

jq #3477 Doesn't work with pnpm−61%

grep+read

156,504

Lettuce

61,701

jq #3484 string ("Request: p...) can't be imploded, unicode codepoint needs to be numeric−49%

grep+read

120,501

Lettuce

61,488

cJSON #1022 Heap-Use-After-Free in `cJSONUtils_MergePatch` via Recursive Deletion−34%

grep+read

119,131

Lettuce

78,386

cJSON #999 Undefined-Behavior (Float-Cast-Overflow) in cJSON_CreateNumber via NaN values−23%

grep+read

102,461

Lettuce

78,511

cJSON #1021 cjson: heap-buffer-overflow in cJSON_ParseWithOpts (cJSON.c:1136:21)−13%

grep+read

89,866

Lettuce

78,171

cJSON #1007 Code Review found 2 critical bugs−59%

grep+read

189,673

Lettuce

78,039

cJSON #1013 Heap-Use-After-Free in `cJSONUtils_MergePatch` via Recursive Deletion−34%

grep+read

119,084

Lettuce

78,122

fmt #4741 sprintf reads past end of format string on trailing %−13%

grep+read

167,784

Lettuce

146,315

fmt #4797 fmt::print hangs on pipe write operation if pipe read end is closed (macOS)−72%

grep+read

279,070

Lettuce

79,027

fmt #4436 failed to print error code `:s` fail on windows due to encoding error−19%

grep+read

117,684

Lettuce

95,579

fmt #4756 Quill (which bundle libfmt) fail to build with CUDA when `using namespace std` is used in the code−47%

grep+read

149,363

Lettuce

78,782

fmt #4755 ThreadSanitizer triggers when calling fmt::print from multiple threads on at least `GCC 12.2` up to and including `GCC 14.3`−49%

grep+read

120,863

Lettuce

61,651

spdlog #1200 Windows: flush_every stops working when adjusting system clock backwards−29%

grep+read

132,792

Lettuce

94,240

spdlog #947 Take care of system clock change−12%

grep+read

88,449

Lettuce

77,911

spdlog #3306 Cannot use any formatting when compiling with Clang−59%

grep+read

191,380

Lettuce

79,091

spdlog #3115 Problem building spdlog with external fmt−64%

grep+read

171,513

Lettuce

62,235

spdlog #3578 test_misc triggers GCC 16 array-bounds warning−33%

grep+read

92,277

Lettuce

61,968

Polly #3086 [Bug]: Timeout strategy does not propagate correct CancellationToken+31%

grep+read

59,988

Lettuce

78,615

Polly #2659 [Bug]: Link to documentation site not working−61%

grep+read

247,192

Lettuce

96,826

Polly #3095 [Feature request]: Allow setting the retry attempt−33%

grep+read

141,826

Lettuce

95,036

Polly #3108 [Bug]: IsAotCompatible is not detected+56%

grep+read

153,508

Lettuce

239,989

Polly #3050 [Feature request]: Expose `EnableReloads` overload that accepts a custom `IOptionsMonitor<TOptions>` instance−32%

grep+read

139,515

Lettuce

94,487

serilog #2173 Serilog does not automatically flush batched output sinks when disposed (and has no manual method for doing so)+28%

grep+read

61,095

Lettuce

78,080

serilog #2235 Destructuring does not work on Android.NET in Release build when logging to Seq+142%

grep+read

60,870

Lettuce

147,541

serilog #2194 IL2067 trim analysis on Serilog.Capturing.PropertyValueConverter.CreateStructureValue−21%

grep+read

120,957

Lettuce

94,967

serilog #2201 Using .UseSerilog() on WebApplicationbuilder.Host disables the auto rendering/opening of the page on startup−26%

grep+read

156,314

Lettuce

115,542

serilog #2168 Invalid link in docs (Debugging and Diagnostics)−27%

grep+read

110,596

Lettuce

81,187

monolog #2027 E_WARNING from http_response_code() in handleException when a status line has been registered via header() but not yet sent−33%

grep+read

116,795

Lettuce

78,157

monolog #1736 NormalizerFormatter skips some stacktraces−11%

grep+read

87,561

Lettuce

78,187

monolog #1498 Pushover notifications are randomly lost−44%

grep+read

137,702

Lettuce

77,496

monolog #1989 StreamHandler choking on php://stderr−26%

grep+read

105,921

Lettuce

78,641

monolog #1788 LogRecord->with does not transfers "formatted" value−25%

grep+read

103,302

Lettuce

77,312

carbon #3255 Memory Leak with Laravel and PHPUnit−56%

grep+read

178,655

Lettuce

78,095

carbon #3321 [3.x] Change in behavior for setTestNow−59%

grep+read

192,232

Lettuce

78,375

carbon #3307 Accessing property `end` after setting end date to `CarbonPeriod` via method `setEndDate` returns `null`−88%

grep+read

810,394

Lettuce

94,753

carbon #3287 CarbonImmutable::endOfTime to immutable returns a value without endOfTime−67%

grep+read

288,071

Lettuce

94,432

carbon #3274 PHP stan Unsafe call to private method Carbon\*::getTranslationArray() through static::.−37%

grep+read

227,695

Lettuce

144,489

mockk #1439 mockK 1.14.6 java.lang.ClassNotFoundException: kotlin.Array−67%

grep+read

190,603

Lettuce

62,234

mockk #1325 Still not working well with jdk 21+1%

grep+read

61,151

Lettuce

61,926

mockk #1402 every { java.nio.Files.<method>() } crashes after mockkStatic(java.nio.Files::class)−59%

grep+read

189,968

Lettuce

78,751

mockk #1369 Intermittent java.lang.VerifyError with Kotest Parallel Execution and MockK−44%

grep+read

140,558

Lettuce

78,651

mockk #1353 UnsupportedOperationException when mocking MediaDrm$KeyRequest with RobolectricTestRunner−50%

grep+read

159,328

Lettuce

79,376

koin #2361 [android][koin annotations] Exception in MessageQueue callback: handleReceiveCallback−25%

grep+read

81,643

Lettuce

61,534

koin #2416 Compose getKoin() uses old closed scope in subsequent test executions+2%

grep+read

98,300

Lettuce

100,651

koin #2417 Compose Multiplatform injection of ViewModel with SavedStateHandle fails with 4.2.0-beta3 onwards−36%

grep+read

123,776

Lettuce

78,657

koin #2404 Koin compiler plugin crashes on Kotlin 2.2.20 (NoSuchMethodError in FIR createTopLevelFunction), and Kotlin 2.3.x builds fail with false “Missing definition” across modules−62%

grep+read

164,290

Lettuce

62,379

koin #2414 The Koin Compiler plugin is missing+51%

grep+read

62,786

Lettuce

95,049

scalatra #722 scalatra-atmosphere and RichRequest conflicts getReader/getInputStream and RichRequest.cachedBody problem−53%

grep+read

166,389

Lettuce

78,582

scalatra #375 RicherString.urlEncode doesn't works as expected−33%

grep+read

91,092

Lettuce

61,212

scalatra #537 Scalate runtime exception−64%

grep+read

171,576

Lettuce

61,841

scalatra #412 file upload in async requests−42%

grep+read

107,240

Lettuce

61,790

scalatra #808 Error starting handlers - DynamicScope issue−8%

grep+read

103,368

Lettuce

94,831

scopt #325 NoClassDefFoundError in scopt(4.0.1)−13%

grep+read

108,750

Lettuce

94,605

scopt #323 NPE in Scala 3 (scopt 4.0.1)−77%

grep+read

333,506

Lettuce

78,310

scopt #142 Error with Predictionio−59%

grep+read

232,820

Lettuce

95,113

scopt #179 Trick to use stackable traits pattern when composing a parser−65%

grep+read

222,517

Lettuce

78,011

scopt #113 making cmd/arg/opt hidden make parsing hang cycling infinitely−35%

grep+read

121,633

Lettuce

78,553

Alamofire #2435 cookies not shown in cURLRepresentation−51%

grep+read

160,217

Lettuce

78,242

Alamofire #3073 DataPreprocessor doesn't have it's own failure reason−58%

grep+read

146,765

Lettuce

61,610

Alamofire #3982 DownloadRequest returns success even for non-2xx responses; propose early failure in urlSession(_:didFinishDownloadingTo:)−40%

grep+read

131,763

Lettuce

79,111

Alamofire #3998 Provide a streamlined AsyncSequence for DataStreamRequest to improve ergonomics−74%

grep+read

237,717

Lettuce

61,927

Alamofire #3021 Path Encoding support−36%

grep+read

123,401

Lettuce

78,701

swift-argument-parser #688 Running Command.main() on an AsyncParseableCommand with AsyncParseableCommand sub commands will crash−62%

grep+read

203,950

Lettuce

77,755

swift-argument-parser #827 cannot find type SendableMetatype in scope−48%

grep+read

118,416

Lettuce

61,491

swift-argument-parser #905 1.8.0 `generate-manual` error: You don’t have permission to save the file “name.docc” in the folder “name”−59%

grep+read

229,946

Lettuce

95,012

swift-argument-parser #658 @Argument(parsing:) can't properly handle arguments outside the leaf subcommand when using @OptionGroup to access the parent−46%

grep+read

113,280

Lettuce

61,637

swift-argument-parser #655 Support/Documentation for Compiling a Single Swift File Against the Argument Parser Package (Without Using SwiftPM for the Executable)+2%

grep+read

60,424

Lettuce

61,792

bats-core #799 `--show-output-of-passing-tests` does show no `run` `$output` on passing tests−80%

grep+read

398,716

Lettuce

79,071

bats-core #1186 If a `load` call fails immediately after a `load` call that is successful, the stack trace incorrectly indicates the first `load` failed−66%

grep+read

233,845

Lettuce

78,716

bats-core #1084 Trouble with bats testing psql stored procedures−73%

grep+read

232,682

Lettuce

61,740

bats-core #819 Bats 1.10.0: Hitting error "tput: No value for $TERM and no -T specified" in Concourse CI−60%

grep+read

156,679

Lettuce

61,919

bats-core #1180 JUnit-formatter outputs fail for skipped test ... sometimes−77%

grep+read

411,341

Lettuce

94,664

n #795 Incorrect cache folder path in show_diagnostics+146%

grep+read

133,381

Lettuce

327,956

n #824 Can n better determine the latest version of node when upstream downloads don't exist?−55%

grep+read

291,205

Lettuce

131,167

n #784 Retry when downloading/extraction of Node.js tar fails−52%

grep+read

264,699

Lettuce

128,069

n #799 Problem with curl 8.7.1+63%

grep+read

202,407

Lettuce

330,292

n #789 corepack is not removed when downgrading node+192%

grep+read

151,127

Lettuce

440,768

lazy.nvim #2150 bug: Error "loop in gettable" with too many child specs+58%

grep+read

59,998

Lettuce

95,029

lazy.nvim #2049 bug: LuaJIT FFI: wrong time_t in stats.lua causes segfault on 32-bit musl (ARMv7)−34%

grep+read

119,813

Lettuce

78,552

lazy.nvim #2041 bug: Runtime path reset picks wrong lib path on some systems+30%

grep+read

128,293

Lettuce

166,485

lazy.nvim #2153 bug: Treesitter artefact install dir gets removed from `runtimepath` on bootstrap−48%

grep+read

188,592

Lettuce

97,780

lazy.nvim #2142 bug: Errors when no `lazy.nvim` plugin spec with branch specified−47%

grep+read

117,410

Lettuce

61,734

plenary.nvim #671 Slow `require('plenary.scandir')`−29%

grep+read

110,241

Lettuce

78,086

plenary.nvim #576 Are some tests expected to Fail?−40%

grep+read

131,687

Lettuce

78,358

plenary.nvim #573 Question about usage−26%

grep+read

106,218

Lettuce

78,415

plenary.nvim #588 Plenary Callback issue in dependent plugin−72%

grep+read

275,400

Lettuce

77,503

plenary.nvim #642 Attempt to call a nil value (`isatty`)−30%

grep+read

89,102

Lettuce

62,004

vs Graphify — see the 3-way comparison →

How we did the research

Real repos, never tuned against. 32 open-source repositories — 16 languages, 2 repos per language — none of which Lettuce was built or tuned for. 160 tasks in total, each a real GitHub issue (bug reports and code questions, picked automatically, not hand-curated).

Same agent, same model, two arms. For every issue the identical Claude Code agent (Claude Opus 4.8) located the relevant code twice: once with only grep + file reads, once with Lettuce. Same task, same model — the only thing that changes is whether the agent can call Lettuce.

How Lettuce is used. Lettuce indexes the repo into a graph once, then exposes two MCP tools — understand and read_snippet. The agent calls understand once and reads only the lines it actually needs, instead of grepping and reading whole files. This is the exact same two-tool setup and prompt a real user gets.

Measured, not modeled. Each arm ran twice; the median is reported. Token counts come straight from the agent's own usage meter — not an estimate. We also confirm the agent genuinely calls understand rather than silently falling back to grep.

Last run 6/14/2026.