From aee59ff0b23c0d4903a7f9bd56c7943a87fffdbf Mon Sep 17 00:00:00 2001 From: rope Date: Sat, 1 Aug 2026 17:49:44 +0100 Subject: [PATCH] =?UTF-8?q?qs-ask:=20disable=20thinking=20=E2=80=94=202.4x?= =?UTF-8?q?=20faster,=20same=20answers?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Profiled where the ~6s actually went, and it was not where I assumed. Haiku defaults to extended thinking, so it spent 225 output tokens and 1.9s deliberating before emitting an 80-character answer about the height of a tower. MAX_THINKING_TOKENS=0: 43 output tokens, time-to-first-text 2657ms -> 1122ms, wall clock 5.7s -> ~2.4s, answer unchanged. Two things measured NOT to be the bottleneck, recorded in the comment so nobody optimises them later: process startup (0.12s boot plus 28ms to fire the request, so a warm/persistent process saves nothing) and prompt size (184 input tokens, no bloat to trim). Co-Authored-By: Claude Opus 5 --- hosts/FredOS-Mediaserver.nix | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/hosts/FredOS-Mediaserver.nix b/hosts/FredOS-Mediaserver.nix index 65da41a..4459f23 100644 --- a/hosts/FredOS-Mediaserver.nix +++ b/hosts/FredOS-Mediaserver.nix @@ -112,7 +112,18 @@ ${pkgs.coreutils}/bin/mkdir -p "$work" || { echo "qs-ask: cannot create $work" >&2; exit 0; } cd "$work" || { echo "qs-ask: cannot enter $work" >&2; exit 0; } - ans=$(printf '%s' "$q" | ${pkgs.claude-code}/bin/claude -p \ + # MAX_THINKING_TOKENS=0 is the single biggest win here, worth ~2.4x. + # Measured: Haiku defaults to extended thinking, and it spent 225 output + # tokens and 1.9s deliberating before emitting an 80-character answer + # about the height of a tower. With thinking off: 43 output tokens, + # time-to-first-text 2657ms -> 1122ms, wall clock 5.7s -> 2.4s, same + # answer. A one-line lookup has nothing to reason about. + # + # For the record, two things that are NOT the bottleneck, both measured: + # process startup (0.12s boot + 28ms to fire the request, so keeping a + # warm process saves nothing) and prompt size (184 input tokens). + ans=$(printf '%s' "$q" | ${pkgs.coreutils}/bin/env MAX_THINKING_TOKENS=0 \ + ${pkgs.claude-code}/bin/claude -p \ --safe-mode \ --no-session-persistence \ --tools "" \