qs-ask: disable thinking — 2.4x faster, same answers
Profiled where the ~6s actually went, and it was not where I assumed. Haiku defaults to extended thinking, so it spent 225 output tokens and 1.9s deliberating before emitting an 80-character answer about the height of a tower. MAX_THINKING_TOKENS=0: 43 output tokens, time-to-first-text 2657ms -> 1122ms, wall clock 5.7s -> ~2.4s, answer unchanged. Two things measured NOT to be the bottleneck, recorded in the comment so nobody optimises them later: process startup (0.12s boot plus 28ms to fire the request, so a warm/persistent process saves nothing) and prompt size (184 input tokens, no bloat to trim). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
309c54d31b
commit
aee59ff0b2
1 changed files with 12 additions and 1 deletions
|
|
@ -112,7 +112,18 @@
|
||||||
${pkgs.coreutils}/bin/mkdir -p "$work" || { echo "qs-ask: cannot create $work" >&2; exit 0; }
|
${pkgs.coreutils}/bin/mkdir -p "$work" || { echo "qs-ask: cannot create $work" >&2; exit 0; }
|
||||||
cd "$work" || { echo "qs-ask: cannot enter $work" >&2; exit 0; }
|
cd "$work" || { echo "qs-ask: cannot enter $work" >&2; exit 0; }
|
||||||
|
|
||||||
ans=$(printf '%s' "$q" | ${pkgs.claude-code}/bin/claude -p \
|
# MAX_THINKING_TOKENS=0 is the single biggest win here, worth ~2.4x.
|
||||||
|
# Measured: Haiku defaults to extended thinking, and it spent 225 output
|
||||||
|
# tokens and 1.9s deliberating before emitting an 80-character answer
|
||||||
|
# about the height of a tower. With thinking off: 43 output tokens,
|
||||||
|
# time-to-first-text 2657ms -> 1122ms, wall clock 5.7s -> 2.4s, same
|
||||||
|
# answer. A one-line lookup has nothing to reason about.
|
||||||
|
#
|
||||||
|
# For the record, two things that are NOT the bottleneck, both measured:
|
||||||
|
# process startup (0.12s boot + 28ms to fire the request, so keeping a
|
||||||
|
# warm process saves nothing) and prompt size (184 input tokens).
|
||||||
|
ans=$(printf '%s' "$q" | ${pkgs.coreutils}/bin/env MAX_THINKING_TOKENS=0 \
|
||||||
|
${pkgs.claude-code}/bin/claude -p \
|
||||||
--safe-mode \
|
--safe-mode \
|
||||||
--no-session-persistence \
|
--no-session-persistence \
|
||||||
--tools "" \
|
--tools "" \
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue