rope/nixos
1
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

qs-ask: name the session to drop a redundant API call

Browse source 
Profiled the second model call visible in modelUsage: it is
`source=generate_session_title`, inventing a title for a session that
--no-session-persistence immediately discards. It costs no wall time —
both requests dispatch ~1ms apart and run concurrently, which is why
duration_api_ms was roughly double duration_ms — but it spent 507 input
tokens against 153 for the actual question.

--name qs-ask removes it: one request instead of two, 660 input tokens
down to 153 per question. No latency change, 77% less input token spend,
which is what actually matters when the budget is a subscription rate
limit rather than a bill.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
rope 2026-08-01 17:56:36 +01:00
parent aee59ff0b2
commit a927cf6ba9
1 changed files with 8 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

8
hosts/FredOS-Mediaserver.nix
View file

@ -119,6 +119,13 @@
# time-to-first-text 2657ms -> 1122ms, wall clock 5.7s -> 2.4s, same # time-to-first-text 2657ms -> 1122ms, wall clock 5.7s -> 2.4s, same
# answer. A one-line lookup has nothing to reason about. # answer. A one-line lookup has nothing to reason about.
# #
# --name is not cosmetic: without it Claude Code fires a second,
# concurrent `source=generate_session_title` request to invent a title
# for a session --no-session-persistence then throws away. It costs no
# wall time (both requests dispatch ~1ms apart), but it spent 507 input
# tokens against 153 for the real question. Naming the session up front
# removes it: one request instead of two, 660 input tokens -> 153.
#
# For the record, two things that are NOT the bottleneck, both measured: # For the record, two things that are NOT the bottleneck, both measured:
# process startup (0.12s boot + 28ms to fire the request, so keeping a # process startup (0.12s boot + 28ms to fire the request, so keeping a
# warm process saves nothing) and prompt size (184 input tokens). # warm process saves nothing) and prompt size (184 input tokens).
@ -132,6 +139,7 @@
--permission-mode dontAsk \ --permission-mode dontAsk \
--output-format text \ --output-format text \
--model claude-haiku-4-5 \ --model claude-haiku-4-5 \
--name qs-ask \
--system-prompt 'Answer the question in one or two short sentences, under 240 characters. Lead with the specific fact asked for, including units. No preamble, no caveats, no markdown, no follow-up offers. If you do not know, or the answer depends on live data you do not have, reply with exactly: UNKNOWN' \ --system-prompt 'Answer the question in one or two short sentences, under 240 characters. Lead with the specific fact asked for, including units. No preamble, no caveats, no markdown, no follow-up offers. If you do not know, or the answer depends on live data you do not have, reply with exactly: UNKNOWN' \
2>/dev/null) \ 2>/dev/null) \
|| { echo "qs-ask: claude exited non-zero (auth expired? run 'claude' once to log in)" >&2; exit 0; } || { echo "qs-ask: claude exited non-zero (auth expired? run 'claude' once to log in)" >&2; exit 0; }