Repository navigation
Description
The shell's e2e takes 25 to 48 minutes per attest, and attests run one at a time under the machine lock. This issue measures where the time goes before choosing how to cut it, and proposes sharding the window's scenarios once the numbers say it pays.
Measured, from one run
The run is #847's attest at 3c16f92's parent: evidence-847, its shell logs' timestamps, in UTC.
| Part | From → to | Duration | Kitokey starts |
|---|---|---|---|
| GNOME's headless session up | 09:12:40 | — | — |
The tray's scenarios (e2e.sh A–S, 10 scenarios), 00-shell…jsonl |
09:12:45 → 09:36:08 | 23.4 min | 28 |
The window's scenarios (e2e_window.py, 47 called from main()), 01-shell…jsonl |
09:37:01 → 09:52:50 | 15.8 min | 150 |
After the window's last log line, to e2e.txt written |
09:52:50 → 10:01:00 | ~8 min | — |
The tray part is the larger. In it, consecutive Kitokey starts are 50 to 55 s apart: 09:13:36, 09:14:40, 09:15:32, 09:16:24 and so on. Each time, nothing is logged between shell.licence_read, at the start, and the scenario's next action. 28 starts × ~50 s ≈ 23 min, so nearly all of the part is spent between a start and its first gesture.
Assumption, not verified: e2e.sh's start() loops up to 100 times, each time calling busctl for the watcher's items, for the bus's names, and for each candidate menu. Its 0.1 s sleeps add 10 s at most, so the rest would be those calls, or the tray item registering late. A start in the window's part costs about 6 s (150 starts in 15.8 min).
Not in the evidence: time per scenario. e2e.txt and output.txt carry no timestamps, and the window's 150 starts cannot be tied to its 47 scenarios by counting: a static count of as browser and __enter__() gives 120. The last ~8 minutes cannot be attributed either.
Proposed, in this order
- Measure. No behaviour changes.
- Each scenario's header gets its elapsed seconds, in both scripts.
start()says how long it took to find the tray item, and in how many tries.- One full run then gives the table this issue lacks.
- If
start()costs what it seems to, fix it first. This is the largest lever: up to ~20 minutes on every run, with no added RAM. It also benefits the tray part, which cannot be sharded across sessions without a tray each. - Then shard the window's scenarios. Run N sealed headless GNOME sessions in parallel, each playing a list given by
KITOKEY_E2E_ONLY. The tray part runs in its own session. The shards'e2e.txtare merged inmain()'s order, with one verdict.- The lists are balanced by step 1's times: greedy longest-first, as Bazel's
shard_count, pytest-xdist's--dist loadgroupand Playwright's--sharddo. - N = 3 to start. With 23 GB of RAM and about 1 to 1.5 GB per GNOME session with WebKit (an estimate to measure in step 1, from
/proc/<pid>/statusVmRSS of a session's processes), three sessions plus the build leave room. N grows only if the measured peak allows. e2e_logs.pyreads every shard's logs. A scenario'sprovokedlines are owed by whichever shard plays it, so the merge judges them once, over all shards.
- The lists are balanced by step 1's times: greedy longest-first, as Bazel's
Oracle (for step 3's pull request)
- Same verdict, same FAILs. On the same head, the sharded run and the serial run give the same verdict and the same list of
FAILlines. That holds with one mutant known to fail a window scenario, for instance shell: e2e — D60's bar waited for whole, not read after a fixed pause (#850) #851'swhole = lambda b: False, and with it removed. - Faster. Wall time below 15.8/N + 3 minutes for the window's part, measured on this machine; the tray part's time is reported separately.
- No interference. Each shard's audio defaults and source-outputs are unchanged at its end. No shard reads another's folder, and each has its own
WORK,XDG_*and bus.
Activity
shell: e2e says each scenario's time, and how long a start takes to f…
kitokey-sam commentedon Nov 15, 2026
The measure, at 7a4beb9 (#864), in the coordinator's full sealed run: « e2e: passed », 1823
ok, 0FAIL, 2495 s in all. It ran beside another attest, so the times are a little high.Tray, 1502 s (25 min): A 59, B 3, C 52, D 105, E 53, F 181, G 73, L 108, M 275, S 593 s. S2's time is counted in S's.
Window, 862 s (14.4 min) over its 47 scenarios, the longest first:
microphone_chosenbar_openednothing_silentannouncedpress_on_the_page_leftmodels_advancedminutes_checkedfirst_runsound_checkedgestures_answercancelled_at_oncemodel_offeredheard_checkedconfidential_keptcontext_writtentrial_overtwo_panesmeeting_renamedcontext_before_startollama_pulledwindow_looksmodels_listedservice_namedtranscript_correctedpages_seenrecording_found_not_forgottenbar_windowapp_languagemeeting_languagecontext_gapstrial_leftmodels_guidedspeech_offeredtranscription_guidedminutes_copiedmicrophone_rollbacksidebar_foldedcorrections_gatedmeeting_told_onceown_modelsmicrophone_latest_answermeeting_namedactions_rowsconfidential_every_wayword_after_a_gaprecording_foundlist_refreshedWhat it means for step 3: after #865, the tray part should drop to about 3 minutes, and the window's 14.4 minutes become the larger. Sharded in three by longest-first, the window's floor is its longest scenario,
microphone_chosenat 166 s, and its ideal is 862/3 ≈ 287 s. So the run would be about 5 min for the window plus about 3 min for the tray, against about 41 min measured, besides the build and GNOME's start.shell: e2e says each scenario's time, and finds a start's tray item i…
shell: e2e ends and counts its own session's Kitokeys alone, never an…
shell: e2e AD sets up the models its Settings are measured with, as K…
shell: e2e starts a window once no Kitokey holds its single-instance …