Repository navigation
shell: the e2e cut in shards played side by side (#862, step 3) - #867
kitokey-sam[bot] wants to merge 6 commits into
Conversation
…f its own (#862, step 3) After #865, the window's 47 scenarios take ~880 s of a ~19-minute run. headless.sh, for a full run, now hands it to e2e_sharded.sh: the tray's part, main()'s own scenarios (H to R), and the others in three lists balanced by their measured times (e2e_times.txt, longest first into the lightest), each a headless.sh session of its own, side by side. Their text is put back in a whole run's order, every shard's logs are judged together as a whole run's, the screenshots gathered; one verdict, 77 if a shard could not run. KITOKEY_E2E_SHARDS=1 runs it whole, as before. e2e.sh writes where KITOKEY_E2E_OUT says and plays its tray part alone (KITOKEY_E2E_PART=tray) or main()'s own alone (=main). Agent-Session: 92ded11e-58ef-492d-ba56-c820b9011de8 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
What a scenario called in a shard might lack, read at 00ccda7 (ines's point; BG and K's Whisper files are the known case, fixed by #866). Reading only, nothing run. Every scenario's
Not covered by this reading: a scenario that depends on what Kitokey itself left in |
…other shard's (#862) 00ccda7's sharded run: the four window shards died together on WebDriver errors (« session deleted because of page crash or hang »). instances() read every process whose exe is the binary under test — the same for all shards — and Window.__exit__ sent each a TERM: a shard closing a window ended the others' Kitokeys (ines's reading). It now keeps those of this session alone, by their XDG_RUNTIME_DIR, as headless.sh's own ended() does; e2e.sh's count of running Kitokeys after a second start the same. Agent-Session: 92ded11e-58ef-492d-ba56-c820b9011de8 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… leaves them (#862) 4882305's sharded run: AD (two_panes) drew Settings with no model on this computer — K, which writes two, played in another shard — so the guided step stood in the models' place: « Settings, 420 px wide » failed on it, and « Avancé » could not be clicked, ending its shard's run there. AD now writes the same two files when they are not there; a whole run, where K wrote them, is unchanged. Agent-Session: 92ded11e-58ef-492d-ba56-c820b9011de8 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…name on the session's bus (#853, #862) 4f93c92's sharded run: shard 4 died in first_run on « tauri-driver opened no session in a minute; Kitokey's processes: [] », its log ending on « shell.started » and nothing — #853's signature, five sessions side by side making it likelier. A start made while the Kitokey before is still leaving hands its arguments to it and leaves; /proc cannot say a process that is exiting is Kitokey, but its single-instance name on the bus is held to its last instant. Window.__enter__ now waits, up to 20 s, until neither is there; a session that still opens none says whether the name was held. Agent-Session: 92ded11e-58ef-492d-ba56-c820b9011de8 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: 92ded11e-58ef-492d-ba56-c820b9011de8 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: 92ded11e-58ef-492d-ba56-c820b9011de8 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Refs #853 (its likely cause, the single-instance name, waited for). Closes #862 once its oracle holds, as step 3 of 3 (steps 1 and 2 merged in #865).
Module
modules/shell— exactly one.cross-modulelabel is required, justification belowDeliverable
Out of cycle: #862, held, the coordinator's GO, an exception to the freeze. After #865, a full run took about 19.5 minutes, measured at 9eae465:
microphone_chosen166 s,bar_opened116 s, …);The window's part is now the larger, and its scenarios are independent: each starts its own Kitokey.
What it changes
Legend: each shard is a
headless.shsession of its own, with its own GNOME, buses and PipeWire, played side by side. The three lists are balanced bye2e_times.txt.scripts/headless.sh: a full run is handed toe2e_sharded.sh.KITOKEY_E2E_SHARDS(default 3) sets the number of lists;KITOKEY_E2E_SHARDS=1runs it whole in one session, as before. A partial run (KITOKEY_E2E_ONLY) is never sharded. Its text and the session's log go whereKITOKEY_E2E_OUTsays.scripts/e2e_sharded.sh(new):scripts/e2e_shards.py(new):lists: the scenariose2e_only.scenariosreads, longest first into the lightest list (LPT, as Bazel'sshard_count), each list in main()'s order. On the measured times, 3 lists come to 293.9, 293.6 and 293.2 s.merge: the tray's shard, then main()'s own scenarios, then every called scenario in main()'s order, cut on the(name: N s)lines shell: e2e says each scenario's time, and finds a start's tray item in a second, not 50 (#862, steps 1–2) #865 added, then what each shard said outside them.scripts/e2e_times.txt(new): each window scenario's time from 9eae465's run. A scenario not listed counts as the median.scripts/e2e.sh:$KITOKEY_E2E_OUT.KITOKEY_E2E_PART=trayplays the tray's part alone;=mainplays main()'s own scenarios alone, not the tray's.scripts/e2e_window.py: withKITOKEY_E2E_PART=main, the called scenarios are skipped. main()'s own part says its time,(main: N s), before the first call.scripts/e2e_window.py, a session's own Kitokeys (4882305, 0155b8b):instances()keeps those whose environ holds this session'sXDG_RUNTIME_DIR. Before, a shard closing a window sent TERM to the other shards' Kitokeys, which share the binary (ines's reading).Window.__enter__waits, up to 20 s, until no Kitokey holds its single-instance name on the session's bus,com.napkinstack.Kitokey.SingleInstance…. Otherwise a start made while the one before is leaving hands over to it and leaves, which is shell e2e: a start ends right after « shell.started », no error — the single-instance race (#510)? #853's « shell.started, then nothing »; five sessions at once made it likelier.e2e.sh's count of running Kitokeys after a second start is filtered by session the same way.scripts/e2e_window.py, AD (4f93c92): it writes the two model files K leaves, when they are absent.scripts/e2e_logs.py: a shard's logs are judged as a partial run's. The whole run's judgement is the onee2e_sharded.shmakes over all of them.Oracle
KITOKEY_E2E_SHARDS=1) give the same verdict and the sameFAILlines, with a known mutant and without it; the window's part takes less than 880/3 + 3 min; no shard reaches another's folder, bus or sound.whole = lambda b: False): its twoFAILlines.okcount that of the whole run;FAILlines, « e2e: FAILED »;(the run, its shards side by side: N s)), against the whole run's;free -mduring the run). It decides whether N stays 3.Exercised here, without a session:
e2e_shards.py lists 3gives 14, 16 and 17 scenarios, 47 in all, at about 294 s each.mergeon sample shard texts puts the tray first, main()'s own next, then the scenarios in main()'s order whichever shard played them, and each shard's outside lines last.cargo test --locked --test e2e_only: 2 passed.bash -non every script passes;py_compilepasses; ruff (F,PLW2901) finds the same 7 ine2e_window.pyasmain, none in the new files.Kind of oracle: the e2e's own verdict and lines, whole against sharded.
Review budget
main: 323 (+295 −28). Of these, 50 aree2e_times.txt, data of one line a scenario.Contracts
Definition of Done
bash -n,py_compile, ruff as abovee2e_only's, 2 passedTest sheet
Verifier:
FAILlines are those of the same head run whole (KITOKEY_E2E_SHARDS=1), with nothing « not owed » in its logs' judgementFAILlines as the whole run's, and « e2e: FAILED »e2e_shards.py, when the lists are made, then each scenario main() calls is in exactly one list; and when the shards' texts are merged, each scenario's lines appear once, in main()'s orderVerifier's remarks:
Summary
DONE
VERIFIED
e2e_only2 passed; syntax and lint as above.ASSUMED
NOT VERIFIED
RISKS
main;KITOKEY_E2E_SHARDSlowers it, and 1 is the old way.e2e_times.txtgrows stale as scenarios change. The balance suffers, not the verdict. It is refreshed from any full run's(name: N s)lines.🤖 Generated with Claude Code