Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ Help is resolved **first**, before either parser runs and before the dynamic `im

**`src/changedTests.js`**: `resolveChangedTitles(ref, cwd)` shells out to git (`execFileSync` with an argument array, never a string — a ref is user input) and returns the `it()` titles this branch added or changed, to feed the same filter path `--test` uses. `extractTitles(source)` is the pure half. Test files are identified by the **suffix** `*.twd.test.*`, never by directory: the examples use `src/twd-tests`, `app/twd-tests` and `src/twd-test` between them, and the suffix also keeps a project's Vitest suite — which uses `it()` too — from contributing titles. Diffs to the **working tree** (`git diff <base>`, no second ref) and adds untracked test files, so uncommitted work counts; in CI the tree is clean and this is identical to `<base> HEAD`.

**`src/parseArgs.js`**: `parseRunArgs(argv)` returns `{ testFilters, changedSince, record, shard, reportDir, noReport, updateSnapshots, ci, cpuThrottle }`. Supports `--test` (repeatable substring filter), `--changed-since`, `--report-dir`, `--no-report`, `--cpu-throttle`, and the recording flags `--record`, `--record-dir`, `--record-speed`, `--record-pace`. Each accepts both `--flag value` and `--flag=value`; a value starting with `--` is refused, so `--test --record` cannot swallow the flag after it. The returned `record` object is passed to `runTests()` as `recordOverrides` and wins over the config file. `parseReportArgs(argv)` parses `twd-cli report [<dir|run.json>] [--format <f>]` into `{ input, format }`, defaulting `format` to `markdown` and rejecting anything outside `REPORT_FORMAT_CHOICES`.
**`src/parseArgs.js`**: `parseRunArgs(argv)` returns `{ testFilters, changedSince, record, shard, reportDir, noReport, updateSnapshots, ci, cpuThrottle, repeat }`. Supports `--test` (repeatable substring filter), `--changed-since`, `--report-dir`, `--no-report`, `--cpu-throttle`, `--repeat`, and the recording flags `--record`, `--record-dir`, `--record-speed`, `--record-pace`. `repeat` is `null` when the flag is absent. `--repeat <n>` accepts a positive integer in either value form and throws otherwise, the way `--cpu-throttle` does. `--repeat` together with `--record`, `--shard`, `--update-snapshots` or `--ci` is refused in the parser, before launch and before the report folder is cleaned; `record.enabled: true` in the config is refused in `runTests` above `cleanReportDir()` with a message naming that config key. While repeating, the page reloads between rounds (ids are random per load, so rounds join on registration index), `retryCount` is forced to 1, `maxFailures` counts distinct tests that have failed rather than failed rounds, and coverage, contract validation and layout snapshots are skipped (the snapshot injection sets `__TWD_SNAPSHOTS__` to false). A test that failed any round fails the run. `run.json` keeps one row per test and adds `runs` and `passes`, plus `repeat` on the run header; the console counts tests, not rounds, and the console block and both report views call out tests that both passed and failed. Each accepts both `--flag value` and `--flag=value`; a value starting with `--` is refused, so `--test --record` cannot swallow the flag after it. The returned `record` object is passed to `runTests()` as `recordOverrides` and wins over the config file. `parseReportArgs(argv)` parses `twd-cli report [<dir|run.json>] [--format <f>]` into `{ input, format }`, defaulting `format` to `markdown` and rejecting anything outside `REPORT_FORMAT_CHOICES`.

`RUN_FLAGS`, `MERGE_FLAGS` and `REPORT_FLAGS` are the exported list of what each parser recognises, and they do two jobs: `tests/usage.test.js` derives the expected `--help` contents from them, and the "Did you mean" suggestion picks from them. A flag added to a parser has to be added to its list and given a help line in `src/usage.js`, or the suite fails.

Expand Down
3 changes: 2 additions & 1 deletion bin/twd-cli.js
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ if (command === undefined || command === 'help' || isHelp(command)) {
console.log(USAGE[command]());
} else if (command === 'run') {
try {
const { testFilters, changedSince, record, shard, reportDir, noReport, updateSnapshots, ci, cpuThrottle } =
const { testFilters, changedSince, record, shard, reportDir, noReport, updateSnapshots, ci, cpuThrottle, repeat } =
parseRunArgs(args);
const { runTests } = await import('../src/index.js');
const hasFailures = await runTests({
Expand All @@ -37,6 +37,7 @@ if (command === undefined || command === 'help' || isHelp(command)) {
updateSnapshots,
ci,
cpuThrottle,
repeat,
});
process.exit(hasFailures ? 1 : 0);
} catch (error) {
Expand Down
20 changes: 12 additions & 8 deletions docs/cpu-throttling.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,16 +26,20 @@ CPU throttling: 6x (the browser runs 6 times slower; the dev server does not).
Notes:

- **It raises the odds of hitting a race; it does not reproduce one every
time.** Run it in a loop before concluding anything, in either direction:
time.** Repeat the selection in one run before concluding anything, in
either direction. `--repeat` reloads the page between rounds, keeps the
browser open, and names the test that did not pass every time:
```bash
for i in $(seq 10); do
npx twd-cli run --cpu-throttle 6 --test "checkout flow" --no-report > /dev/null && echo pass || echo FAIL
done
npx twd-cli run --repeat 10 --cpu-throttle 6 --test "checkout flow"
```
- **Turn retries off while you hunt.** The default `retryCount` of `2` gives a
failed test a second attempt, and a flake that passes on the second attempt
is exactly what you are looking for. `retryCount` counts attempts, so
`"retryCount": 1` means no retry. A test that only passed on a retry is
A test that passed 8 of those 10 rounds is listed under `Flaky` with
`(8/10 passed)`. The run exits 1 if any round failed. While repeating,
`--repeat` skips coverage, contracts and layout snapshots.
- **Retries are off while `--repeat` is running.** The default `retryCount`
of `2` gives a failed test a second attempt, and a flake that passes on
the second attempt is exactly what you are looking for, so `--repeat`
forces a single attempt. `retryCount` counts attempts: `"retryCount": 1`
means no retry on an ordinary run. A test that only passed on a retry is
listed under `Retried` in the run-complete block.
- **Only the browser is slowed.** The dev server, and anything else outside
the page, runs at full speed.
Expand Down
Loading
Loading