diff --git a/.github/workflows/pr.yml b/.github/workflows/pr.yml index 391aa3b1c..52405f5a3 100644 --- a/.github/workflows/pr.yml +++ b/.github/workflows/pr.yml @@ -127,6 +127,10 @@ jobs: run: pnpm run typecheck working-directory: ./frontend + - name: Frontend telemetry tests + run: pnpm run test:telemetry + working-directory: ./frontend + - name: Frontend build run: pnpm run build working-directory: ./frontend diff --git a/Makefile b/Makefile index f63990b58..bdfa70962 100644 --- a/Makefile +++ b/Makefile @@ -108,6 +108,7 @@ check-js: $(DEV) pnpm --dir frontend run check check-frontend: + $(DEV) pnpm --dir frontend run test:telemetry $(DEV) pnpm --dir frontend run typecheck $(DEV) pnpm --dir frontend run build diff --git a/docs/antibot-review.md b/docs/antibot-review.md new file mode 100644 index 000000000..dfcd1ea2f --- /dev/null +++ b/docs/antibot-review.md @@ -0,0 +1,120 @@ +# Fair play reports and evaluation + +`/games/:id/ml` is a heuristic review aid, not a trained ML model or a cheating +verdict. The worker persists reports after games and can alert tournament +moderators. Scores do not automatically ban users. + +## What changed after the production audit + +The October 27, 2026 export contained 2,222 available non-site-bot reports, with +866 medium/high results, but no independently confirmed outcomes. It exposed +contradictory timing counters, missing batches in cached reports, legacy Enter +handling and inconsistent offsets. These are collection defects, not labels for +whether a player cheated. The export and scratch analysis live under ignored +`tmp/`; they are not application assets or test fixtures. + +The earlier version-5 offline experiment reduced medium/high results to 264. +That measured alert volume only, using exported batches and cached code/history +features. Those figures do not describe the final version-6 scorer, and do not +establish precision, recall or a false-positive rate. + +Report version 6 implements these changes: + +- Code style, GPT-like phrases, comment ratios, missing backspaces, zero idle + pauses, incomplete edit coverage, browser focus loss and missing telemetry do + not add suspicion. Measurements can remain available for review. Incomplete + coverage is explicitly a collection limitation. +- Overlapping edit rules use their maximum. Timing and blocked-paste support is + capped at four points, below the review threshold of five. These weights are + priorities, not calibrated probabilities. +- Modern version-3 collectors record bounded individual bulk-edit evidence with + offsets and explanations from the corresponding editor change. A paste or + undo elsewhere cannot explain that edit. Vim commands, IME composition, + completions, indentation, undo and permitted self-paste are accounted for. + Client-declared origins remain untrusted hints. +- Legacy edit counters are evaluated per collection window, with explanations + limited to that window. They cannot identify the source of an individual edit. + Server replay jumps remain review context; a client throttle does not prove + that code was typed within 250 ms. Server diffs exclude unchanged code between + separate edits. +- Reports persist structured evidence: stable code, category, weight, message, + measurements and replay offset where available. Compatibility signal strings + remain for existing consumers. + +## Collection quality and competitive evidence + +Version 2 introduced first/last key offsets; aggregation includes pauses between +windows. Version 3 adds collector-session IDs, batch sequence numbers and edit +records. A database uniqueness constraint makes retries of an identified batch +idempotent. Missing sequences and multiple sessions are limitations, never +positive suspicion. Legacy batches without identities remain supported. + +Timing is checked for contradictory samples, interval ordering, pause counts, +missing boundaries, overlaps, session changes and offsets beyond game duration. +Unreliable timing is excluded from rhythm scoring. Missing optional fields stay +absent; they are not silently converted into measured zeroes. Unavailable timing +metrics display as unavailable. + +Competitive code and replay history stop at the first completion marker, including +submission code that arrived before the final throttled editor update. Finished +non-tournament give-ups also close this boundary. Later practice edits are not +used as the competitive solution. The report stores that solution and language, +so cached pages display the same code that was analyzed. + +Only complete telemetry windows ending within the known competitive duration are +used. Duration is recorded in whole seconds; its last partial second is allowed. +A window crossing the boundary is excluded as a whole rather than attributing its +aggregate counters to competitive play. Excluded windows and incomplete coverage +are shown as limitations when retained telemetry is available; an entirely +missing competitive sample shows insufficient telemetry. + +Finished caches include batch count, maximum ID, game state and duration. Every +read checks the snapshot and requires a current report for each eligible player. +Late flushes, lower-ID transactions committing late and changes from playing to +finished invalidate reports. Live reports recompute even without new batches. +Writes after an analysis read are detected on the next read. + +## Moderator decisions and evaluation + +Admins and moderators can append a review on the report page with one of: +`confirmed_assistance` (confirmed unauthorized assistance), `legitimate`, or +`inconclusive`. A reason is required; a replay offset is optional. Records retain +the reviewer, timestamp, detector version and report snapshot. Re-analysis does +not overwrite decisions. Decisions are distinct from user complaints and solution +approval; their reasons are only sent to moderators. + +Run the read-only evaluation command inside the dev container: + +```sh +./bin/dev mix codebattle.antibot.evaluate +``` + +It starts only the database reader and registry, without application workers, +and uses a read-only, repeatable-read transaction. It recomputes the current rules +against the latest decision for each player/game in finished games, +without persisting reports or broadcasting alerts. It reports confusion counts, +precision, recall, false-positive rate, inconclusive/missing cases and language/ +collector cohorts. Undefined rates are null. These metrics describe the reviewed +sample; selection bias and repeated players still matter. Without labels the +command does not claim accuracy. + +For a silent historical report backfill, invoke +`Codebattle.Game.BotDetection.analyze_and_persist(game_id)` for a bounded set of +finished games. The worker path can broadcast tournament alerts and should not +be used for a silent audit. + +## Rollout and ML + +Apply all additive report/review migrations before deploying the application. +Deploy the collector and scorer together; old clients remain supported. Monitor +missing/legacy/inconsistent telemetry separately from score and alert counts. +Review low-scoring games as well as flagged games to measure missed detections. + +Build independently reviewed examples covering mobile/IME, Vim, completions, +undo, self-paste, reconnects, slow networks and known injection/autotyper cases. +Do not turn complaints, existing heuristic scores or practice bots into cheating +labels. Split future ML datasets by player and reserve a later time period for +drift evaluation. Compare candidates with these rules on the same held-out cases, +set a moderator workload and false-positive target, and run models in shadow mode +before using them for alerts. Keep replay review and avoid automatic bans from +scores alone. diff --git a/frontend/package.json b/frontend/package.json index b2ca627d1..e28955cb0 100644 --- a/frontend/package.json +++ b/frontend/package.json @@ -7,6 +7,7 @@ "dev": "vite", "build": "vite build", "preview": "vite preview", + "test:telemetry": "node --test test/telemetry.test.mjs", "typecheck": "tsc --noEmit", "lint": "oxlint --deny-warnings", "check": "tsc --noEmit && oxlint --deny-warnings" diff --git a/frontend/src/features/game/components/TopBar.tsx b/frontend/src/features/game/components/TopBar.tsx index 052d078af..d0ac0777e 100644 --- a/frontend/src/features/game/components/TopBar.tsx +++ b/frontend/src/features/game/components/TopBar.tsx @@ -165,8 +165,8 @@ export function TopBar({ {(canManage || over) && ( - - + + diff --git a/frontend/src/features/game/editor/telemetry.ts b/frontend/src/features/game/editor/telemetry.ts index 72453ca47..ce26319f7 100644 --- a/frontend/src/features/game/editor/telemetry.ts +++ b/frontend/src/features/game/editor/telemetry.ts @@ -24,6 +24,7 @@ export interface TelemetryEvent { large_insert_count?: number; assisted_chars?: number; completion_insert_count?: number; + command_insert_count?: number; paste_chars?: number; /** focus_return: how long the page was hidden or unfocused. */ away_ms?: number; @@ -31,9 +32,13 @@ export interface TelemetryEvent { export const WINDOW_MS = 10_000; export const EVENT_LIMIT = 100; +// Version 2 records key boundaries so pauses between batches remain observable. +export const COLLECTOR_VERSION = 3; interface TelemetryWindow { + edit_evidence: { offset_ms: number; inserted_chars: number; assisted_chars: number; paste_chars: number; printable_keys: number }[]; event_count: number; + collector_version: number; window_start_offset_ms: number; window_end_offset_ms: number; lang_slug: string; @@ -64,6 +69,7 @@ interface TelemetryWindow { large_insert_count: number; assisted_chars: number; completion_insert_count: number; + command_insert_count: number; paste_chars: number; blur_count: number; away_ms: number; @@ -74,10 +80,13 @@ interface TelemetryWindow { max_key_delta_ms: number; idle_pause_over_2s_count: number; last_key_offset_ms: number | null; + first_key_offset_ms: number | null; } const newWindow = (event: TelemetryEvent): TelemetryWindow => ({ + edit_evidence: [], event_count: 0, + collector_version: COLLECTOR_VERSION, window_start_offset_ms: event.offset_ms, window_end_offset_ms: event.offset_ms, lang_slug: event.lang_slug, @@ -108,6 +117,7 @@ const newWindow = (event: TelemetryEvent): TelemetryWindow => ({ large_insert_count: 0, assisted_chars: 0, completion_insert_count: 0, + command_insert_count: 0, paste_chars: 0, blur_count: 0, away_ms: 0, @@ -118,6 +128,7 @@ const newWindow = (event: TelemetryEvent): TelemetryWindow => ({ max_key_delta_ms: 0, idle_pause_over_2s_count: 0, last_key_offset_ms: null, + first_key_offset_ms: null, }); const ARROWS = new Set(['ArrowUp', 'ArrowDown', 'ArrowLeft', 'ArrowRight']); @@ -132,6 +143,7 @@ export function addEvent(current: TelemetryWindow | null, event: TelemetryEvent) switch (event.type) { case 'keydown': { + if (w.first_key_offset_ms == null) w.first_key_offset_ms = event.offset_ms; w.key_event_count += 1; const modifier = event.ctrl_key || event.meta_key; if (typeof event.key === 'string' && event.key.length === 1 && !modifier && !event.alt_key) w.printable_key_count += 1; @@ -167,6 +179,11 @@ export function addEvent(current: TelemetryWindow | null, event: TelemetryEvent) w.drop_blocked_count += 1; break; case 'content_change': + if ((event.inserted_chars ?? 0) >= 8) w.edit_evidence.push({ + offset_ms: event.offset_ms, inserted_chars: event.inserted_chars ?? 0, + assisted_chars: event.assisted_chars ?? 0, paste_chars: event.paste_chars ?? 0, + printable_keys: w.last_key_offset_ms != null && event.offset_ms - w.last_key_offset_ms <= 500 ? 1 : 0, + }); w.content_change_count += event.change_count ?? 0; w.chars_inserted += event.inserted_chars ?? 0; w.chars_deleted += event.deleted_chars ?? 0; @@ -179,6 +196,7 @@ export function addEvent(current: TelemetryWindow | null, event: TelemetryEvent) w.large_insert_count += event.large_insert_count ?? 0; w.assisted_chars += event.assisted_chars ?? 0; w.completion_insert_count += event.completion_insert_count ?? 0; + w.command_insert_count += event.command_insert_count ?? 0; w.paste_chars += event.paste_chars ?? 0; break; case 'focus_return': @@ -195,7 +213,7 @@ export function addEvent(current: TelemetryWindow | null, event: TelemetryEvent) /** The summary pushed to the server; null for an empty window. */ export function finalize(w: TelemetryWindow | null) { if (!w || w.event_count <= 0) return null; - const { last_key_offset_ms: _last, key_delta_total_ms: total, ...rest } = w; + const { key_delta_total_ms: total, ...rest } = w; return { ...rest, avg_key_delta_ms: w.key_delta_sample_count > 0 ? Math.round(total / w.key_delta_sample_count) : 0 }; } @@ -226,16 +244,20 @@ export interface ContentChangeEvent { /** The `executeEdits` source of our own text syncs (language switch, reset, the server's copy). */ export const SYNC_EDIT_SOURCE = 'cb-sync'; -type EditOrigin = 'sync' | 'typed' | 'completion' | 'command' | 'paste' | 'other'; +type EditOrigin = 'sync' | 'composition' | 'typed' | 'completion' | 'command' | 'paste' | 'other'; const originOf = (reason: EditReason | undefined, event: ContentChangeEvent): EditOrigin => { if (event.isFlush) return 'sync'; if (event.isUndoing || event.isRedoing) return 'command'; const { source, kind, name } = reason?.metadata ?? {}; if (source === 'setValue' || (source === 'unknown' && name === SYNC_EDIT_SOURCE)) return 'sync'; + // Our installed Vim adapter uses executeEdits('vim') for normal-mode commands. + // This is an origin hint, not a trusted assertion that the player is human. + if (source === 'unknown' && name === 'vim') return 'command'; if (source === 'suggest' || source === 'snippet') return 'completion'; if (source === 'cursor') { - if (kind === 'type' || kind === 'compositionType' || kind === 'compositionEnd') return 'typed'; + if (kind === 'compositionType' || kind === 'compositionEnd') return 'composition'; + if (kind === 'type') return 'typed'; // Tab / indent / comment toggles and the like. if (kind === 'executeCommand' || kind === 'executeCommands') return 'command'; // The paste guard only lets through text copied from this editor (pasteGuard.ts). @@ -278,6 +300,7 @@ export function contentMetrics(event: ContentChangeEvent) { large_insert_count: 0, assisted_chars: 0, completion_insert_count: 0, + command_insert_count: 0, paste_chars: 0, }; @@ -294,7 +317,7 @@ export function contentMetrics(event: ContentChangeEvent) { // One key, more text: auto-indent ("\n" + spaces) or an auto-closed pair. const keyAssisted = origin === 'typed' && (inserted <= 2 || change.text.trim() === ''); - const assisted = keyAssisted || origin === 'completion' || origin === 'command'; + const assisted = keyAssisted || origin === 'completion' || origin === 'command' || origin === 'composition'; if (origin === 'paste') { metrics.paste_chars += inserted; return; @@ -303,6 +326,7 @@ export function contentMetrics(event: ContentChangeEvent) { if (origin === 'typed') metrics.max_single_insert_len = Math.max(metrics.max_single_insert_len, Math.min(inserted, 1)); metrics.assisted_chars += origin === 'typed' ? Math.max(0, inserted - 1) : inserted; if (origin === 'completion') metrics.completion_insert_count += 1; + if (origin === 'command' && inserted > 0) metrics.command_insert_count += 1; return; } diff --git a/frontend/src/features/game/editor/useEditorTelemetry.ts b/frontend/src/features/game/editor/useEditorTelemetry.ts index fea74b686..a2511c682 100644 --- a/frontend/src/features/game/editor/useEditorTelemetry.ts +++ b/frontend/src/features/game/editor/useEditorTelemetry.ts @@ -5,7 +5,7 @@ import { useLatest } from '@/shared/useLatest'; import type { EditorTelemetry } from './CodeEditor'; import { addEvent, EVENT_LIMIT, finalize, WINDOW_MS } from './telemetry'; -type Send = (summary: NonNullable>, lang: string) => void; +type Send = (summary: NonNullable> & { collector_session: string; batch_sequence: number }, lang: string) => void; /** * Collects my editor's telemetry into windows (≤ 10 s / ≤ 100 events, one language each) @@ -13,6 +13,8 @@ type Send = (summary: NonNullable>, lang: string) => */ export function useEditorTelemetry({ enabled, lang, startedAtMs, send }: { enabled: boolean; lang: string; startedAtMs: number; send: Send }) { const windowRef = useRef | null>(null); + const collectorSession = useRef(crypto.randomUUID()); + const sequence = useRef(0); const timer = useRef(null); const sendRef = useLatest(send); @@ -23,7 +25,7 @@ export function useEditorTelemetry({ enabled, lang, startedAtMs, send }: { enabl } const summary = finalize(windowRef.current); windowRef.current = null; - if (summary) sendRef.current(summary, summary.lang_slug); + if (summary) sendRef.current({ ...summary, collector_session: collectorSession.current, batch_sequence: sequence.current++ }, summary.lang_slug); }, [sendRef]); const add = useCallback( diff --git a/frontend/src/features/gameMl/Comparison.tsx b/frontend/src/features/gameMl/Comparison.tsx index 82a32f5bb..767ef9bbf 100644 --- a/frontend/src/features/gameMl/Comparison.tsx +++ b/frontend/src/features/gameMl/Comparison.tsx @@ -47,9 +47,22 @@ function PlayerHeader({ column }: { column: MlColumn }) { const { t } = useTranslation(); const { player } = column; const report = player.report; - const risk = RISK[report?.level ?? 'none'] ?? RISK.none; + const hasActivity = report?.stats && (report.stats.total_key_events > 0 || report.stats.total_content_changes > 0); + const risk = hasActivity ? RISK[report?.level ?? 'none'] ?? RISK.none : { label: 'Insufficient telemetry', color: 'gray' }; const score = report?.score ?? 0; const signals = report?.signals ?? []; + const qualityIssues: Record = { + missing_batch_identity: t('Missing telemetry batch identity'), + missing_batch_sequence: t('Missing telemetry batches'), + multiple_collector_sessions: t('Multiple editor sessions'), + incomplete_edit_coverage: t('Incomplete edit coverage'), + excluded_post_game_batches: t('Batches outside competitive play excluded'), + legacy_collector: t('Legacy telemetry'), + inconsistent_key_timing: t('Conflicting timing counters'), + outside_game_window: t('Timing exceeds game duration'), + overlapping_key_windows: t('Overlapping telemetry windows'), + incomplete_key_boundaries: t('Missing key timing boundaries'), + }; return (
@@ -73,17 +86,19 @@ function PlayerHeader({ column }: { column: MlColumn }) { {report && ( - + {t(risk.label)} - - - - {score} - - + {(hasActivity || score > 0) && ( + + + + {score} + + + )} )} @@ -95,9 +110,18 @@ function PlayerHeader({ column }: { column: MlColumn }) { )} - {report && !report.stats && ( - } py={6} mt="sm" title={t('No telemetry batches recorded.')}> - {t('Either the player never typed, or their solution was injected via WebSocket without using the editor.')} + {report && !hasActivity && ( + } py={6} mt="sm" title={t('Insufficient telemetry')}> + {t('Missing telemetry does not establish automation. Check the replay for connection gaps and editor activity.')} + + )} + + {hasActivity && Boolean(report?.stats?.quality_issues?.length) && ( + } py={6} mt="sm" title={t('Telemetry limitations')}> + {t('Unreliable timing features are excluded from scoring.')} + + {report?.stats?.quality_issues?.map((issue) => qualityIssues[issue] ?? t('Incomplete telemetry')).join(', ')} + )} @@ -105,14 +129,17 @@ function PlayerHeader({ column }: { column: MlColumn }) { } py={6} mt="sm" title={t('Signals')}> {/* Reasons are the risk scorer's English text. */}
    - {signals.map((signal) => ( -
  • {signal}
  • + {(report?.evidence?.length ? report.evidence : signals.map(message => ({ message, code: 'legacy', weight: 0, replay_offset_ms: null }))).map((signal, index) => ( +
  • + {signal.message} + {signal.replay_offset_ms != null && <> · {Math.round(signal.replay_offset_ms / 1000)} s} +
  • ))}
)} - {report && report.stats && signals.length === 0 && ( + {hasActivity && signals.length === 0 && ( {t('No signals.')} @@ -181,15 +208,15 @@ function useMetricGroups(): MetricGroup[] { }, { label: t('Effective added length'), cell: report((r) => count(r.effective_added_length)) }, { - label: t('Typed coverage'), - help: t('Chars typed in the editor against the code added on top of the template.'), + label: t('Recorded edit coverage'), + help: t('Gross inserted characters compared with final added length. Includes editor assistance and repeated replacements.'), cell: report((r) => { const added = r.effective_added_length ?? 0; const typed = r.stats?.total_chars_inserted; if (!isNum(typed) || added <= 0) return { value: '—' }; const coverage = typed / added; const low = added >= 60 && coverage < 0.6; - return { value: `${(coverage * 100).toFixed(0)}%`, bar: coverage, warn: low, hint: low ? t('low → likely injected/pasted') : undefined }; + return { value: `${(coverage * 100).toFixed(0)}%`, bar: coverage, warn: low, hint: low ? t('Incomplete telemetry or unrecorded edits') : undefined }; }), }, { label: t('Total chars / lines'), cell: code((c) => ({ value: pair(c.total_chars, c.total_lines), bar: c.total_chars })) }, @@ -205,7 +232,7 @@ function useMetricGroups(): MetricGroup[] { const phrases = c.gpt_phrase_matches ?? []; return { ...count(c.gpt_phrase_hits), - warn: phrases.length > 0, + hint: phrases.length > 0 ? `“${phrases.slice(0, 2).join('”, “')}”${phrases.length > 2 ? ', …' : ''}` : undefined, }; }), @@ -216,7 +243,7 @@ function useMetricGroups(): MetricGroup[] { title: t('Keystrokes'), rows: [ { label: t('Batches'), cell: stats((s) => count(s.batch_count)) }, - { label: t('Total events'), cell: stats((s) => ({ ...count(s.total_events), hint: `${(s.events_per_sec ?? 0).toFixed(2)}/s` })) }, + { label: t('Total events'), cell: stats((s) => ({ ...count(s.total_events), hint: s.timing_reliable === false ? undefined : `${(s.events_per_sec ?? 0).toFixed(2)}/s` })) }, { label: t('Key events'), cell: stats((s) => count(s.total_key_events)) }, { label: t('Printable keys'), cell: stats((s) => count(s.total_printable_keys)) }, { @@ -224,7 +251,7 @@ function useMetricGroups(): MetricGroup[] { cell: stats((s) => ({ value: pair(s.total_chars_inserted, s.total_chars_deleted), bar: s.total_chars_inserted, - hint: t('%{rate} ins/s', { rate: (s.chars_per_sec ?? 0).toFixed(2) }), + hint: s.timing_reliable === false ? undefined : t('%{rate} ins/s', { rate: (s.chars_per_sec ?? 0).toFixed(2) }), })), }, { @@ -232,6 +259,11 @@ function useMetricGroups(): MetricGroup[] { help: t('Inserted by the editor for a keystroke: auto-indent, closing brackets, accepted autocomplete. Not counted as pasted.'), cell: stats((s) => ({ value: pair(s.total_assisted_chars, s.total_completion_inserts), bar: s.total_assisted_chars ?? 0 })), }, + { + label: t('Editor command inserts'), + help: t('Edits attributed to editor commands, including Vim. An origin hint, not proof of fair play.'), + cell: stats((s) => count(s.total_command_inserts)), + }, { label: t('Pasted chars'), help: t('Pastes can only bring back code copied from this editor.'), @@ -254,13 +286,13 @@ function useMetricGroups(): MetricGroup[] { { title: t('Timing and edits'), rows: [ - { label: t('Session length'), cell: stats((s) => ({ value: formatMs(s.elapsed_ms), bar: s.elapsed_ms })) }, - { label: t('Avg key delta'), cell: stats((s) => ({ value: formatMs(s.avg_key_delta_ms), bar: s.avg_key_delta_ms })) }, + { label: t('Session length'), cell: stats((s) => s.timing_reliable === false ? { value: '—' } : { value: formatMs(s.elapsed_ms), bar: s.elapsed_ms }) }, + { label: t('Avg key delta'), cell: stats((s) => s.timing_reliable === false ? { value: '—' } : { value: formatMs(s.avg_key_delta_ms), bar: s.avg_key_delta_ms }) }, { label: t('Min / Max key delta'), - cell: stats((s) => ({ value: `${formatMs(s.min_key_delta_ms)} / ${formatMs(s.max_key_delta_ms)}`, bar: s.max_key_delta_ms })), + cell: stats((s) => s.timing_reliable === false ? { value: '—' } : { value: `${formatMs(s.min_key_delta_ms)} / ${formatMs(s.max_key_delta_ms)}`, bar: s.max_key_delta_ms }), }, - { label: t('Idle pauses >2s'), cell: stats((s) => count(s.total_idle_pauses)) }, + { label: t('Idle pauses >2s'), cell: stats((s) => s.idle_pauses_reliable ? count(s.total_idle_pauses) : { value: '—' }) }, { label: t('Multi-char inserts / deletes'), cell: stats((s) => ({ value: pair(s.total_multi_char_inserts, s.total_multi_char_deletes), bar: s.total_multi_char_inserts })), @@ -272,19 +304,16 @@ function useMetricGroups(): MetricGroup[] { value: pair(s.max_single_insert_len, s.max_single_delete_len), bar: s.max_single_insert_len, warn: s.max_single_insert_len >= 8, - hint: s.max_single_insert_len >= 8 ? t('≥8 → likely paste') : undefined, + hint: s.max_single_insert_len >= 8 ? t('Bulk edit; review its origin') : undefined, })), }, { label: t('Large inserts (≥50)'), cell: stats((s) => ({ ...count(s.total_large_inserts), warn: s.total_large_inserts > 0 })) }, { label: t('Content changes / printable keys'), cell: stats((s) => { - const programmatic = s.total_content_changes > s.total_printable_keys + 1; return { value: pair(s.total_content_changes, s.total_printable_keys), bar: s.total_content_changes, - warn: programmatic, - hint: programmatic ? t('more changes than keys → programmatic') : undefined, }; }), }, @@ -294,11 +323,11 @@ function useMetricGroups(): MetricGroup[] { title: t('Focus and the server copy'), rows: [ { - label: t('Left the game / time away'), - help: t('Switches to another tab or window while playing.'), + label: t('Page focus loss / time unfocused'), + help: t('Browser focus or visibility changes, including other windows and dialogs. Context only; does not affect the score.'), cell: stats((s) => - isNum(s.total_blurs) - ? { value: `${formatNumber(s.total_blurs)} / ${formatMs(s.total_away_ms)}`, bar: s.total_away_ms ?? 0, warn: (s.total_away_ms ?? 0) >= 30_000 } + s.focus_tracked && isNum(s.total_blurs) + ? { value: `${formatNumber(s.total_blurs)} / ${formatMs(s.total_away_ms)}`, bar: s.total_away_ms ?? 0 } : { value: '—' }, ), }, @@ -309,11 +338,11 @@ function useMetricGroups(): MetricGroup[] { }, { label: t('Largest jump between updates'), - help: t('Most characters (not counting spaces) one update added. Typing adds a few; a big jump without a paste was not typed.'), + help: t('Most inserted characters (excluding spaces) in one received update. Connection gaps, undo and editor commands can also produce large updates.'), cell: report((r) => { if (!r.editor_history) return { value: '—' }; const jump = r.editor_history.max_jump; - return { ...count(jump), warn: jump >= 80, hint: jump >= 80 ? t('≥80 → not typed') : undefined }; + return { ...count(jump), warn: jump >= 80, hint: jump >= 80 ? t('Large update; check the replay') : undefined }; }), }, ], diff --git a/frontend/src/features/gameMl/format.ts b/frontend/src/features/gameMl/format.ts index 6666b0ad4..f5ec38932 100644 --- a/frontend/src/features/gameMl/format.ts +++ b/frontend/src/features/gameMl/format.ts @@ -1,10 +1,10 @@ import type { RiskLevel } from './types'; export const RISK: Record = { - none: { label: 'Looks human', color: 'green' }, - low: { label: 'Low risk', color: 'blue' }, - medium: { label: 'Suspicious', color: 'orange' }, - high: { label: 'Likely bot', color: 'red' }, + none: { label: 'No suspicious signals', color: 'gray' }, + low: { label: 'Weak signals', color: 'blue' }, + medium: { label: 'Review suggested', color: 'orange' }, + high: { label: 'Strong signals', color: 'red' }, }; /** The score at which the risk scorer says "high" (RiskScorer.classify/1). */ diff --git a/frontend/src/features/gameMl/types.ts b/frontend/src/features/gameMl/types.ts index 1e4d89f2d..50a5ef438 100644 --- a/frontend/src/features/gameMl/types.ts +++ b/frontend/src/features/gameMl/types.ts @@ -21,9 +21,15 @@ export interface MlStats { /** Auto-indent, auto-closed pairs, accepted completions (newer telemetry only). */ total_assisted_chars?: number; total_completion_inserts?: number; + total_command_inserts?: number; + collector_versions?: number[]; + quality_issues?: string[]; + timing_reliable?: boolean; + idle_pauses_reliable?: boolean; /** Pastes (of the player's own code) and leaving the game (newer telemetry only). */ total_paste_chars?: number; paste_tracked?: boolean; + focus_tracked?: boolean; total_blurs?: number; total_away_ms?: number; total_paste_attempts: number; @@ -62,6 +68,7 @@ export interface MlReport { level: RiskLevel; /** Human-readable reasons from the risk scorer (server English). */ signals: string[]; + evidence?: { code: string; category: string; weight: number; message: string; evidence: Record; replay_offset_ms: number | null }[]; stats: MlStats | null; code_analysis: MlCodeAnalysis | null; /** BotDetection.EditorHistory: the editor updates the server received. */ diff --git a/frontend/src/features/lobby/components/Tournaments.tsx b/frontend/src/features/lobby/components/Tournaments.tsx index 50576e363..aa0a0da55 100644 --- a/frontend/src/features/lobby/components/Tournaments.tsx +++ b/frontend/src/features/lobby/components/Tournaments.tsx @@ -1,4 +1,4 @@ -import { ActionIcon, Button, Card, Group, Popover, SimpleGrid, Stack, Text, Title, Tooltip } from '@mantine/core'; +import { ActionIcon, Badge, Button, Card, Group, Popover, SimpleGrid, Stack, Tabs, Text, Title, Tooltip } from '@mantine/core'; import { useInterval } from '@mantine/hooks'; import { IconCalendarUser, IconClockFilled, IconHistory, IconInfoCircle, IconPlus, IconUsers, IconUserStar } from '@tabler/icons-react'; import { type ReactNode, useEffect, useMemo, useState } from 'react'; @@ -184,6 +184,8 @@ export function Tournaments({ isAdmin }: { isAdmin: boolean }) { ({ state }) => state !== 'finished' && state !== 'canceled', ); }, [live, upcoming]); + const seasonTournaments = tournaments.filter(({ grade }) => grade !== 'open'); + const customTournaments = tournaments.filter(({ grade }) => grade === 'open'); return ( @@ -201,19 +203,48 @@ export function Tournaments({ isAdmin }: { isAdmin: boolean }) { - {tournaments.length === 0 ? ( - - - {t("Competition hasn't started yet. Check the schedule for upcoming tournaments.")} - - - ) : ( - - {tournaments.map((tournament) => ( - - ))} - - )} + + + {t('Season tournaments')} + 0 ? {customTournaments.length} : undefined} + > + {t('Custom tournaments')} + + + + {seasonTournaments.length === 0 ? ( + + + {t("Competition hasn't started yet. Check the schedule for upcoming tournaments.")} + + + ) : ( + + {seasonTournaments.map((tournament) => ( + + ))} + + )} + + + {customTournaments.length === 0 ? ( + + {t('No open custom tournaments right now.')} + + + ) : ( + + {customTournaments.map((tournament) => ( + + ))} + + )} + + ); } diff --git a/frontend/src/pages/GameMl.tsx b/frontend/src/pages/GameMl.tsx index bbad37e0b..5d11038ec 100644 --- a/frontend/src/pages/GameMl.tsx +++ b/frontend/src/pages/GameMl.tsx @@ -1,5 +1,5 @@ -import { Head } from '@inertiajs/react'; -import { Alert, Button, Stack } from '@mantine/core'; +import { Head, router } from '@inertiajs/react'; +import { Alert, Button, NativeSelect, NumberInput, Paper, Stack, Textarea } from '@mantine/core'; import { IconAlertTriangle, IconArrowLeft, IconRefresh } from '@tabler/icons-react'; import { useMemo } from 'react'; import { useTranslation } from 'react-i18next'; @@ -18,12 +18,13 @@ interface GameMlProps { can_manage?: boolean; players: MlPlayer[]; batches: MlBatch[]; + reviews?: { id: number; user_id: number; reviewer_id: number; outcome: string; reason: string; replay_offset_ms: number | null }[]; } /** Bot/human signal review of a game: each player's risk report from editor telemetry. */ export default function GameMl() { const { t } = useTranslation(); - const { game_id: gameId, can_manage: canManage = false, players = [], batches = [] } = useSharedProps(); + const { game_id: gameId, can_manage: canManage = false, players = [], batches = [], reviews = [] } = useSharedProps(); // One column per player, left to right: verdict, per-batch charts, the metrics head to head, code. const columns = useMemo(() => { @@ -43,7 +44,7 @@ export default function GameMl() { @@ -69,6 +70,28 @@ export default function GameMl() { + {canManage && players.filter(player => !player.is_bot).map(player => ( +
{ + event.preventDefault(); + const data = Object.fromEntries(new FormData(event.currentTarget)); + router.post(`/games/${gameId}/ml/reviews`, { ...data, user_id: player.id }); + }}> + + {t('Moderator review')}: {player.name} + + +