diff --git a/CHANGELOG.md b/CHANGELOG.md index 29e5731..4a121f0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,12 @@ entry per wave, newest first; the first six waves (V1–V6, the MVP) predate the are recorded in PLAN.md §B and §J. Each entry carries the opening sentence of its row and the reason the wave was built. +## V48 — A damaged share link ends on a refusal, never on a crash + +The plan for this row was a deeper validator: check `insights.importance`, numeric metrics, `classes` and `confusion`, the fields the audit had named. + +_Why:_ Approved 09/10/2026 after V47. A link is input from a stranger, and the most common stranger is a chat client that cut it. The decoder already refused the shapes the V35 audit had found; the measurement showed it could not keep up with the format, and that the robust answer was at the point of failure rather than at the gate. + ## V47 — Workers that cannot answer the wrong question Six lifecycle defects in shipped code, each reproduced by a test that failed before its fix. diff --git a/PLAN.md b/PLAN.md index a7fe492..9067568 100644 --- a/PLAN.md +++ b/PLAN.md @@ -613,7 +613,7 @@ before features, and the one that produces wrong data comes first of all. | Wave | Content | Why | | ------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **V47 — delivered** | **Workers that cannot answer the wrong question.** Six lifecycle defects in shipped code, each reproduced by a test that failed before its fix. **(1) A result filed under a target it was not trained for**: `setTarget` and `toggleColumn` reset the training state without cancelling the run, the worker kept posting `model-result`, and the auto-save recorded the new target beside the old scores. The store now refuses both changes while anything computed for the current question is running (training, tuning, learning curve, robust ranking, batch scoring), the target picker and the column buttons are disabled for the same span, and a fence (`isStaleResponse`) drops any training message outside a run and any run-derived answer when no run is done; the worker answers in order, so the training status is enough to tell a late answer from a current one, without a run id in the protocol. **(2) Cancel did not reach the parallel phase**: V37's helpers were only checked after every one of them had finished, so cancelling waited for the heaviest families; the phase now polls the flag every 100 ms and terminates the helpers mid-fit. **(3) A crashed ML worker** left `trainStatus` at `training` forever and kept the dead reference; `onerror` now terminates it, clears every busy flag, and the next request starts a fresh worker. **(4) The same crash in the chat** left « thinking » up and still offered a local model that died with its worker. **(5) DuckDB was never closed**: each visit to the SQL panel left a worker and its Wasm heap behind, a failed start leaked its worker too, and a failed Parquet export left its file in the heap; the panel now closes the engine on unmount, on a late start and on failure, and export names come from a counter rather than a millisecond clock. **(6) A webcam stream granted after the page was left** kept the camera lit; it is now stopped on arrival. **(7) Found by this wave's own CI**: touching the chat ran the interpretation bench, which failed on `size-mismatch:tokenizer_config.json`. `prepare-llm.mjs` called its revision « pinned » while it held `main`, and upstream commit `1e94c62` (07/10/2026) had grown that file from 9 762 to 10 416 bytes; the revision is now the commit the sizes were measured on (`d6beae7`), which also keeps the next deploy cache miss from failing the same way. **What this wave deliberately does not do**: add a run id to the worker protocol (the order guarantee makes it redundant today, and it is the first thing to add if requests ever become concurrent), cancel analyses the user cannot cancel today, or touch the deeper share-link validation, which is V48. 883 unit tests, 123 e2e. | Approved 09/10/2026 after the post-V46 audit. The defect in (1) does not crash and does not warn: it saves a run that looks valid and is not, which is the one kind of failure an honest lab must never produce. The others are the same class of bug, a worker outliving the state it was started for, and fixing them together keeps the rule in one place. | -| **V48 — pending** | **Share links validated as deeply as they are read.** `isRenderable` checks that `insights` is an object, then `RunView` calls `insights.importance.slice`; a link carrying `insights: {}` passes the gate and crashes the page, and metric values reach `.toFixed` unchecked. Validate every field the view dereferences (`importance` as an array of `{column, value}`, metrics as finite numbers, `classes` and `confusion` when present), extend `share-hostile.test.ts` with each shape, and merge the five copies of the download helper into one that revokes its object URL after the click has been handled rather than in the same tick. | A link is input from a stranger. The page already refuses most hostile shapes by name; the ones it lets through should be refused the same way, not crash. | +| **V48 — delivered** | **A damaged share link ends on a refusal, never on a crash.** The plan for this row was a deeper validator: check `insights.importance`, numeric metrics, `classes` and `confusion`, the fields the audit had named. The wave opened with a measurement instead, and the measurement changed the plan. Three real links were captured from the running app (titanic with uncertainty, threshold and segments; iris with k-NN tuning and groups; mpg as a regression) and every field of each was deleted, nulled, or retyped as a string, a number, an object or an array: **3 948 damaged links, 3 660 accepted by the decoder, 1 020 of those crashed the page, on 99 distinct paths**, from `insights.importance` to `artifacts.exploration.clusters[].traits[].clusterMean`. A validator for 99 paths would be the largest file in the share module and would drift with every analysis a run learns to carry. So the decoder keeps checking what every view reads first, and `SharedRunPage` wraps the run in a render guard: any failure while drawing it shows the same refusal page, with its own explanation (the link decoded, but part of the run does not have the shape LabML writes) and nothing from the run, so no half-read figure can pass for a real one. The three links are committed under `tests/share-links/`, and the measurement is now a test: every field shape is damaged six ways and the page must end on the run or on the refusal; it fails on the code before this wave and adds about 25 s to the unit suite, the price of covering the whole format rather than a sample. The five copies of the download helper become one, `src/lib/download.ts`, which revokes its object URL after the click has been handled instead of in the same tick, where some engines cancel the download silently. **What this wave deliberately does not do**: write a field-by-field schema for every artifact, show a partially readable run, or guard event handlers (a click on a damaged run is not reachable: the refusal replaces it first). 889 unit tests, 123 e2e. | Approved 09/10/2026 after V47. A link is input from a stranger, and the most common stranger is a chat client that cut it. The decoder already refused the shapes the V35 audit had found; the measurement showed it could not keep up with the format, and that the robust answer was at the point of failure rather than at the gate. | | **V49 — pending** | **CI that cannot be steered by a branch name.** `ci.yml` interpolates `github.head_ref` straight into a `run:` line; pass it through `env:` instead. Pin third-party actions by commit SHA (Dependabot keeps them current), add a coverage report with a floor set from the measured baseline rather than a round number, and run zizmor on the workflows. | Continues V44. The injection needs write access to the repository today, which is why it is low severity, and why the fix should land before that stops being true. | | **V50 — pending** | **Metrics that agree with the reference on every edge.** `macroPrf` averages over every class the task knows, so a class absent from both truth and prediction in the test split adds a zero; scikit-learn averages over the labels present. `rmse` and `mae` return 0 on an empty set, which reads as a perfect score. A stratified split sends a single-row class entirely to test. Decide each case against the V42 generator (add the absent-class case to it), then align or document. | The same honesty rule as V35: a number must not flatter or punish by accident. Measure the difference on the demo datasets before changing anything. | | **V51 — pending** | **A store that fits on a screen.** `lab-store.ts` (about 1 000 lines, a 250-line `onmessage` chain) split into feature slices with a dispatch table; tests named after waves (`v35.test.ts` to `v40.test.ts`) renamed after the behaviour they pin; the duplicated missing-ratio and histogram code in `drift.ts` and `reference.ts` merged. No behaviour change, proven by the suite staying identical. | Only after V47 to V50: refactoring code whose defects are not yet pinned by tests moves the defects around instead of removing them. | diff --git a/README.md b/README.md index 3e11a14..0af7dc2 100644 --- a/README.md +++ b/README.md @@ -241,7 +241,7 @@ Wikimedia Commons), the portrait is NASA, public domain._ a doc page carries its whole article, in English until the app mounts); Lighthouse mobile ≈ 0.99 on `/ml` under real throttling. Heavy dependencies (Dexie, SheetJS, ONNX Runtime) load lazily. -- **Quality bar.** 883 unit tests and 123 Playwright end-to-end tests across five +- **Quality bar.** 889 unit tests and 123 Playwright end-to-end tests across five projects — desktop, a phone viewport in English and in French, dark mode, and Cloudflare Pages' own routing emulated by `wrangler pages dev` (a real 404, the security headers as served) — covering offline PWA, a fake webcam, a horizontal-overflow guard on diff --git a/docs/superpowers/plans/2026-10-09-v48-share-links.md b/docs/superpowers/plans/2026-10-09-v48-share-links.md new file mode 100644 index 0000000..2568227 --- /dev/null +++ b/docs/superpowers/plans/2026-10-09-v48-share-links.md @@ -0,0 +1,58 @@ +# V48: a damaged share link ends on a refusal, never on a crash + +## Why + +The post-V46 audit found that `isRenderable` checked `insights` was an object +and `RunView` then called `insights.importance.slice`: a link carrying +`insights: {}` passed the gate and crashed the page. The plan row proposed +validating that field and a few others. + +## Measure first + +Three real share links were captured from the running app and decoded: + +| Link | Run | Analyses carried | +| ------- | ------------------- | ------------------------------------------------ | +| titanic | survived, binary | uncertainty, threshold, segments | +| iris | species, multiclass | uncertainty, threshold, k-NN tuning, exploration | +| mpg | mpg, regression | uncertainty, segments | + +Every field (first two items of each array) was deleted, nulled, or replaced by +a string, a number, an object or an array, and each result was rendered +through the real `SharedRunPage`: + +| Damaged links | Accepted by the decoder | Crashed the page | Distinct paths | +| ------------- | ----------------------- | ---------------- | -------------- | +| 3 948 | 3 660 | 1 020 | 99 | + +The paths run from `insights.importance` to +`artifacts.exploration.clusters[].traits[].clusterMean`. A validator covering +them would be the largest file in the share module and would drift every time +a run learns to carry a new analysis. + +## Decision + +- The decoder keeps checking what every view reads first (V35). +- `SharedRunPage` wraps the run in a render guard. Any failure while drawing it + shows the refusal page with its own explanation, and nothing from the run. + Showing the readable half was rejected: a half-read run lets a damaged figure + pass for a real one. +- The measurement becomes a permanent test over the committed links + (`tests/share-links/`). Each field shape is damaged six ways once, in the + first link that carries it. It fails on the code before this wave. + +Cost: the test adds about 25 s to the unit suite (66 s to 90 s on four cores). +Sampling fewer shapes would have been faster and would have reopened the gap +the wave closes. + +## Also in this wave + +The five copies of the download helper become `src/lib/download.ts`. The +object URL is revoked after the click has been handled rather than in the same +tick, where some engines cancel the download without a word. + +## What this wave deliberately does not do + +Write a field-by-field schema for every artifact, render a partially readable +run, or guard event handlers: a click on a damaged run is not reachable because +the refusal replaces it first. diff --git a/package-lock.json b/package-lock.json index 95bcac9..56ab55d 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "labml", - "version": "1.47.0", + "version": "1.48.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "labml", - "version": "1.47.0", + "version": "1.48.0", "license": "MIT", "dependencies": { "@duckdb/duckdb-wasm": "1.28.0", diff --git a/package.json b/package.json index 311fd88..d18fbea 100644 --- a/package.json +++ b/package.json @@ -1,7 +1,7 @@ { "name": "labml", "private": true, - "version": "1.47.0", + "version": "1.48.0", "license": "MIT", "type": "module", "engines": { diff --git a/src/features/data/SqlPanel.tsx b/src/features/data/SqlPanel.tsx index 283d09a..edcc957 100644 --- a/src/features/data/SqlPanel.tsx +++ b/src/features/data/SqlPanel.tsx @@ -9,6 +9,7 @@ import { useDataStore } from '@/features/data/data-store'; import { BUNDLE_BYTES, type SqlEngine } from '@/features/data/sql/engine'; import { readerFor, tableNameFor, tableToCsv, type SqlTable } from '@/features/data/sql/table'; import { ScrollRegion } from '@/components/ui/scroll-region'; +import { downloadFile } from '@/lib/download'; /** Rows painted at once. The count above the table always tells the truth. */ const DISPLAY_CAP = 200; @@ -141,13 +142,7 @@ export function SqlPanel() { const download = useCallback(() => { if (!result) return; - const blob = new Blob([tableToCsv(result)], { type: 'text/csv' }); - const url = URL.createObjectURL(blob); - const anchor = document.createElement('a'); - anchor.href = url; - anchor.download = 'query.csv'; - anchor.click(); - URL.revokeObjectURL(url); + downloadFile('query.csv', tableToCsv(result), 'text/csv'); }, [result]); const toLab = useCallback(() => { diff --git a/src/features/data/components/DataExportBar.tsx b/src/features/data/components/DataExportBar.tsx index 85e305a..f638fc4 100644 --- a/src/features/data/components/DataExportBar.tsx +++ b/src/features/data/components/DataExportBar.tsx @@ -5,15 +5,7 @@ import { useNavigate } from 'react-router'; import { Button } from '@/components/ui/button'; import { useDataStore } from '@/features/data/data-store'; import { useLabStore } from '@/features/ml/lab-store'; - -function downloadFile(name: string, mime: string, content: string) { - const url = URL.createObjectURL(new Blob([content], { type: mime })); - const anchor = document.createElement('a'); - anchor.href = url; - anchor.download = name; - anchor.click(); - URL.revokeObjectURL(url); -} +import { downloadFile } from '@/lib/download'; /** Exports of the cleaned dataset — generated locally, like everything else. */ export function DataExportBar() { @@ -30,7 +22,7 @@ export function DataExportBar() { useEffect(() => { if (!exportedFile) return; - downloadFile(exportedFile.name, exportedFile.mime, exportedFile.content); + downloadFile(exportedFile.name, exportedFile.content, exportedFile.mime); clearExportedFile(); }, [exportedFile, clearExportedFile]); diff --git a/src/features/ml/components/BatchScorePanel.tsx b/src/features/ml/components/BatchScorePanel.tsx index adf1145..65d194e 100644 --- a/src/features/ml/components/BatchScorePanel.tsx +++ b/src/features/ml/components/BatchScorePanel.tsx @@ -7,15 +7,7 @@ import { Eyebrow } from '@/components/ui/eyebrow'; import { useLabStore } from '@/features/ml/lab-store'; import { METRIC_ROWS, metricDelta } from '@/features/ml/train/score-view'; import { ScrollRegion } from '@/components/ui/scroll-region'; - -function downloadCsv(name: string, content: string) { - const url = URL.createObjectURL(new Blob([content], { type: 'text/csv' })); - const anchor = document.createElement('a'); - anchor.href = url; - anchor.download = name; - anchor.click(); - URL.revokeObjectURL(url); -} +import { downloadFile } from '@/lib/download'; /** The production gesture: score a NEW file with the inspected model. */ export function BatchScorePanel() { @@ -177,7 +169,11 @@ export function BatchScorePanel() { variant="outline" size="sm" onClick={() => - downloadCsv(result.fileName.replace(/\.[a-z]+$/i, '') + '-scored.csv', result.csv) + downloadFile( + result.fileName.replace(/\.[a-z]+$/i, '') + '-scored.csv', + result.csv, + 'text/csv', + ) } > {t('ml.lab.batch.downloadCsv')} diff --git a/src/features/ml/components/ExportBar.tsx b/src/features/ml/components/ExportBar.tsx index 7f458c8..6e0493f 100644 --- a/src/features/ml/components/ExportBar.tsx +++ b/src/features/ml/components/ExportBar.tsx @@ -5,15 +5,7 @@ import { Button } from '@/components/ui/button'; import { buildReportHtml } from '@/features/ml/projects/report'; import { encodeShareFragment } from '@/features/ml/projects/share'; import { useLabStore } from '@/features/ml/lab-store'; - -function downloadFile(name: string, mime: string, content: string) { - const url = URL.createObjectURL(new Blob([content], { type: mime })); - const anchor = document.createElement('a'); - anchor.href = url; - anchor.download = name; - anchor.click(); - URL.revokeObjectURL(url); -} +import { downloadFile } from '@/lib/download'; /** Export & share actions for the completed run — everything generated locally. */ export function ExportBar() { @@ -28,7 +20,7 @@ export function ExportBar() { useEffect(() => { if (!exportedFile) return; - downloadFile(exportedFile.name, exportedFile.mime, exportedFile.content); + downloadFile(exportedFile.name, exportedFile.content, exportedFile.mime); clearExportedFile(); }, [exportedFile, clearExportedFile]); @@ -45,8 +37,8 @@ export function ExportBar() { if (!currentRun) return; downloadFile( `labml-${currentRun.dataset.name.replace(/\.[a-z]+$/i, '')}-report.html`, - 'text/html', buildReportHtml(currentRun, t, lang), + 'text/html', ); } diff --git a/src/features/ml/components/ImportModelPanel.tsx b/src/features/ml/components/ImportModelPanel.tsx index 21617c4..bad6ada 100644 --- a/src/features/ml/components/ImportModelPanel.tsx +++ b/src/features/ml/components/ImportModelPanel.tsx @@ -6,15 +6,7 @@ import { Eyebrow } from '@/components/ui/eyebrow'; import { METRIC_ROWS, metricDelta } from '@/features/ml/train/score-view'; import { useLabStore } from '@/features/ml/lab-store'; import { cn } from '@/lib/utils'; - -function downloadCsv(content: string, name: string) { - const url = URL.createObjectURL(new Blob([content], { type: 'text/csv' })); - const anchor = document.createElement('a'); - anchor.href = url; - anchor.download = name; - anchor.click(); - URL.revokeObjectURL(url); -} +import { downloadFile } from '@/lib/download'; /** Maps a named worker refusal to its translation key (suffixes stripped). */ function errorKey(message: string): { key: string; detail?: string } { @@ -217,7 +209,9 @@ export function ImportModelPanel() { diff --git a/src/features/ml/pages/SharedRunPage.test.tsx b/src/features/ml/pages/SharedRunPage.test.tsx new file mode 100644 index 0000000..e775e2a --- /dev/null +++ b/src/features/ml/pages/SharedRunPage.test.tsx @@ -0,0 +1,119 @@ +import { cleanup, render, screen } from '@testing-library/react'; +import { compressToEncodedURIComponent } from 'lz-string'; +import { MemoryRouter } from 'react-router'; +import { afterEach, beforeAll, beforeEach, describe, expect, it, vi } from 'vitest'; +import iris from '../../../../tests/share-links/iris.json'; +import mpg from '../../../../tests/share-links/mpg.json'; +import titanic from '../../../../tests/share-links/titanic.json'; +import { SharedRunPage } from '@/features/ml/pages/SharedRunPage'; +import i18n from '@/lib/i18n'; + +/** + * V48 — a share link is input from a stranger, and the page must end on the + * run or on a refusal, never on a crash. Before V48, deleting, nulling or + * retyping single fields of three real links produced 3 660 links the decoder + * accepted, and 1 020 of them crashed the page on 99 distinct paths. + */ +const LINKS = { titanic, iris, mpg } as const; + +type Path = (string | number)[]; + +const MUTATIONS: [string, unknown][] = [ + ['delete', undefined], + ['null', null], + ['string', 'x'], + ['number', 1], + ['object', {}], + ['array', []], +]; + +/** Every field of the payload, through the first item of each array. */ +function paths(node: unknown, prefix: Path = [], out: Path[] = []): Path[] { + if (prefix.length > 0) out.push(prefix); + if (Array.isArray(node)) node.slice(0, 1).forEach((v, i) => paths(v, [...prefix, i], out)); + else if (node && typeof node === 'object') { + for (const [key, value] of Object.entries(node)) paths(value, [...prefix, key], out); + } + return out; +} + +function mutate(root: unknown, path: Path, kind: string, value: unknown): unknown { + const copy = structuredClone(root); + let node = copy as Record; + for (const key of path.slice(0, -1)) node = node[key] as Record; + const last = path[path.length - 1]; + if (kind !== 'delete') node[last] = value; + else if (Array.isArray(node)) node.splice(last as number, 1); + else delete node[last]; + return copy; +} + +function openLink(payload: unknown) { + window.location.hash = compressToEncodedURIComponent(JSON.stringify(payload)); + return render( + + + , + ); +} + +beforeAll(async () => { + await i18n.changeLanguage('en'); +}); + +beforeEach(() => { + // React reports every error a boundary catches; hundreds are expected here. + vi.spyOn(console, 'error').mockImplementation(() => undefined); +}); + +afterEach(() => { + cleanup(); + vi.restoreAllMocks(); +}); + +describe('SharedRunPage', () => { + it.each(Object.entries(LINKS))('renders the real %s link', (_name, payload) => { + openLink(payload); + expect(screen.getByTestId('run-view')).toBeInTheDocument(); + }); + + it('refuses, with its own explanation, a link that decodes but cannot be drawn', () => { + // The audit's example: `insights: {}` passed the decoder, then + // `importance.slice` crashed the page. + openLink({ ...titanic, insights: {} }); + expect(screen.queryByTestId('run-view')).not.toBeInTheDocument(); + expect(screen.getByTestId('share-refused')).toHaveTextContent( + 'does not have the shape LabML writes', + ); + }); + + it('ends every damaged link on the run or on a refusal, never on a crash', () => { + // Each field shape is damaged once, in the first link that carries it: + // the three links share most of their structure, and rendering the same + // damage three times tests nothing new. + const seen = new Set(); + const crashes: string[] = []; + for (const [name, payload] of Object.entries(LINKS)) { + for (const path of paths(payload)) { + const shape = path.map((key) => (typeof key === 'number' ? '[]' : key)).join('.'); + if (seen.has(shape)) continue; + seen.add(shape); + for (const [kind, value] of MUTATIONS) { + try { + openLink(mutate(payload, path, kind, value)); + const shown = + screen.queryByTestId('run-view') !== null || + screen.queryByTestId('share-refused') !== null; + if (!shown) crashes.push(`${name}: ${shape} (${kind}): blank page`); + } catch (error) { + crashes.push(`${name}: ${shape} (${kind}): ${(error as Error).message}`); + } finally { + cleanup(); + } + } + } + } + expect(seen.size).toBeGreaterThan(100); + expect(crashes).toEqual([]); + }, 120_000); +}); diff --git a/src/features/ml/pages/SharedRunPage.tsx b/src/features/ml/pages/SharedRunPage.tsx index f43c707..5faa3b4 100644 --- a/src/features/ml/pages/SharedRunPage.tsx +++ b/src/features/ml/pages/SharedRunPage.tsx @@ -1,5 +1,5 @@ import { ArrowLeft, ShieldCheck } from 'lucide-react'; -import { useState } from 'react'; +import { Component, useState, type ReactNode } from 'react'; import { useTranslation } from 'react-i18next'; import { Link } from 'react-router'; import { buttonVariants } from '@/components/ui/button'; @@ -7,6 +7,62 @@ import { RunView } from '@/features/ml/components/RunView'; import { decodeShareFragment } from '@/features/ml/projects/share'; import { cn } from '@/lib/utils'; +/** + * V48 — the last line of defence for a link that decodes but cannot be drawn. + * + * `decodeShareFragment` checks what every view reads first. It cannot check + * everything: a run carries up to nine analyses, each with its own nested + * shape, and a validator for all of them would drift every time an analysis + * is added. Measured on three real links (titanic, iris with tuning and + * groups, mpg) and every field of them deleted, nulled or retyped: 3 660 of + * 3 948 crafted links passed the decoder and 1 020 of those crashed the page, + * on 99 distinct paths. Rather than chase them one by one, a render failure + * inside the run is caught here and turned into the same refusal the decoder + * gives, with its own explanation. Nothing from the run is shown: a page + * that rendered half a run would let a damaged figure pass for a real one. + */ +class ShareRenderGuard extends Component< + { children: ReactNode; fallback: ReactNode }, + { failed: boolean } +> { + state = { failed: false }; + + static getDerivedStateFromError() { + return { failed: true }; + } + + render() { + return this.state.failed ? this.props.fallback : this.props.children; + } +} + +function InvalidShare({ reason }: { reason: 'invalid' | 'unreadable' }) { + const { t } = useTranslation(); + /* + V35 wave 4 — the refusal is a page, not a stray sentence. + + Looking at all 17 routes at three widths turned up exactly one with + no `

` at all: this branch, which rendered a single grey line into + an otherwise empty page. It is also the branch that now runs far more + often, because `decodeShareFragment` stopped accepting payloads it + could not render — before this wave those reached `RunView` and left + a white page instead. + + The most common way to land here is not an attacker: it is a real + share link that a chat client cut in half. So the page says which + part is missing and what to do about it. + */ + return ( +
+

{t('ml.lab.share.invalidTitle')}

+

{t('ml.lab.share.invalid')}

+

+ {t(reason === 'unreadable' ? 'ml.lab.share.unreadable' : 'ml.lab.share.invalidHelp')} +

+
+ ); +} + export function SharedRunPage() { const { t } = useTranslation(); // The payload travels in the URL fragment — never sent to any server. @@ -25,27 +81,11 @@ export function SharedRunPage() {

{payload ? ( - + }> + + ) : ( - /* - V35 wave 4 — the refusal is a page, not a stray sentence. - - Looking at all 17 routes at three widths turned up exactly one with - no `

` at all: this branch, which rendered a single grey line into - an otherwise empty page. It is also the branch that now runs far more - often, because `decodeShareFragment` stopped accepting payloads it - could not render — before this wave those reached `RunView` and left - a white page instead. - - The most common way to land here is not an attacker: it is a real - share link that a chat client cut in half. So the page says which - part is missing and what to do about it. - */ -
-

{t('ml.lab.share.invalidTitle')}

-

{t('ml.lab.share.invalid')}

-

{t('ml.lab.share.invalidHelp')}

-
+ )} ); diff --git a/src/lib/download.test.ts b/src/lib/download.test.ts new file mode 100644 index 0000000..58c4002 --- /dev/null +++ b/src/lib/download.test.ts @@ -0,0 +1,32 @@ +import { afterEach, describe, expect, it, vi } from 'vitest'; +import { REVOKE_DELAY_MS, downloadFile } from './download'; + +afterEach(() => { + vi.useRealTimers(); + vi.restoreAllMocks(); +}); + +describe('downloadFile', () => { + it('keeps the object URL alive until the click has been handled', () => { + vi.useFakeTimers(); + // jsdom implements neither half of the object-URL API. + const create = vi.fn(() => 'blob:labml/1'); + const revoke = vi.fn(); + Object.assign(URL, { createObjectURL: create, revokeObjectURL: revoke }); + const click = vi + .spyOn(HTMLAnchorElement.prototype, 'click') + .mockImplementation(() => undefined); + + downloadFile('query.csv', 'a,b\n1,2\n', 'text/csv'); + + expect(click).toHaveBeenCalledTimes(1); + const anchor = click.mock.contexts[0] as HTMLAnchorElement; + expect(anchor.download).toBe('query.csv'); + expect(anchor.href).toBe('blob:labml/1'); + // Revoked in the same tick, some engines cancel the download silently. + expect(revoke).not.toHaveBeenCalled(); + + vi.advanceTimersByTime(REVOKE_DELAY_MS); + expect(revoke).toHaveBeenCalledWith('blob:labml/1'); + }); +}); diff --git a/src/lib/download.ts b/src/lib/download.ts new file mode 100644 index 0000000..50467b6 --- /dev/null +++ b/src/lib/download.ts @@ -0,0 +1,19 @@ +/** + * V48 — one way to hand the visitor a file built in the tab. + * + * Five components each carried their own copy of this, and every copy revoked + * the object URL in the same tick as `click()`. Chromium starts the download + * synchronously and does not mind; other engines resolve the URL a little + * later and can find it already revoked, which cancels the download without a + * word. The URL is now released after the click has been handled. + */ +export const REVOKE_DELAY_MS = 1_000; + +export function downloadFile(name: string, content: BlobPart, mime: string): void { + const url = URL.createObjectURL(new Blob([content], { type: mime })); + const anchor = document.createElement('a'); + anchor.href = url; + anchor.download = name; + anchor.click(); + setTimeout(() => URL.revokeObjectURL(url), REVOKE_DELAY_MS); +} diff --git a/src/locales/en.json b/src/locales/en.json index 8df5a96..456cc3d 100644 --- a/src/locales/en.json +++ b/src/locales/en.json @@ -690,7 +690,8 @@ "note": "This link carries only metrics and charts — never the original data. It lives in the URL fragment, which browsers do not send to any server.", "invalid": "Invalid or incomplete share link.", "invalidTitle": "This link cannot be opened", - "invalidHelp": "A share link carries the whole run in the URL fragment, after the « # ». If it was cut short — by a chat client, a line break, a partial copy-paste — there is not enough left to rebuild the run. Ask whoever sent it for the complete link." + "invalidHelp": "A share link carries the whole run in the URL fragment, after the « # ». If it was cut short — by a chat client, a line break, a partial copy-paste — there is not enough left to rebuild the run. Ask whoever sent it for the complete link.", + "unreadable": "The link decoded, but part of the run inside it does not have the shape LabML writes: most often a link edited by hand, or damaged somewhere in a way that still decompresses. Nothing from it is shown, so no half-read figure can pass for a real one." }, "reportFooter": "Generated locally by LabML — no data ever left the browser.", "splitMode": { diff --git a/src/locales/fr.json b/src/locales/fr.json index 61676d8..3bda01d 100644 --- a/src/locales/fr.json +++ b/src/locales/fr.json @@ -690,7 +690,8 @@ "note": "Ce lien ne transporte que les métriques et graphiques — jamais les données d'origine. Il vit dans le fragment d'URL, que les navigateurs n'envoient à aucun serveur.", "invalid": "Lien de partage invalide ou incomplet.", "invalidTitle": "Ce lien ne peut pas être ouvert", - "invalidHelp": "Un lien de partage porte tout le run dans le fragment d'URL, après le « # ». S'il a été coupé — par une messagerie, un retour à la ligne, un copier-coller partiel — il ne reste pas assez pour reconstruire le run. Redemandez le lien complet à la personne qui vous l'a envoyé." + "invalidHelp": "Un lien de partage porte tout le run dans le fragment d'URL, après le « # ». S'il a été coupé — par une messagerie, un retour à la ligne, un copier-coller partiel — il ne reste pas assez pour reconstruire le run. Redemandez le lien complet à la personne qui vous l'a envoyé.", + "unreadable": "Le lien s'est décodé, mais une partie du run qu'il contient n'a pas la forme qu'écrit LabML : le plus souvent un lien modifié à la main, ou abîmé d'une façon qui se décompresse encore. Rien n'en est affiché, pour qu'aucun chiffre lu à moitié ne passe pour un vrai." }, "reportFooter": "Généré localement par LabML — aucune donnée n'a quitté le navigateur.", "splitMode": { diff --git a/tests/share-links/README.md b/tests/share-links/README.md new file mode 100644 index 0000000..d18aaed --- /dev/null +++ b/tests/share-links/README.md @@ -0,0 +1,19 @@ +# Real share links + +Three share payloads captured from the running app on 09/10/2026 (V48), decoded +from the `#` fragment that « Copy share link » produced: + +| File | Run | Analyses carried | +| -------------- | --------------------------- | ------------------------------------------------ | +| `titanic.json` | titanic · survived (binary) | uncertainty, threshold, segments | +| `iris.json` | iris · species (multiclass) | uncertainty, threshold, k-NN tuning, exploration | +| `mpg.json` | mpg · mpg (regression) | uncertainty, segments | + +They contain metrics and charts only, never rows of data, exactly like the +links themselves. `src/features/ml/pages/SharedRunPage.test.tsx` deletes, +nulls and retypes every field of each one and asserts that the page always +ends on the run or on a refusal, never on a crash. + +To refresh them after a format change, train the same three demos, copy each +share link, and decode the fragment with `lz-string`'s +`decompressFromEncodedURIComponent`. diff --git a/tests/share-links/iris.json b/tests/share-links/iris.json new file mode 100644 index 0000000..727019c --- /dev/null +++ b/tests/share-links/iris.json @@ -0,0 +1,752 @@ +{ + "v": 2, + "name": "iris · species", + "createdAt": 1791510834045, + "dataset": { + "name": "iris.csv", + "rowCount": 150, + "columnCount": 5 + }, + "target": "species", + "taskType": "multiclass", + "seed": 42, + "results": [ + { + "key": "baseline", + "ok": true, + "metrics": { + "accuracy": 0.3333333333333333, + "precision": 0.1111111111111111, + "recall": 0.3333333333333333, + "f1": 0.16666666666666666, + "logLoss": 1.0986122886681102 + }, + "primary": 0.3333333333333333, + "trainMs": 0.2000000000698492, + "inferP50Ms": 0, + "inferP95Ms": 0, + "trainedRows": 96, + "valMetrics": { + "accuracy": 0.3333333333333333, + "precision": 0.1111111111111111, + "recall": 0.3333333333333333, + "f1": 0.16666666666666666, + "logLoss": 1.09861228866811 + }, + "valPrimary": 0.3333333333333333 + }, + { + "key": "logistic", + "ok": true, + "metrics": { + "accuracy": 1, + "precision": 1, + "recall": 1, + "f1": 1, + "logLoss": 0.22384000271990642 + }, + "primary": 1, + "trainMs": 78.19999999995343, + "inferP50Ms": 0, + "inferP95Ms": 0.09999999997671694, + "trainedRows": 96, + "valMetrics": { + "accuracy": 1, + "precision": 1, + "recall": 1, + "f1": 1, + "logLoss": 0.2098850088987609 + }, + "valPrimary": 1 + }, + { + "key": "knn", + "ok": true, + "metrics": { + "accuracy": 0.9666666666666667, + "precision": 0.9696969696969697, + "recall": 0.9666666666666667, + "f1": 0.9665831244778613, + "logLoss": 0.06773361628150759 + }, + "primary": 0.9666666666666667, + "trainMs": 0.09999999997671694, + "inferP50Ms": 0, + "inferP95Ms": 0, + "trainedRows": 96, + "valMetrics": { + "accuracy": 0.9583333333333334, + "precision": 0.9629629629629629, + "recall": 0.9583333333333334, + "f1": 0.9581699346405229, + "logLoss": 0.11623725792261408 + }, + "valPrimary": 0.9583333333333334 + }, + { + "key": "naiveBayes", + "ok": true, + "metrics": { + "accuracy": 0.9666666666666667, + "precision": 0.9696969696969697, + "recall": 0.9666666666666667, + "f1": 0.9665831244778613, + "logLoss": 0.06089483024701834 + }, + "primary": 0.9666666666666667, + "trainMs": 0.39999999990686774, + "inferP50Ms": 0, + "inferP95Ms": 0.09999999997671694, + "trainedRows": 96, + "valMetrics": { + "accuracy": 1, + "precision": 1, + "recall": 1, + "f1": 1, + "logLoss": 0.01259259294744664 + }, + "valPrimary": 1 + }, + { + "key": "tree", + "ok": true, + "metrics": { + "accuracy": 0.9333333333333333, + "precision": 0.9333333333333332, + "recall": 0.9333333333333332, + "f1": 0.9333333333333332 + }, + "primary": 0.9333333333333333, + "trainMs": 16.199999999953434, + "inferP50Ms": 0, + "inferP95Ms": 0.19999999995343387, + "trainedRows": 96, + "valMetrics": { + "accuracy": 1, + "precision": 1, + "recall": 1, + "f1": 1 + }, + "valPrimary": 1 + }, + { + "key": "forest", + "ok": true, + "metrics": { + "accuracy": 0.9333333333333333, + "precision": 0.9444444444444445, + "recall": 0.9333333333333332, + "f1": 0.9326599326599326 + }, + "primary": 0.9333333333333333, + "trainMs": 182.90000000002328, + "inferP50Ms": 0.09999999997671694, + "inferP95Ms": 0.19999999995343387, + "trainedRows": 96, + "valMetrics": { + "accuracy": 1, + "precision": 1, + "recall": 1, + "f1": 1 + }, + "valPrimary": 1 + }, + { + "key": "gbdt", + "ok": true, + "metrics": { + "accuracy": 0.9333333333333333, + "precision": 0.9444444444444445, + "recall": 0.9333333333333332, + "f1": 0.9326599326599326, + "logLoss": 0.1162702139744338 + }, + "primary": 0.9333333333333333, + "trainMs": 54.40000000002328, + "inferP50Ms": 0, + "inferP95Ms": 0.2999999999301508, + "trainedRows": 96, + "valMetrics": { + "accuracy": 0.9583333333333334, + "precision": 0.9629629629629629, + "recall": 0.9583333333333334, + "f1": 0.9581699346405229, + "logLoss": 0.09820758906505994 + }, + "valPrimary": 0.9583333333333334 + }, + { + "key": "mlp", + "ok": true, + "metrics": { + "accuracy": 0.9333333333333333, + "precision": 0.9444444444444445, + "recall": 0.9333333333333332, + "f1": 0.9326599326599326, + "logLoss": 0.07680106256835449 + }, + "primary": 0.9333333333333333, + "trainMs": 95.19999999995343, + "inferP50Ms": 0, + "inferP95Ms": 0.10000000009313226, + "trainedRows": 96, + "valMetrics": { + "accuracy": 1, + "precision": 1, + "recall": 1, + "f1": 1, + "logLoss": 0.033122106806359644 + }, + "valPrimary": 1 + }, + { + "key": "ensemble", + "ok": true, + "metrics": { + "accuracy": 0.9666666666666667, + "precision": 0.9696969696969697, + "recall": 0.9666666666666667, + "f1": 0.9665831244778613, + "logLoss": 0.11004045125586442 + }, + "primary": 0.9666666666666667, + "trainMs": 0.09999999997671694, + "inferP50Ms": 0, + "inferP95Ms": 0.09999999997671694, + "valMetrics": { + "accuracy": 1, + "precision": 1, + "recall": 1, + "f1": 1, + "logLoss": 0.07778426283162687 + }, + "valPrimary": 1 + } + ], + "summary": { + "task": { + "type": "multiclass", + "classes": ["setosa", "versicolor", "virginica"] + }, + "taskType": "multiclass", + "seed": 42, + "trainRows": 96, + "testRows": 30, + "featureCount": 4, + "featureColumns": ["sepal_length", "sepal_width", "petal_length", "petal_width"], + "skippedColumns": [], + "totalMs": 77.09999999997672, + "validationRows": 24, + "leakWarnings": [ + { + "column": "petal_width", + "score": 1 + } + ], + "majorityShare": 0.3333333333333333, + "imbalanced": false, + "ensemble": { + "members": ["logistic", "mlp", "naiveBayes"], + "method": "probability" + }, + "parallel": { + "helpers": 3, + "families": ["logistic", "tree", "mlp", "forest", "gbdt"] + } + }, + "insights": { + "model": "ensemble", + "classes": ["setosa", "versicolor", "virginica"], + "confusion": [ + [10, 0, 0], + [0, 10, 0], + [0, 1, 9] + ], + "importance": [ + { + "column": "petal_length", + "value": 0.2888888888888889 + }, + { + "column": "petal_width", + "value": 0.2333333333333334 + }, + { + "column": "sepal_width", + "value": 0.011111111111111108 + }, + { + "column": "sepal_length", + "value": 0 + } + ] + }, + "artifacts": { + "uncertainty": { + "isClassification": true, + "metricLabel": "accuracy", + "testRows": 30, + "resamples": 1000, + "seed": 42, + "intervals": [ + { + "model": "logistic", + "point": 1, + "lo": 1, + "hi": 1 + }, + { + "model": "ensemble", + "point": 0.9666666666666667, + "lo": 0.9, + "hi": 1 + }, + { + "model": "knn", + "point": 0.9666666666666667, + "lo": 0.9, + "hi": 1 + }, + { + "model": "naiveBayes", + "point": 0.9666666666666667, + "lo": 0.9, + "hi": 1 + }, + { + "model": "forest", + "point": 0.9333333333333333, + "lo": 0.8333333333333334, + "hi": 1 + }, + { + "model": "gbdt", + "point": 0.9333333333333333, + "lo": 0.8333333333333334, + "hi": 1 + }, + { + "model": "mlp", + "point": 0.9333333333333333, + "lo": 0.8333333333333334, + "hi": 1 + }, + { + "model": "tree", + "point": 0.9333333333333333, + "lo": 0.8333333333333334, + "hi": 1 + }, + { + "model": "baseline", + "point": 0.3333333333333333, + "lo": 0.16666666666666666, + "hi": 0.5 + } + ], + "verdict": { + "winner": "logistic", + "against": "baseline", + "delta": 0.6666666666666667, + "lo": 0.5, + "hi": 0.8333333333333334, + "winShare": 1, + "decisive": true + } + }, + "threshold": { + "model": "ensemble", + "positiveClass": "setosa", + "positiveRate": 0.3333333333333333, + "averagePrecision": 1, + "brier": 0.00012891718303245826, + "prPoints": [ + { + "recall": 0.1, + "precision": 1, + "threshold": 0.9989841762694955 + }, + { + "recall": 0.2, + "precision": 1, + "threshold": 0.9989736437992863 + }, + { + "recall": 0.3, + "precision": 1, + "threshold": 0.9953509958341368 + }, + { + "recall": 0.4, + "precision": 1, + "threshold": 0.9927367134910309 + }, + { + "recall": 0.5, + "precision": 1, + "threshold": 0.9924251252953539 + }, + { + "recall": 0.6, + "precision": 1, + "threshold": 0.9922771028656922 + }, + { + "recall": 0.7, + "precision": 1, + "threshold": 0.991985464200722 + }, + { + "recall": 0.8, + "precision": 1, + "threshold": 0.9904926969383044 + }, + { + "recall": 0.9, + "precision": 1, + "threshold": 0.990377242411173 + }, + { + "recall": 1, + "precision": 1, + "threshold": 0.9898382434212136 + }, + { + "recall": 1, + "precision": 0.9090909090909091, + "threshold": 0.04823616632544208 + }, + { + "recall": 1, + "precision": 0.8333333333333334, + "threshold": 0.018663345438208388 + }, + { + "recall": 1, + "precision": 0.7692307692307693, + "threshold": 0.01761508482705448 + }, + { + "recall": 1, + "precision": 0.7142857142857143, + "threshold": 0.011625630156225197 + }, + { + "recall": 1, + "precision": 0.6666666666666666, + "threshold": 0.008484905256572714 + }, + { + "recall": 1, + "precision": 0.625, + "threshold": 0.007743118775509016 + }, + { + "recall": 1, + "precision": 0.5882352941176471, + "threshold": 0.006452325263073283 + }, + { + "recall": 1, + "precision": 0.5555555555555556, + "threshold": 0.004385589637704831 + }, + { + "recall": 1, + "precision": 0.5263157894736842, + "threshold": 0.0019184572570196694 + }, + { + "recall": 1, + "precision": 0.5, + "threshold": 0.0016523838776071637 + }, + { + "recall": 1, + "precision": 0.47619047619047616, + "threshold": 0.0013227311234399036 + }, + { + "recall": 1, + "precision": 0.45454545454545453, + "threshold": 0.0009373267148303778 + }, + { + "recall": 1, + "precision": 0.43478260869565216, + "threshold": 0.0008237030139886186 + }, + { + "recall": 1, + "precision": 0.4166666666666667, + "threshold": 0.0005160281844922815 + }, + { + "recall": 1, + "precision": 0.4, + "threshold": 0.0003654544984385398 + }, + { + "recall": 1, + "precision": 0.38461538461538464, + "threshold": 0.0002910877535896989 + }, + { + "recall": 1, + "precision": 0.37037037037037035, + "threshold": 0.00014853431352716292 + }, + { + "recall": 1, + "precision": 0.35714285714285715, + "threshold": 0.00012966204797255017 + }, + { + "recall": 1, + "precision": 0.3448275862068966, + "threshold": 0.00009078260402987172 + }, + { + "recall": 1, + "precision": 0.3333333333333333, + "threshold": 0.000008230903321906275 + } + ], + "calibrationBins": [ + { + "meanPredicted": 0.006570527398602384, + "observedRate": 0, + "count": 20 + }, + { + "meanPredicted": 0.9933441404526407, + "observedRate": 1, + "count": 10 + } + ], + "chosen": { + "threshold": 0.5, + "tp": 10, + "fp": 0, + "fn": 0, + "tn": 20, + "precision": 1, + "recall": 1, + "f1": 1, + "accuracy": 1, + "cost": 0, + "costFp": 1, + "costFn": 1 + } + }, + "tuning": { + "model": "knn", + "isClassification": true, + "folds": 3, + "budget": 6, + "bestParams": { + "k": 1 + }, + "bestCv": 0.9583333333333334, + "defaultPrimary": 0.9666666666666667, + "tunedPrimary": 0.9333333333333333, + "tunedMetrics": { + "accuracy": 0.9333333333333333, + "precision": 0.9333333333333332, + "recall": 0.9333333333333332, + "f1": 0.9333333333333332, + "logLoss": 2.3025850929940463 + }, + "trials": [ + { + "params": { + "k": 1 + }, + "cvScore": 0.9583333333333334 + }, + { + "params": { + "k": 3 + }, + "cvScore": 0.9583333333333334 + }, + { + "params": { + "k": 5 + }, + "cvScore": 0.9479166666666666 + }, + { + "params": { + "k": 9 + }, + "cvScore": 0.9166666666666666 + }, + { + "params": { + "k": 15 + }, + "cvScore": 0.8854166666666666 + } + ], + "totalMs": 56.40000000002328, + "trainedRows": 96 + }, + "exploration": { + "k": 2, + "silhouette": 0.5817500491982808, + "tried": [ + { + "k": 2, + "silhouette": 0.5817500491982808 + }, + { + "k": 3, + "silhouette": 0.45653522558312604 + }, + { + "k": 4, + "silhouette": 0.4188923398171003 + }, + { + "k": 5, + "silhouette": 0.34503345350274617 + } + ], + "points": [ + [-2.265, 0.48, 1], + [-2.364, -0.342, 1], + [-2.39, 0.647, 1], + [-2.444, 0.048, 1], + [-2.335, -1.115, 1], + [-2.166, 1.044, 1], + [-2.218, -0.729, 1], + [-2.199, 1.86, 1], + [-2.208, 1.484, 1], + [-1.899, 1.405, 1], + [-1.914, 0.409, 1], + [-2.774, 0.458, 1], + [-2.227, 0.137, 1], + [-2.051, 0.242, 1], + [-2.14, 0.313, 1], + [-2.14, -0.505, 1], + [-2.615, 1.794, 1], + [-2.11, -0.46, 1], + [-2.045, 0.662, 1], + [-2.43, -0.904, 1], + [-2.286, 0.442, 1], + [-2.554, -0.479, 1], + [-2.137, 1.142, 1], + [-2.385, 1.12, 1], + [-2.229, 0.998, 1], + [1.102, 0.863, 0], + [1.241, 0.616, 0], + [1.075, -0.208, 0], + [0.747, 0.773, 0], + [0.928, 0.032, 0], + [-0.11, -2.654, 0], + [0.562, -1.765, 0], + [-0.033, -0.439, 0], + [0.35, -0.196, 0], + [1.225, -1.622, 0], + [0.738, 0.397, 0], + [1.234, -0.933, 0], + [0.703, -0.063, 0], + [1.257, -0.077, 0], + [0.665, -0.226, 0], + [0.131, -1.562, 0], + [0.242, -0.777, 0], + [0.224, -0.288, 0], + [1.049, 0.522, 0], + [0.07, -0.22, 0], + [0.279, -1.12, 0], + [0.337, -0.988, 0], + [0.289, -0.856, 0], + [0.228, -0.385, 0], + [-0.448, -1.544, 0], + [1.845, 0.87, 0], + [2.205, 0.562, 0], + [1.868, 0.295, 0], + [0.367, -1.562, 0], + [2.007, -0.711, 0], + [1.364, 0.693, 0], + [1.884, 0.419, 0], + [1.468, -0.442, 0], + [1.471, 0.256, 0], + [3.311, 0.018, 0], + [2.038, 0.91, 0], + [2.898, 0.414, 0], + [1.701, 1.014, 0], + [1.175, -0.316, 0], + [1.788, -0.187, 0], + [2.436, 0.259, 0], + [1.863, -0.179, 0], + [1.202, -0.811, 0], + [1.576, 1.069, 0], + [0.925, 0.017, 0], + [2.015, 0.614, 0], + [1.158, -0.699, 0], + [1.998, 1.049, 0], + [1.565, -0.897, 0], + [1.373, 1.011, 0], + [0.961, -0.024, 0] + ], + "explained": [0.7296244541329987, 0.2285076178670176], + "clusters": [ + { + "id": 0, + "size": 100, + "share": 0.6666666666666666, + "traits": [ + { + "kind": "numeric", + "column": "petal_length", + "clusterMean": 4.906000000000001, + "overallMean": 3.7580000000000027 + }, + { + "kind": "numeric", + "column": "petal_width", + "clusterMean": 1.6760000000000006, + "overallMean": 1.199333333333334 + }, + { + "kind": "numeric", + "column": "sepal_length", + "clusterMean": 6.261999999999998, + "overallMean": 5.843333333333335 + } + ] + }, + { + "id": 1, + "size": 50, + "share": 0.3333333333333333, + "traits": [ + { + "kind": "numeric", + "column": "petal_length", + "clusterMean": 1.4620000000000002, + "overallMean": 3.7580000000000027 + }, + { + "kind": "numeric", + "column": "petal_width", + "clusterMean": 0.2459999999999999, + "overallMean": 1.199333333333334 + }, + { + "kind": "numeric", + "column": "sepal_length", + "clusterMean": 5.005999999999999, + "overallMean": 5.843333333333335 + } + ] + } + ], + "featureColumns": ["sepal_length", "sepal_width", "petal_length", "petal_width"], + "rowsUsed": 150, + "seed": 42 + } + } +} diff --git a/tests/share-links/mpg.json b/tests/share-links/mpg.json new file mode 100644 index 0000000..683a346 --- /dev/null +++ b/tests/share-links/mpg.json @@ -0,0 +1,919 @@ +{ + "v": 2, + "name": "mpg · mpg", + "createdAt": 1791510829266, + "dataset": { + "name": "mpg.csv", + "rowCount": 398, + "columnCount": 9 + }, + "target": "mpg", + "taskType": "regression", + "seed": 42, + "results": [ + { + "key": "baseline", + "ok": true, + "metrics": { + "rmse": 6.998789015115067, + "mae": 5.780108267716537, + "r2": -0.000025248976541547563 + }, + "primary": 6.998789015115067, + "trainMs": 0.2000000000698492, + "inferP50Ms": 0, + "inferP95Ms": 0, + "trainedRows": 254, + "valMetrics": { + "rmse": 7.288105463087612, + "mae": 6.09026820866142, + "r2": -0.004809204177528281 + }, + "valPrimary": 7.288105463087612 + }, + { + "key": "linear", + "ok": true, + "metrics": { + "rmse": 3.3636534719087474, + "mae": 2.6373125329246103, + "r2": 0.7690129363832534 + }, + "primary": 3.3636534719087474, + "trainMs": 78.5, + "inferP50Ms": 0, + "inferP95Ms": 0.10000000009313226, + "trainedRows": 254, + "valMetrics": { + "rmse": 3.145766347227879, + "mae": 2.43656552600629, + "r2": 0.812799401946135 + }, + "valPrimary": 3.145766347227879 + }, + { + "key": "knn", + "ok": true, + "metrics": { + "rmse": 2.8795503121147235, + "mae": 2.1674999999999995, + "r2": 0.8307165488311525 + }, + "primary": 2.8795503121147235, + "trainMs": 1.6000000000931323, + "inferP50Ms": 0.09999999997671694, + "inferP95Ms": 0.10000000009313226, + "trainedRows": 254, + "valMetrics": { + "rmse": 2.8422064492221533, + "mae": 2.1081250000000002, + "r2": 0.8471851537798616 + }, + "valPrimary": 2.8422064492221533 + }, + { + "key": "tree", + "ok": true, + "metrics": { + "rmse": 3.7170192272523814, + "mae": 2.479510416666667, + "r2": 0.7179313775598625 + }, + "primary": 3.7170192272523814, + "trainMs": 197.30000000004657, + "inferP50Ms": 0, + "inferP95Ms": 0.10000000009313226, + "trainedRows": 254, + "valMetrics": { + "rmse": 3.3624713717931463, + "mae": 2.3477213541666666, + "r2": 0.7861193484401149 + }, + "valPrimary": 3.3624713717931463 + }, + { + "key": "forest", + "ok": true, + "metrics": { + "rmse": 2.6224033444966888, + "mae": 2.014078553015182, + "r2": 0.8596009586088527 + }, + "primary": 2.6224033444966888, + "trainMs": 9106.900000000023, + "inferP50Ms": 0.2000000000698492, + "inferP95Ms": 3.400000000023283, + "trainedRows": 254, + "valMetrics": { + "rmse": 2.448163262349476, + "mae": 1.8335608653777828, + "r2": 0.8866203634014098 + }, + "valPrimary": 2.448163262349476 + }, + { + "key": "gbdt", + "ok": true, + "metrics": { + "rmse": 2.6781090036962056, + "mae": 2.0131786652716928, + "r2": 0.8535728324779502 + }, + "primary": 2.6781090036962056, + "trainMs": 557.7999999999302, + "inferP50Ms": 0, + "inferP95Ms": 0.10000000009313226, + "trainedRows": 254, + "valMetrics": { + "rmse": 2.9455986971392494, + "mae": 2.2744791328404528, + "r2": 0.835864899377924 + }, + "valPrimary": 2.9455986971392494 + }, + { + "key": "mlp", + "ok": true, + "metrics": { + "rmse": 3.228038936380911, + "mae": 2.4692217061460378, + "r2": 0.7872631661559584 + }, + "primary": 3.228038936380911, + "trainMs": 1904, + "inferP50Ms": 0, + "inferP95Ms": 0.2000000000698492, + "trainedRows": 254, + "valMetrics": { + "rmse": 3.143797706225727, + "mae": 2.357818860776324, + "r2": 0.8130336313311659 + }, + "valPrimary": 3.143797706225727 + }, + { + "key": "ensemble", + "ok": true, + "metrics": { + "rmse": 2.5451474376054186, + "mae": 1.9745686616560298, + "r2": 0.867751409040083 + }, + "primary": 2.5451474376054186, + "trainMs": 0, + "inferP50Ms": 0.19999999995343387, + "inferP95Ms": 0.5999999999767169, + "valMetrics": { + "rmse": 2.553264282082699, + "mae": 1.9982941468902977, + "r2": 0.8766764984169112 + }, + "valPrimary": 2.553264282082699 + } + ], + "summary": { + "task": { + "type": "regression" + }, + "taskType": "regression", + "seed": 42, + "trainRows": 254, + "testRows": 80, + "featureCount": 109, + "featureColumns": [ + "cylinders", + "displacement", + "horsepower", + "weight", + "acceleration", + "model_year", + "origin", + "name" + ], + "skippedColumns": [], + "totalMs": 229.5, + "validationRows": 64, + "ensemble": { + "members": ["forest", "knn", "gbdt"], + "method": "mean" + }, + "parallel": { + "helpers": 3, + "families": ["gbdt", "linear", "mlp", "forest", "tree"] + } + }, + "insights": { + "model": "forest", + "scatter": [ + { + "actual": 17, + "predicted": 15.748334776334776 + }, + { + "actual": 14, + "predicted": 13.140276504541209 + }, + { + "actual": 18, + "predicted": 20.44229752886003 + }, + { + "actual": 21, + "predicted": 21.296032647907644 + }, + { + "actual": 25, + "predicted": 22.754264069264075 + }, + { + "actual": 21, + "predicted": 20.886449314574314 + }, + { + "actual": 17, + "predicted": 17.90986093073593 + }, + { + "actual": 14, + "predicted": 13.670943521347931 + }, + { + "actual": 27, + "predicted": 26.48517857142857 + }, + { + "actual": 20, + "predicted": 23.44696248196248 + }, + { + "actual": 13, + "predicted": 13.59551204460744 + }, + { + "actual": 15, + "predicted": 13.818124510293625 + }, + { + "actual": 27, + "predicted": 26.352609126984127 + }, + { + "actual": 23, + "predicted": 20.620690386002885 + }, + { + "actual": 12, + "predicted": 12.384562171359772 + }, + { + "actual": 18, + "predicted": 26.733363095238094 + }, + { + "actual": 19, + "predicted": 23.193192640692637 + }, + { + "actual": 21, + "predicted": 21.715569444444444 + }, + { + "actual": 24, + "predicted": 20.99777083333333 + }, + { + "actual": 16, + "predicted": 16.078482142857144 + }, + { + "actual": 14, + "predicted": 14.365768849206347 + }, + { + "actual": 29, + "predicted": 25.748907828282825 + }, + { + "actual": 31, + "predicted": 32.564375 + }, + { + "actual": 32, + "predicted": 31.86902777777778 + }, + { + "actual": 28, + "predicted": 28.978633658008654 + }, + { + "actual": 26, + "predicted": 23.513996031746036 + }, + { + "actual": 15, + "predicted": 13.863871587433891 + }, + { + "actual": 16, + "predicted": 13.514396075737986 + }, + { + "actual": 24, + "predicted": 22.417324404761903 + }, + { + "actual": 24, + "predicted": 22.92776785714286 + }, + { + "actual": 25, + "predicted": 21.142845238095237 + }, + { + "actual": 27, + "predicted": 28.02829274891775 + }, + { + "actual": 17.5, + "predicted": 14.725535873440283 + }, + { + "actual": 14.5, + "predicted": 14.121547041065227 + }, + { + "actual": 22.5, + "predicted": 19.806595238095237 + }, + { + "actual": 33, + "predicted": 32.50479166666666 + }, + { + "actual": 20, + "predicted": 18.30813095238095 + }, + { + "actual": 17.5, + "predicted": 19.860583333333334 + }, + { + "actual": 29.5, + "predicted": 29.05860119047619 + }, + { + "actual": 20, + "predicted": 20.76591865079365 + }, + { + "actual": 19, + "predicted": 22.65316071428572 + }, + { + "actual": 19, + "predicted": 21.293487103174606 + }, + { + "actual": 16.5, + "predicted": 13.34129377309137 + }, + { + "actual": 15.5, + "predicted": 16.39684821428571 + }, + { + "actual": 17.5, + "predicted": 18.553577380952376 + }, + { + "actual": 20.5, + "predicted": 19.92934523809524 + }, + { + "actual": 19, + "predicted": 18.305214285714282 + }, + { + "actual": 36.1, + "predicted": 33.16464285714285 + }, + { + "actual": 19.9, + "predicted": 21.449833333333327 + }, + { + "actual": 20.2, + "predicted": 18.13682417582417 + }, + { + "actual": 20.5, + "predicted": 20.67926190476191 + }, + { + "actual": 19.4, + "predicted": 19.940345238095247 + }, + { + "actual": 20.6, + "predicted": 20.89057142857143 + }, + { + "actual": 20.3, + "predicted": 22.642514880952383 + }, + { + "actual": 21.6, + "predicted": 22.457785714285713 + }, + { + "actual": 21.5, + "predicted": 20.02354761904762 + }, + { + "actual": 19.8, + "predicted": 23.45575 + }, + { + "actual": 22.3, + "predicted": 26.397916666666664 + }, + { + "actual": 17, + "predicted": 18.415041666666664 + }, + { + "actual": 37.3, + "predicted": 33.76345833333333 + }, + { + "actual": 32.1, + "predicted": 35.12132738095238 + }, + { + "actual": 26.4, + "predicted": 26.900208333333335 + }, + { + "actual": 34.3, + "predicted": 34.49245833333334 + }, + { + "actual": 44.3, + "predicted": 40.91175 + }, + { + "actual": 30, + "predicted": 31.72954166666665 + }, + { + "actual": 40.9, + "predicted": 34.59645833333334 + }, + { + "actual": 23.6, + "predicted": 26.129309523809525 + }, + { + "actual": 27.2, + "predicted": 30.10283333333333 + }, + { + "actual": 25.8, + "predicted": 27.66032142857143 + }, + { + "actual": 35.1, + "predicted": 39.92874999999999 + }, + { + "actual": 32.9, + "predicted": 27.908511904761905 + }, + { + "actual": 30.7, + "predicted": 26.705166666666667 + }, + { + "actual": 26.6, + "predicted": 23.023958333333333 + }, + { + "actual": 28, + "predicted": 29.594202380952375 + }, + { + "actual": 36, + "predicted": 35.12495238095237 + }, + { + "actual": 36, + "predicted": 33.36861904761905 + }, + { + "actual": 32, + "predicted": 39.32783333333332 + }, + { + "actual": 22, + "predicted": 25.83063095238095 + }, + { + "actual": 27, + "predicted": 26.736160714285717 + }, + { + "actual": 28, + "predicted": 29.66124999999999 + } + ], + "residuals": { + "counts": [1, 1, 0, 4, 6, 11, 18, 17, 8, 10, 3, 1], + "min": -8.733363095238094, + "max": 6.303541666666661 + }, + "importance": [ + { + "column": "model_year", + "value": 1.6334632371203923 + }, + { + "column": "weight", + "value": 1.1171348301220037 + }, + { + "column": "displacement", + "value": 0.9615956947918342 + }, + { + "column": "horsepower", + "value": 0.5980831863419449 + }, + { + "column": "acceleration", + "value": 0.09231679335671468 + }, + { + "column": "origin", + "value": 0.07908705339185085 + }, + { + "column": "cylinders", + "value": -0.012758593610394295 + }, + { + "column": "name", + "value": -0.048438182852825605 + } + ], + "words": [ + { + "column": "name", + "term": "civic", + "effect": 1.3616666666666621, + "rows": 3 + }, + { + "column": "name", + "term": "honda", + "effect": 0.7145833333333348, + "rows": 3 + }, + { + "column": "name", + "term": "ford", + "effect": -0.16102690242763704, + "rows": 12 + }, + { + "column": "name", + "term": "amc", + "effect": -0.15249206349206368, + "rows": 6 + }, + { + "column": "name", + "term": "dodge", + "effect": 0.1442589285714284, + "rows": 4 + }, + { + "column": "name", + "term": "torino", + "effect": 0.12296296296296265, + "rows": 3 + }, + { + "column": "name", + "term": "toyota", + "effect": 0.11843750000000064, + "rows": 4 + }, + { + "column": "name", + "term": "datsun", + "effect": 0.11083333333333793, + "rows": 3 + }, + { + "column": "name", + "term": "oldsmobile", + "effect": 0.10078854016353962, + "rows": 3 + }, + { + "column": "name", + "term": "audi", + "effect": -0.06616666666666642, + "rows": 3 + }, + { + "column": "name", + "term": "mercury", + "effect": -0.06004166666666677, + "rows": 4 + }, + { + "column": "name", + "term": "plymouth", + "effect": 0.04817841880341837, + "rows": 6 + } + ], + "pdp": [ + { + "column": "model_year", + "points": [ + { + "x": 70, + "y": 21.198389336174436 + }, + { + "x": 70.63157894736842, + "y": 21.14895312846767 + }, + { + "x": 71.26315789473685, + "y": 21.114605658229586 + }, + { + "x": 71.89473684210526, + "y": 21.12711376834863 + }, + { + "x": 72.52631578947368, + "y": 21.0815449210654 + }, + { + "x": 73.15789473684211, + "y": 21.085659504398734 + }, + { + "x": 73.78947368421052, + "y": 21.36380007693383 + }, + { + "x": 74.42105263157895, + "y": 21.493604392410024 + }, + { + "x": 75.05263157894737, + "y": 21.64342701596703 + }, + { + "x": 75.6842105263158, + "y": 22.379589514059205 + }, + { + "x": 76.3157894736842, + "y": 22.42490982655922 + }, + { + "x": 76.94736842105263, + "y": 23.538297330447328 + }, + { + "x": 77.57894736842105, + "y": 24.078549520184673 + }, + { + "x": 78.21052631578948, + "y": 24.082096395184678 + }, + { + "x": 78.84210526315789, + "y": 25.61820270718864 + }, + { + "x": 79.47368421052632, + "y": 25.69482964171246 + }, + { + "x": 80.10526315789474, + "y": 26.57716550480769 + }, + { + "x": 80.73684210526315, + "y": 26.60027552083333 + }, + { + "x": 81.36842105263158, + "y": 26.704802083333323 + }, + { + "x": 82, + "y": 27.702696279761902 + } + ] + }, + { + "column": "weight", + "points": [ + { + "x": 1649, + "y": 26.747057022828066 + }, + { + "x": 1806.3157894736842, + "y": 26.701213272828067 + }, + { + "x": 1963.6315789473683, + "y": 26.590733064494735 + }, + { + "x": 2120.9473684210525, + "y": 26.55369340675665 + }, + { + "x": 2278.2631578947367, + "y": 25.805080365631106 + }, + { + "x": 2435.578947368421, + "y": 24.911383416226343 + }, + { + "x": 2592.894736842105, + "y": 24.158503677995817 + }, + { + "x": 2750.210526315789, + "y": 23.847057249424388 + }, + { + "x": 2907.5263157894738, + "y": 23.103436068868834 + }, + { + "x": 3064.842105263158, + "y": 22.02033051218593 + }, + { + "x": 3222.157894736842, + "y": 22.002759554844666 + }, + { + "x": 3379.4736842105262, + "y": 21.941928926011695 + }, + { + "x": 3536.7894736842104, + "y": 21.56535260008068 + }, + { + "x": 3694.1052631578946, + "y": 21.252849663260548 + }, + { + "x": 3851.4210526315787, + "y": 21.010328550205006 + }, + { + "x": 4008.736842105263, + "y": 20.8355517998414 + }, + { + "x": 4166.0526315789475, + "y": 20.851570897063628 + }, + { + "x": 4323.368421052632, + "y": 20.833450113333463 + }, + { + "x": 4480.684210526316, + "y": 20.770664911989666 + }, + { + "x": 4638, + "y": 20.691142473847748 + } + ] + } + ] + }, + "artifacts": { + "uncertainty": { + "isClassification": false, + "metricLabel": "rmse", + "testRows": 80, + "resamples": 1000, + "seed": 42, + "intervals": [ + { + "model": "ensemble", + "point": 2.5451474376054186, + "lo": 2.1390671361514726, + "hi": 2.9477344849587115 + }, + { + "model": "forest", + "point": 2.6224033444966888, + "lo": 2.154495403196923, + "hi": 3.1268449426648988 + }, + { + "model": "gbdt", + "point": 2.6781090036962056, + "lo": 2.2402285422775727, + "hi": 3.1407384584413087 + }, + { + "model": "knn", + "point": 2.8795503121147235, + "lo": 2.4126173740099075, + "hi": 3.2994267271471656 + }, + { + "model": "mlp", + "point": 3.228038936380911, + "lo": 2.5795046174721885, + "hi": 3.8311961538386536 + }, + { + "model": "linear", + "point": 3.3636534719087474, + "lo": 2.7333112533374377, + "hi": 4.009147609903389 + }, + { + "model": "tree", + "point": 3.7170192272523814, + "lo": 2.640402785231174, + "hi": 4.8293576408075385 + }, + { + "model": "baseline", + "point": 6.998789015115067, + "lo": 5.979119972759923, + "hi": 8.029770486693325 + } + ], + "verdict": { + "winner": "ensemble", + "against": "baseline", + "delta": -4.453641577509648, + "lo": -5.491168230803046, + "hi": -3.360022289002219, + "winShare": 1, + "decisive": true + } + }, + "segments": { + "model": "forest", + "isClassification": false, + "metricLabel": "rmse", + "overall": 2.6224033444966888, + "testRows": 80, + "minRows": 8, + "columns": [ + { + "column": "origin", + "inFeatures": true, + "segments": [ + { + "value": "japan", + "rows": 13, + "metric": 3.955019495511234, + "delta": 1.3326161510145451 + }, + { + "value": "europe", + "rows": 17, + "metric": 2.9092716937097007, + "delta": 0.28686834921301196 + }, + { + "value": "usa", + "rows": 50, + "metric": 2.014576699839863, + "delta": -0.6078266446568259 + } + ], + "spread": 1.3326161510145451, + "smallSegments": 0 + } + ] + } + } +} diff --git a/tests/share-links/titanic.json b/tests/share-links/titanic.json new file mode 100644 index 0000000..1aeeda5 --- /dev/null +++ b/tests/share-links/titanic.json @@ -0,0 +1,1003 @@ +{ + "v": 2, + "name": "titanic · survived", + "createdAt": 1791510834876, + "dataset": { + "name": "titanic.csv", + "rowCount": 891, + "columnCount": 15 + }, + "target": "survived", + "taskType": "binary", + "seed": 42, + "results": [ + { + "key": "baseline", + "ok": true, + "metrics": { + "accuracy": 0.6179775280898876, + "precision": 0.3089887640449438, + "recall": 0.5, + "f1": 0.3819444444444444, + "logLoss": 0.6650556742624868, + "auc": 0.5 + }, + "primary": 0.6179775280898876, + "trainMs": 0.5, + "inferP50Ms": 0, + "inferP95Ms": 0, + "trainedRows": 570, + "valMetrics": { + "accuracy": 0.6153846153846154, + "precision": 0.3076923076923077, + "recall": 0.5, + "f1": 0.380952380952381, + "logLoss": 0.6662787887663314, + "auc": 0.5 + }, + "valPrimary": 0.6153846153846154 + }, + { + "key": "logistic", + "ok": true, + "metrics": { + "accuracy": 0.8202247191011236, + "precision": 0.8104707792207793, + "recall": 0.8068181818181819, + "f1": 0.8085249428533011, + "logLoss": 0.4071660206806714, + "auc": 0.8867647058823529 + }, + "primary": 0.8202247191011236, + "trainMs": 506.4000000000233, + "inferP50Ms": 0, + "inferP95Ms": 0.09999999997671694, + "trainedRows": 570, + "valMetrics": { + "accuracy": 0.7902097902097902, + "precision": 0.784020390070922, + "recall": 0.7647727272727273, + "f1": 0.7714194373401535, + "logLoss": 0.49622421529068594, + "auc": 0.8051652892561983 + }, + "valPrimary": 0.7902097902097902 + }, + { + "key": "knn", + "ok": true, + "metrics": { + "accuracy": 0.7921348314606742, + "precision": 0.7800860562054592, + "recall": 0.778475935828877, + "f1": 0.7792525557231438, + "logLoss": 1.6663753758401623, + "auc": 0.8690508021390374 + }, + "primary": 0.7921348314606742, + "trainMs": 0.7000000000698492, + "inferP50Ms": 0, + "inferP95Ms": 0.09999999997671694, + "trainedRows": 570, + "valMetrics": { + "accuracy": 0.8181818181818182, + "precision": 0.8349358974358975, + "recall": 0.7806818181818183, + "f1": 0.7939937943262412, + "logLoss": 2.46211816234138, + "auc": 0.7992768595041322 + }, + "valPrimary": 0.8181818181818182 + }, + { + "key": "naiveBayes", + "ok": true, + "metrics": { + "accuracy": 0.7584269662921348, + "precision": 0.7516169942929614, + "recall": 0.7652406417112299, + "f1": 0.7527378452592473, + "logLoss": 2.5329002707200727, + "auc": 0.8557486631016042 + }, + "primary": 0.7584269662921348, + "trainMs": 5.699999999953434, + "inferP50Ms": 0, + "inferP95Ms": 0, + "trainedRows": 570, + "valMetrics": { + "accuracy": 0.7552447552447552, + "precision": 0.7429718875502007, + "recall": 0.75, + "f1": 0.745486905669972, + "logLoss": 3.449070538778885, + "auc": 0.7550619834710743 + }, + "valPrimary": 0.7552447552447552 + }, + { + "key": "tree", + "ok": true, + "metrics": { + "accuracy": 0.8314606741573034, + "precision": 0.8278248587570621, + "recall": 0.8102941176470588, + "f1": 0.8170230263157896 + }, + "primary": 0.8314606741573034, + "trainMs": 660, + "inferP50Ms": 0, + "inferP95Ms": 0.09999999997671694, + "trainedRows": 570, + "valMetrics": { + "accuracy": 0.7902097902097902, + "precision": 0.8057113717491076, + "recall": 0.7477272727272727, + "f1": 0.7596369341102645 + }, + "valPrimary": 0.7902097902097902 + }, + { + "key": "forest", + "ok": true, + "metrics": { + "accuracy": 0.8258426966292135, + "precision": 0.8204427630657138, + "recall": 0.8057486631016042, + "f1": 0.8115630229143189 + }, + "primary": 0.8258426966292135, + "trainMs": 14795, + "inferP50Ms": 0.09999999997671694, + "inferP95Ms": 0.40000000002328306, + "trainedRows": 570, + "valMetrics": { + "accuracy": 0.7622377622377622, + "precision": 0.7644230769230769, + "recall": 0.7215909090909091, + "f1": 0.7306072695035462 + }, + "valPrimary": 0.7622377622377622 + }, + { + "key": "gbdt", + "ok": true, + "metrics": { + "accuracy": 0.8202247191011236, + "precision": 0.8116776315789473, + "recall": 0.8040106951871657, + "f1": 0.8073593073593073, + "logLoss": 0.43734092668516666, + "auc": 0.8579545454545454 + }, + "primary": 0.8202247191011236, + "trainMs": 119, + "inferP50Ms": 0, + "inferP95Ms": 0.09999999997671694, + "trainedRows": 570, + "valMetrics": { + "accuracy": 0.7902097902097902, + "precision": 0.794595887135342, + "recall": 0.7545454545454545, + "f1": 0.7648026315789473, + "logLoss": 0.5163606037892963, + "auc": 0.7763429752066116 + }, + "valPrimary": 0.7902097902097902 + }, + { + "key": "mlp", + "ok": true, + "metrics": { + "accuracy": 0.7865168539325843, + "precision": 0.7744282744282744, + "recall": 0.7823529411764706, + "f1": 0.7774121363696197, + "logLoss": 0.8804804043956851, + "auc": 0.8061497326203209 + }, + "primary": 0.7865168539325843, + "trainMs": 1825.9000000000233, + "inferP50Ms": 0, + "inferP95Ms": 0.10000000009313226, + "trainedRows": 570, + "valMetrics": { + "accuracy": 0.7342657342657343, + "precision": 0.7192028985507246, + "recall": 0.7124999999999999, + "f1": 0.7151991614255766, + "logLoss": 1.0637954777972207, + "auc": 0.7278925619834711 + }, + "valPrimary": 0.7342657342657343 + }, + { + "key": "ensemble", + "ok": true, + "metrics": { + "accuracy": 0.8146067415730337, + "precision": 0.8040204383487966, + "recall": 0.8022727272727272, + "f1": 0.8031171442936148, + "logLoss": 0.3962067040584552, + "auc": 0.8957219251336899 + }, + "primary": 0.8146067415730337, + "trainMs": 0, + "inferP50Ms": 0, + "inferP95Ms": 0.10000000009313226, + "valMetrics": { + "accuracy": 0.7902097902097902, + "precision": 0.794595887135342, + "recall": 0.7545454545454545, + "f1": 0.7648026315789473, + "logLoss": 0.4816078088696139, + "auc": 0.8074380165289257 + }, + "valPrimary": 0.7902097902097902 + } + ], + "summary": { + "task": { + "type": "binary", + "classes": ["0", "1"] + }, + "taskType": "binary", + "seed": 42, + "trainRows": 570, + "testRows": 178, + "featureCount": 30, + "featureColumns": [ + "pclass", + "sex", + "age", + "sibsp", + "parch", + "fare", + "embarked", + "class", + "who", + "adult_male", + "deck", + "embark_town", + "alone" + ], + "skippedColumns": [], + "totalMs": 189.5, + "validationRows": 143, + "majorityShare": 0.6157894736842106, + "imbalanced": true, + "ensemble": { + "members": ["knn", "gbdt", "logistic"], + "method": "probability" + }, + "parallel": { + "helpers": 3, + "families": ["logistic", "tree", "mlp", "forest", "gbdt"] + } + }, + "insights": { + "model": "knn", + "classes": ["0", "1"], + "confusion": [ + [92, 18], + [19, 49] + ], + "importance": [ + { + "column": "sibsp", + "value": 0.03370786516853941 + }, + { + "column": "pclass", + "value": 0.022471910112359605 + }, + { + "column": "sex", + "value": 0.022471910112359605 + }, + { + "column": "class", + "value": 0.014981273408239737 + }, + { + "column": "who", + "value": 0.01310861423220977 + }, + { + "column": "adult_male", + "value": 0.01310861423220977 + }, + { + "column": "parch", + "value": -0.009363295880149725 + }, + { + "column": "deck", + "value": -0.011235955056179692 + }, + { + "column": "age", + "value": -0.013108614232209659 + }, + { + "column": "fare", + "value": -0.014981273408239626 + } + ], + "roc": { + "points": [ + { + "fpr": 0, + "tpr": 0 + }, + { + "fpr": 0.00909090909090909, + "tpr": 0.04411764705882353 + }, + { + "fpr": 0.00909090909090909, + "tpr": 0.10294117647058823 + }, + { + "fpr": 0.00909090909090909, + "tpr": 0.16176470588235295 + }, + { + "fpr": 0.01818181818181818, + "tpr": 0.20588235294117646 + }, + { + "fpr": 0.01818181818181818, + "tpr": 0.2647058823529412 + }, + { + "fpr": 0.02727272727272727, + "tpr": 0.3088235294117647 + }, + { + "fpr": 0.02727272727272727, + "tpr": 0.36764705882352944 + }, + { + "fpr": 0.03636363636363636, + "tpr": 0.4117647058823529 + }, + { + "fpr": 0.03636363636363636, + "tpr": 0.47058823529411764 + }, + { + "fpr": 0.05454545454545454, + "tpr": 0.5 + }, + { + "fpr": 0.06363636363636363, + "tpr": 0.5441176470588235 + }, + { + "fpr": 0.07272727272727272, + "tpr": 0.5882352941176471 + }, + { + "fpr": 0.1, + "tpr": 0.6029411764705882 + }, + { + "fpr": 0.10909090909090909, + "tpr": 0.6470588235294118 + }, + { + "fpr": 0.12727272727272726, + "tpr": 0.6764705882352942 + }, + { + "fpr": 0.14545454545454545, + "tpr": 0.7058823529411765 + }, + { + "fpr": 0.16363636363636364, + "tpr": 0.7352941176470589 + }, + { + "fpr": 0.18181818181818182, + "tpr": 0.7647058823529411 + }, + { + "fpr": 0.20909090909090908, + "tpr": 0.7794117647058824 + }, + { + "fpr": 0.23636363636363636, + "tpr": 0.7941176470588235 + }, + { + "fpr": 0.2545454545454545, + "tpr": 0.8235294117647058 + }, + { + "fpr": 0.2727272727272727, + "tpr": 0.8529411764705882 + }, + { + "fpr": 0.3, + "tpr": 0.8676470588235294 + }, + { + "fpr": 0.3090909090909091, + "tpr": 0.9117647058823529 + }, + { + "fpr": 0.33636363636363636, + "tpr": 0.9264705882352942 + }, + { + "fpr": 0.37272727272727274, + "tpr": 0.9264705882352942 + }, + { + "fpr": 0.4090909090909091, + "tpr": 0.9264705882352942 + }, + { + "fpr": 0.43636363636363634, + "tpr": 0.9411764705882353 + }, + { + "fpr": 0.4727272727272727, + "tpr": 0.9411764705882353 + }, + { + "fpr": 0.5, + "tpr": 0.9558823529411765 + }, + { + "fpr": 0.5363636363636364, + "tpr": 0.9558823529411765 + }, + { + "fpr": 0.5727272727272728, + "tpr": 0.9558823529411765 + }, + { + "fpr": 0.6090909090909091, + "tpr": 0.9558823529411765 + }, + { + "fpr": 0.6454545454545455, + "tpr": 0.9558823529411765 + }, + { + "fpr": 0.6818181818181818, + "tpr": 0.9558823529411765 + }, + { + "fpr": 0.7, + "tpr": 0.9852941176470589 + }, + { + "fpr": 0.7363636363636363, + "tpr": 0.9852941176470589 + }, + { + "fpr": 0.7727272727272727, + "tpr": 0.9852941176470589 + }, + { + "fpr": 0.8090909090909091, + "tpr": 0.9852941176470589 + }, + { + "fpr": 0.8454545454545455, + "tpr": 0.9852941176470589 + }, + { + "fpr": 0.8727272727272727, + "tpr": 1 + }, + { + "fpr": 0.9090909090909091, + "tpr": 1 + }, + { + "fpr": 0.9454545454545454, + "tpr": 1 + }, + { + "fpr": 0.9818181818181818, + "tpr": 1 + }, + { + "fpr": 1, + "tpr": 1 + } + ], + "auc": 0.8690508021390374 + }, + "pdp": [ + { + "column": "sibsp", + "points": [ + { + "x": 0, + "y": 0.46516853932584273 + }, + { + "x": 0.4210526315789474, + "y": 0.46404494382022476 + }, + { + "x": 0.8421052631578947, + "y": 0.46629213483146076 + }, + { + "x": 1.2631578947368423, + "y": 0.4629213483146068 + }, + { + "x": 1.6842105263157894, + "y": 0.4438202247191014 + }, + { + "x": 2.105263157894737, + "y": 0.41123595505618005 + }, + { + "x": 2.5263157894736845, + "y": 0.3943820224719103 + }, + { + "x": 2.9473684210526323, + "y": 0.3651685393258429 + }, + { + "x": 3.368421052631579, + "y": 0.3303370786516856 + }, + { + "x": 3.789473684210527, + "y": 0.2921348314606741 + }, + { + "x": 4.210526315789474, + "y": 0.30224719101123604 + }, + { + "x": 4.631578947368421, + "y": 0.27415730337078664 + }, + { + "x": 5.052631578947368, + "y": 0.24157303370786526 + }, + { + "x": 5.473684210526316, + "y": 0.22022471910112346 + }, + { + "x": 5.894736842105264, + "y": 0.1808988764044942 + }, + { + "x": 6.31578947368421, + "y": 0.08651685393258422 + }, + { + "x": 6.7368421052631575, + "y": 0.06741573033707862 + }, + { + "x": 7.157894736842106, + "y": 0.032584269662921356 + }, + { + "x": 7.578947368421053, + "y": 0.017977528089887642 + }, + { + "x": 7.999999999999999, + "y": 0.006741573033707866 + } + ] + }, + { + "column": "pclass", + "points": [ + { + "x": 1, + "y": 0.5359550561797757 + }, + { + "x": 1.1052631578947367, + "y": 0.5337078651685397 + }, + { + "x": 1.2105263157894737, + "y": 0.5292134831460678 + }, + { + "x": 1.3157894736842106, + "y": 0.524719101123596 + }, + { + "x": 1.4210526315789473, + "y": 0.5056179775280902 + }, + { + "x": 1.5263157894736843, + "y": 0.452808988764045 + }, + { + "x": 1.6315789473684212, + "y": 0.4460674157303372 + }, + { + "x": 1.736842105263158, + "y": 0.43820224719101125 + }, + { + "x": 1.842105263157895, + "y": 0.43707865168539317 + }, + { + "x": 1.9473684210526316, + "y": 0.4325842696629213 + }, + { + "x": 2.0526315789473686, + "y": 0.4213483146067415 + }, + { + "x": 2.1578947368421053, + "y": 0.4191011235955055 + }, + { + "x": 2.2631578947368425, + "y": 0.4179775280898876 + }, + { + "x": 2.368421052631579, + "y": 0.4146067415730336 + }, + { + "x": 2.473684210526316, + "y": 0.41011235955056174 + }, + { + "x": 2.578947368421053, + "y": 0.4101123595505616 + }, + { + "x": 2.68421052631579, + "y": 0.4044943820224717 + }, + { + "x": 2.7894736842105265, + "y": 0.3853932584269662 + }, + { + "x": 2.8947368421052633, + "y": 0.3831460674157304 + }, + { + "x": 3, + "y": 0.3752808988764045 + } + ] + } + ] + }, + "artifacts": { + "uncertainty": { + "isClassification": true, + "metricLabel": "accuracy", + "testRows": 178, + "resamples": 1000, + "seed": 42, + "intervals": [ + { + "model": "tree", + "point": 0.8314606741573034, + "lo": 0.7752808988764045, + "hi": 0.8820224719101124 + }, + { + "model": "forest", + "point": 0.8258426966292135, + "lo": 0.7696629213483146, + "hi": 0.8764044943820225 + }, + { + "model": "gbdt", + "point": 0.8202247191011236, + "lo": 0.7639044943820225, + "hi": 0.8707865168539326 + }, + { + "model": "logistic", + "point": 0.8202247191011236, + "lo": 0.7584269662921348, + "hi": 0.8707865168539326 + }, + { + "model": "ensemble", + "point": 0.8146067415730337, + "lo": 0.7528089887640449, + "hi": 0.8707865168539326 + }, + { + "model": "knn", + "point": 0.7921348314606742, + "lo": 0.7359550561797753, + "hi": 0.848314606741573 + }, + { + "model": "mlp", + "point": 0.7865168539325843, + "lo": 0.7247191011235955, + "hi": 0.8426966292134831 + }, + { + "model": "naiveBayes", + "point": 0.7584269662921348, + "lo": 0.6966292134831461, + "hi": 0.8202247191011236 + }, + { + "model": "baseline", + "point": 0.6179775280898876, + "lo": 0.5449438202247191, + "hi": 0.6853932584269663 + } + ], + "verdict": { + "winner": "tree", + "against": "baseline", + "delta": 0.2134831460674158, + "lo": 0.1348314606741573, + "hi": 0.2921348314606742, + "winShare": 1, + "decisive": true + } + }, + "threshold": { + "model": "knn", + "positiveClass": "1", + "positiveRate": 0.38202247191011235, + "averagePrecision": 0.7702377002052695, + "brier": 0.14584269662921334, + "prPoints": [ + { + "recall": 0.45588235294117646, + "precision": 0.8857142857142857, + "threshold": 1 + }, + { + "recall": 0.6764705882352942, + "precision": 0.7796610169491526, + "threshold": 0.8 + }, + { + "recall": 0.7205882352941176, + "precision": 0.7313432835820896, + "threshold": 0.6 + }, + { + "recall": 0.8970588235294118, + "precision": 0.648936170212766, + "threshold": 0.4 + }, + { + "recall": 0.9558823529411765, + "precision": 0.5241935483870968, + "threshold": 0.2 + }, + { + "recall": 1, + "precision": 0.38202247191011235, + "threshold": 0 + } + ], + "calibrationBins": [ + { + "meanPredicted": 0, + "observedRate": 0.05555555555555555, + "count": 54 + }, + { + "meanPredicted": 0.2000000000000001, + "observedRate": 0.13333333333333333, + "count": 30 + }, + { + "meanPredicted": 0.40000000000000013, + "observedRate": 0.4444444444444444, + "count": 27 + }, + { + "meanPredicted": 0.6, + "observedRate": 0.375, + "count": 8 + }, + { + "meanPredicted": 0.8000000000000003, + "observedRate": 0.625, + "count": 24 + }, + { + "meanPredicted": 1, + "observedRate": 0.8857142857142857, + "count": 35 + } + ], + "chosen": { + "threshold": 0.5, + "tp": 49, + "fp": 18, + "fn": 19, + "tn": 92, + "precision": 0.7313432835820896, + "recall": 0.7205882352941176, + "f1": 0.7259259259259259, + "accuracy": 0.7921348314606742, + "cost": 37, + "costFp": 1, + "costFn": 1 + } + }, + "segments": { + "model": "knn", + "isClassification": true, + "metricLabel": "accuracy", + "overall": 0.7921348314606742, + "testRows": 178, + "minRows": 8, + "columns": [ + { + "column": "deck", + "inFeatures": true, + "segments": [ + { + "value": "C", + "rows": 16, + "metric": 0.625, + "delta": -0.1671348314606742 + }, + { + "value": "D", + "rows": 9, + "metric": 1, + "delta": 0.2078651685393258 + } + ], + "spread": 0.2078651685393258, + "smallSegments": 5 + }, + { + "column": "class", + "inFeatures": true, + "segments": [ + { + "value": "First", + "rows": 45, + "metric": 0.7333333333333333, + "delta": -0.05880149812734092 + }, + { + "value": "Third", + "rows": 100, + "metric": 0.77, + "delta": -0.022134831460674187 + }, + { + "value": "Second", + "rows": 33, + "metric": 0.9393939393939394, + "delta": 0.14725910793326524 + } + ], + "spread": 0.14725910793326524, + "smallSegments": 0 + }, + { + "column": "embark_town", + "inFeatures": true, + "segments": [ + { + "value": "Cherbourg", + "rows": 39, + "metric": 0.6666666666666666, + "delta": -0.12546816479400758 + }, + { + "value": "Queenstown", + "rows": 18, + "metric": 0.7777777777777778, + "delta": -0.014357053682896415 + }, + { + "value": "Southampton", + "rows": 121, + "metric": 0.8347107438016529, + "delta": 0.04257591234097868 + } + ], + "spread": 0.12546816479400758, + "smallSegments": 0 + }, + { + "column": "embarked", + "inFeatures": true, + "segments": [ + { + "value": "C", + "rows": 39, + "metric": 0.6666666666666666, + "delta": -0.12546816479400758 + }, + { + "value": "Q", + "rows": 18, + "metric": 0.7777777777777778, + "delta": -0.014357053682896415 + }, + { + "value": "S", + "rows": 121, + "metric": 0.8347107438016529, + "delta": 0.04257591234097868 + } + ], + "spread": 0.12546816479400758, + "smallSegments": 0 + }, + { + "column": "who", + "inFeatures": true, + "segments": [ + { + "value": "woman", + "rows": 65, + "metric": 0.7230769230769231, + "delta": -0.06905790838375114 + }, + { + "value": "man", + "rows": 97, + "metric": 0.8247422680412371, + "delta": 0.032607436580562865 + }, + { + "value": "child", + "rows": 16, + "metric": 0.875, + "delta": 0.0828651685393258 + } + ], + "spread": 0.0828651685393258, + "smallSegments": 0 + }, + { + "column": "alive", + "inFeatures": false, + "segments": [ + { + "value": "yes", + "rows": 68, + "metric": 0.7205882352941176, + "delta": -0.07154659616655656 + }, + { + "value": "no", + "rows": 110, + "metric": 0.8363636363636363, + "delta": 0.044228804902962104 + } + ], + "spread": 0.07154659616655656, + "smallSegments": 0 + } + ] + } + } +}