Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .github/workflows/project-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
name: project-ci

on:
push:
branches: ["review/**"]
pull_request:
merge_group:
workflow_dispatch:
Expand Down Expand Up @@ -86,3 +88,8 @@ jobs:
if [[ "$ran_any" -eq 0 ]]; then
echo "No recognized project CI harness found; passing as metadata-only repo."
fi

- name: Verify published aggregate evidence
run: |
python scripts/verify_evidence.py --check
python scripts/verify_queue_payoff.py --check
91 changes: 91 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -61,3 +61,94 @@ data/databento/
# Cross-agent working notes. Kept locally, not published: it is scratch context
# for whoever picks the work up next, not part of the repo.
handoff.md

# Reviewed aggregate evidence only. Raw replay outputs remain ignored.
!/report/backend_sweep.csv
!/report/hawkes/compare_AAPL_2024-02-01.csv
!/report/hawkes/compare_AAPL_2024-04-01.csv
!/report/hawkes/compare_AAPL_2024-06-03.csv
!/report/hawkes/compare_AAPL_2024-08-01.csv
!/report/hawkes/compare_AAPL_2024-10-01.csv
!/report/hawkes/compare_INTC_2024-02-01.csv
!/report/hawkes/compare_INTC_2024-04-01.csv
!/report/hawkes/compare_INTC_2024-08-02.csv
!/report/hawkes/compare_INTC_2024-10-01.csv
!/report/hawkes/compare_INTC_2024-12-02.csv
!/report/hawkes/compare_MSFT_2024-02-01.csv
!/report/hawkes/compare_MSFT_2024-04-01.csv
!/report/hawkes/compare_MSFT_2024-06-03.csv
!/report/hawkes/compare_MSFT_2024-10-01.csv
!/report/hawkes/compare_MSFT_2024-12-02.csv
!/report/hawkes/compare_base_AAPL_2024-02-01.csv
!/report/hawkes/compare_base_AAPL_2024-04-01.csv
!/report/hawkes/compare_base_AAPL_2024-06-03.csv
!/report/hawkes/compare_base_AAPL_2024-08-01.csv
!/report/hawkes/compare_base_AAPL_2024-10-01.csv
!/report/hawkes/compare_base_INTC_2024-02-01.csv
!/report/hawkes/compare_base_INTC_2024-04-01.csv
!/report/hawkes/compare_base_INTC_2024-08-02.csv
!/report/hawkes/compare_base_INTC_2024-10-01.csv
!/report/hawkes/compare_base_INTC_2024-12-02.csv
!/report/hawkes/compare_base_MSFT_2024-02-01.csv
!/report/hawkes/compare_base_MSFT_2024-04-01.csv
!/report/hawkes/compare_base_MSFT_2024-06-03.csv
!/report/hawkes/compare_base_MSFT_2024-10-01.csv
!/report/hawkes/compare_base_MSFT_2024-12-02.csv
!/report/hawkes/kernel_AAPL_2024-02-01.csv
!/report/hawkes/kernel_AAPL_2024-04-01.csv
!/report/hawkes/kernel_AAPL_2024-06-03.csv
!/report/hawkes/kernel_AAPL_2024-08-01.csv
!/report/hawkes/kernel_AAPL_2024-10-01.csv
!/report/hawkes/kernel_INTC_2024-02-01.csv
!/report/hawkes/kernel_INTC_2024-04-01.csv
!/report/hawkes/kernel_INTC_2024-08-02.csv
!/report/hawkes/kernel_INTC_2024-10-01.csv
!/report/hawkes/kernel_INTC_2024-12-02.csv
!/report/hawkes/kernel_MSFT_2024-02-01.csv
!/report/hawkes/kernel_MSFT_2024-04-01.csv
!/report/hawkes/kernel_MSFT_2024-06-03.csv
!/report/hawkes/kernel_MSFT_2024-10-01.csv
!/report/hawkes/kernel_MSFT_2024-12-02.csv
!/report/hawkes/lob_bench.csv
!/report/hawkes/lob_bench_AAPL_2024-02-01.csv
!/report/hawkes/lob_bench_AAPL_2024-04-01.csv
!/report/hawkes/lob_bench_AAPL_2024-06-03.csv
!/report/hawkes/lob_bench_AAPL_2024-08-01.csv
!/report/hawkes/lob_bench_AAPL_2024-10-01.csv
!/report/hawkes/lob_bench_INTC_2024-02-01.csv
!/report/hawkes/lob_bench_INTC_2024-04-01.csv
!/report/hawkes/lob_bench_INTC_2024-08-02.csv
!/report/hawkes/lob_bench_INTC_2024-10-01.csv
!/report/hawkes/lob_bench_INTC_2024-12-02.csv
!/report/hawkes/lob_bench_MSFT_2024-02-01.csv
!/report/hawkes/lob_bench_MSFT_2024-04-01.csv
!/report/hawkes/lob_bench_MSFT_2024-06-03.csv
!/report/hawkes/lob_bench_MSFT_2024-10-01.csv
!/report/hawkes/lob_bench_MSFT_2024-12-02.csv
!/report/hawkes/summary.csv
!/report/latency/latency_curve.csv
!/report/lob_bench_INTC_2024-08-02.csv
!/report/lob_bench_MSFT_2024-06-03.csv
!/report/ofi_sweep.csv
!/report/queue_position/deciles.csv
!/report/queue_position/summary.csv
!/report/queue_reactive/compare_INTC_2024-08-02.csv
!/report/queue_reactive/compare_MSFT_2024-06-03.csv
!/report/queue_reactive/coupled_INTC_2024-08-02.csv
!/report/queue_reactive/coupled_MSFT_2024-06-03.csv
!/report/queue_reactive/intensities_INTC_2024-08-02.csv
!/report/queue_reactive/intensities_MSFT_2024-06-03.csv
!/report/queue_reactive/lob_bench_INTC_2024-08-02.csv
!/report/queue_reactive/lob_bench_MSFT_2024-06-03.csv
!/report/session_profile.csv
!/report/turing/auc.csv
!/report/turing/families.csv
!/report/evidence_manifest.csv
!/report/evidence/
!/report/evidence/*.csv
!/report/readme_tables/
!/report/readme_tables/*.csv
!/report/queue_payoff/
!/report/queue_payoff/deciles.csv
!/report/queue_payoff/comparison.csv
!/report/queue_payoff/summary.csv
39 changes: 34 additions & 5 deletions DATA.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,12 @@ LOBSTER is a reconstruction of the same Nasdaq TotalView-ITCH feed, so Databento

- Source code, build files, and tests
- Small reduced message fixtures under `data/*.csv` (25 lines, 20 valid messages each) so the build and tests run on a fresh clone with no external data
- Derived results: benchmark tables, latency percentiles, analytics summaries
- Eighty reviewed aggregate CSVs under `report/`, with row counts and SHA-256
hashes in `report/evidence_manifest.csv`; fitted rates, distances, deciles,
latency summaries and classifier scores, not event records
- `report/evidence/claims.csv`, recomputed from those aggregates by the offline verifier
- `report/readme_tables/`, explicitly labelled transcriptions of historical
README tables, including measurements without a surviving machine-output CSV

## What is not committed

Expand All @@ -18,14 +23,14 @@ LOBSTER is a reconstruction of the same Nasdaq TotalView-ITCH feed, so Databento

## Sessions used

**Fifteen symbol-days on three names, all in 2024.** Every one is an ordinary
trading day; the sample contains no stress days, no halts and no index events.
`INTC 2024-08-02` is the single exception and is a single-name event day, the
**Fifteen symbol-days on three names, all in 2024.** No complete external audit
of stress, halts or index events was performed; their absence is not asserted.
`INTC 2024-08-02` is an identified single-name event day, the
session after Intel's Q2 report and dividend suspension. It is kept because it
is genuinely different and it is flagged wherever it behaves differently.

Nothing measured on this sample is a population or a regime claim. Three names
on ordinary days support statements about Microsoft, Intel and Apple on those
on selected days support statements about Microsoft, Intel and Apple on those
days, and nothing wider.

All fifteen cost **$0.0000**, confirmed with `metadata.get_cost` rather than
Expand Down Expand Up @@ -76,3 +81,27 @@ Requires `DATABENTO_API_KEY` and an entitlement to `XNAS.ITCH`.
## Licence and retention

Databento data is accessed under a programme licence and is not redistributed here. Raw extracts are deleted at the end of the associated academic affiliation; code, derived statistics, and figures are not derived-from-restriction and remain.

## Publication audit, 2026-09-06

The eighty aggregate CSVs were previously local ignored artifacts. They are now
explicitly allowlisted and published without a new fit or new source. The
manifest records the available analysis-code revision and recovered file hashes;
it is not proof that the raw-to-aggregate pipeline was rerun in this review.
The event streams, books and raw vendor extracts remain excluded.

`python scripts/verify_evidence.py --check` needs only the Python standard
library. It verifies the frozen sample and source integrity and recalculates
eleven headline summaries. Raw vendor agreement and historical host timings
require the corresponding licensed-data validators and original measurement
conditions. A README transcription is archival evidence of what was claimed,
not an independent reproduction of the underlying measurement.

## Direct queue-payoff aggregates, 2026-09-08

`report/queue_payoff/deciles.csv` contains per-decile counts, aggregate share
quantities, marked payoffs and bootstrap intervals. `comparison.csv` compares
old and new estimands per session; `summary.csv` reports the session ranges and
counts. These are derived from the existing fifteen cached message/book pairs.
No source, licence, quote sequence, order identifier or raw market-data file is
added. The offline verifier reconciles aggregates, not the raw reconstruction.
Loading
Loading