feat!: add PE-A evidence retrieval and agent evaluation - #153

Open

tulayha wants to merge 57 commits into

mainfrom

codex/fix-macos-benchmark-setup

Commits on Sep 7, 2026

fix(benchmarks): make macOS pilot setup resumable
tulayha
committed
9483700
fix(benchmarks): harden Codex agent ablation
tulayha
committed
a465912
fix(sound): restore FineLAP text tokenization
tulayha
committed
8e89cdb
fix(benchmarks): require fresh retrieval jobs
tulayha
committed
1294060
fix(benchmarks): remove workflow bias
tulayha
committed
2329b23
fix(benchmarks): pin scorer Python runtime
tulayha
committed
cc4912d
feat(benchmarks): report paired run metrics
tulayha
committed
cd3846b
fix(benchmarks): correct reporting and model direction
tulayha
committed
9733468
fix(search): preserve precise fused boundaries
tulayha
committed
566bc6c
Revert "fix(search): preserve precise fused boundaries"
tulayha
committed
3bc2c4b
docs(benchmarking): record research adoption
tulayha
committed
5818541
docs(benchmarking): record traced localization failure
tulayha
committed
32977ed
fix(benchmarks): expose durable retrieval boundaries
tulayha
committed
19c4e08
docs(benchmarking): define localization prerequisites
tulayha
committed
1b61ccc
feat(benchmarks): export dense scene score curves
tulayha
committed
9de7843
fix(benchmarks): probe every task modality
tulayha
committed
1c0c303
docs(benchmarking): record full modality probe
tulayha
committed
572c73d
test(benchmarks): compare adaptive span localization
tulayha
committed
ea73eb2
docs(benchmarking): conclude span diagnostic
tulayha
committed
ea33d7a
test(benchmarks): compare overlapping action windows
tulayha
committed
e908a7b
docs(benchmarking): record overlap control result
tulayha
committed
0954b99
test(benchmarks): add disjoint shot proposal control
tulayha
committed
a1453f6
test(benchmarks): evaluate proposal-preserving fusion
tulayha
committed
e7fb610
test(benchmarks): reject proposal rrf on held-out tasks
tulayha
committed
8c60a1a
test(benchmarks): isolate FineLAP retrieval streams
tulayha
committed
a36062b
fix(sound): separate FineLAP retrieval outputs
tulayha
committed
e10e0f4
test(benchmarks): evaluate multiscale action retrieval
tulayha
committed
e35a8d1
fix(action): align VideoPrism query preprocessing
tulayha
committed
82e6765
test(benchmarks): record FineLAP selector failure
tulayha
committed
e15c05a
docs: pin two-stage sound measurement
tulayha
committed
04fcb04
docs: ground sound direction in current AMR results
tulayha
committed
2ce91d3
fix(search): return bounded multimodal evidence chunks
tulayha
committed
3ebcf80
test(benchmarks): add individual modality gates
tulayha
committed
ae0d635
test(benchmarks): select sound localization provider
tulayha
committed
55a5ff3
feat(sound)!: integrate PE-A frame retrieval
tulayha
committed
eab8bb7
fix(benchmarks): isolate indexes by schema
tulayha
committed
c1251de
test(benchmarks): add three-condition agent ablation
tulayha
committed
4f24f41
fix(benchmarks): make agent ablation reproducible
tulayha
committed
ed20d18
fix(benchmarks): record machine and indexing cost
tulayha
committed
c020656
fix(benchmarks): audit held-out agent pilot
tulayha
committed
df4b626
fix(benchmarks): harden isolated agent evaluation
tulayha
committed
bf0f3ba
feat(query): add local agent benchmark path
tulayha
committed
2670f9c
fix(benchmarks): manage local SLM runtime
tulayha
committed
73bd1ac
fix(benchmarks): run local agent through promptfoo
tulayha
committed
e3809fb
fix(benchmarks): validate local agent construction
tulayha
committed
cce5f43
fix(benchmarks): validate local provider wiring
tulayha
committed
ad1ee6e
fix(benchmarks): reuse local model settings
tulayha
committed
b2b889b
fix(local-answers): align Ollama agent requests
tulayha
committed
d9f6d85
fix(local-answers): align managed model execution
tulayha
committed
7911fdd
test(benchmarks): narrow local SLM to query routing
tulayha
committed
6dbabf5
docs: allow reviewed benchmark artifacts
tulayha
committed
2995441
test(benchmarks): compare local SLM search policies
tulayha
committed
25e04ac
fix(cli): keep benchmark dependencies optional
tulayha
committed
fdccbee
docs: correct benchmark methodology references
tulayha
committed
95757c1
test: correct lazy benchmark mock targets
tulayha
committed
8493102
chore(desktop): refresh generated package inputs
tulayha
committed
227e1a7
build(desktop): generate model catalogs during builds
tulayha
committed
59636e1