A Python pipeline that discovers, extracts, and monitors replication and reproduction studies for the FLoRA database.
Part of the FORRT project.
Starting from keyword searches of academic databases, FLoRA Extractor:
- Discovers candidate replication/reproduction papers by scanning the OpenAlex snapshot
- Filters false positives with a declarative filter engine: rules route and discard, LLM screening tiers admit
- Extracts the target study and replication outcome from each paper
- Monitors extraction progress through a web dashboard; validation happens in a separate Supabase-backed repo
Stage 1: search/ → the survivor pool (search only — no filtering)
Stage 2: filter/engine → routing store + screen verdicts (route the pool, screen)
Stage 3: extract/ → data/extracted.csv (link original + code outcome)
Stage 4: validate/ → monitoring web app (dashboard at localhost:5001)
↕
Supabase (separate validation repo)
Each stage is independently runnable. The split is deliberate: Stage 1 searches and Stage 2 decides. Stage 1's only keyword logic is the search gate that makes scanning 510M works tractable (a broad token/stem alternation plus concept membership); every precision decision — exclusions, phrases, vocabulary, rescues — lives in Stage 2's spec bundle, so there is one rule set to reason about.
git clone <repo-url>
cd flora-extractor
pip install -r requirements.txt
cp .env.example .env # fill in your API keys
# Run the pipeline
python -m search.run_search --scan
python -m filter.engine route
python -m filter.engine screen --tier screen_expensive --run
python -m extract.tier --run
python -m extract.export --release <id>
# Start the monitoring web app
python -m validate.app # → http://localhost:5001Stage 2's screen is a dry run without --run, printing what a tier would cost
before anything is claimed or spent. Stage 3 reads no CSV: extract.tier builds its
worklist from the routing store and the pool, and extract.export renders
data/extracted.csv from the stored verdicts.
See docs/setup.md for full setup instructions.
RESEARCHER_EMAIL=you@example.com # for OpenAlex/Crossref API politeness
GEMINI_API_KEY=... # primary LLM (free at aistudio.google.com)
OPENAI_API_KEY=... # Stage 2's second screen voter (default SCREENING_MODEL_2) and Stage 3's linking/outcome models
OPENROUTER_API_KEY is needed only when a model id contains a / (e.g. a
SCREENING_MODEL_2 or the optional pre-screen voters routed through OpenRouter).
Optional: SUPABASE_URL, SUPABASE_SERVICE_KEY, GROBID_URL.
.env.example is the authoritative list of every variable and its default — copy it
rather than this excerpt.
docs/README.md is the documentation index — every guide, reference and code-flow walkthrough is listed there.
AI coding agent? Read CLAUDE.md first.
| Source | Role |
|---|---|
| OpenAlex snapshot | Discovery — the whole corpus is scanned locally by Stage 1's search gate; this is the only discovery source |
The API-harvest discovery sources (Semantic Scholar search, Bob Reed's Replication
Network, I4R) are retired — nothing downstream read their output, so they were parked
on wip/api-harvest-sources (PR #158). Semantic Scholar survives only as one of the
abstract-backfill sources in search/fetch_abstracts.py. Wiring curated harvesters
back in is issue #150.
Full-text: Unpaywall, CORE, arXiv, OSF. DOI resolution: Crossref. Abstract backfill: Europe PMC, Semantic Scholar, Crossref, Scopus.
This is a research pipeline whose screening, linking, and extraction decisions affect the resulting scientific dataset. We welcome issues with bug reports, evidence, and suggestions. Unsolicited pull requests are unlikely to be merged. Please discuss a proposed change with maintainers before implementing it.
See CONTRIBUTING.md for useful issue reports and the workflow for agreed changes, and follow the FORRT Code of Conduct.
- FLoRA database — the database this pipeline feeds
- flora_search_approaches — original R-based pipeline
MIT