Website · Quickstart · How it works · API keys · Docs

Read the story: My little VHS digitization project that turned into a family archive

tapesplit turns a box of digitized VHS and camcorder tapes into a family archive you can browse, search and trust. It watches every hour of footage, splits each tape into moments, and works out when each moment happened, where, and who was there. Anything it isn't sure about waits in a review queue. It runs on your Mac; cloud models are optional and capped at a dollar amount you set.

Every photo on this page is a real frame from my family's tapes (1997–2007), labeled by tapesplit; the faces in the family tree are drawn. People's names other than mine are changed.

Why I made this

I'm Phillip, the kid in these clips. My family's nineteen tapes cover 1997 to 2007: Madison, then Eugene, and trips to Hawaiʻi, Alaska, Davos and Moscow. Much of it is narrated in Russian, some of it straight to my grandparents in Moscow. On my first day of school, my dad asks me, in Russian, if I'm a little scared. I tell him no.

For a decade those tapes sat in the attic. Last year I started a project to digitize all of them, and what came back was 36 clips: thirty-one hours of footage with no labels and no dates. I didn't want to sit through all of it, and I didn't know where to start. So I built something to watch it for me.

From the tapes

The captions are scrapbook notes. The label tape is tapesplit's: where each trip was and what year, worked out from what's on screen, what's said on tape and the camcorder's date stamp.

One tape, split

Tape 16 came back from the digitizer as tape-18.mp4: two hours and four minutes with no label. tapesplit found 43 moments in it across a trip to Hawaii, a weekend on the Oregon coast and a spring at home, and skipped the blank tape, blue screens and static in between. The whole first archive was 19 tapes and 31 hours; it came back as 279 moments, 101 people and 244 places.

One morning, taken apart

Tape 18 has no label either. An hour in, the camcorder's clock reads SEP 7 2005 on a lawn in Eugene, Oregon: my first day of first grade. The very next scene is five months later. Here's what tapesplit made of that Wednesday.

When. For a few seconds after you press record, a camcorder burns the date into the picture. The video model read this one as SEP 7 2005 (Apple Vision found it too, in two pieces), and the teacher's whiteboard says the same day. Dates that don't describe the moment are left out: a file's own date (the day it was digitized) and years mentioned in passing.

Where. My dad says we're back in Eugene, and the video model reads Edison School off the sign above the entrance. A search for that name alone picked a school in Florida, with others in Utah and Arizona. None of those is anywhere we lived, so tapesplit flagged the match and searched again near home, and a separate verifier agreed the school belongs in Eugene.

Who. Two voices, separated and lined up with their words: my dad asks me, in Russian, what I think of my first day, and I tell him I'm not scared. tapesplit suggests who's who; the names are mine.

Multilingual. Russian and English, sometimes in one sentence. Every line stays in the language it was spoken, and search works across both: type "grandma and grandpa" and it finds «Скажи бабушкам, дедушкам».

Relationships. You know who's who from context: what people call each other, who turns up together, and where. tapesplit is learning to read the same clues, starting with family, then friends and teachers. Nobody on the tapes says "this is my dad"; they say Mom and Dad to me, in Russian, usually with my name in the same breath. tapesplit reads that context in both languages, works out who each word points to, and sketches the family as guesses: Dad at 0.84 from lines like "Give Dad the yogurt, Phillip" («Дай папе йогурт, Филипп»), Grandma and Grandpa at 0.76 and 0.72. Friends and teachers get the same treatment, grouped by where they turn up: "Max goes to kindergarten with Phillip" («Макс ходит с Филиппом в детский садик») makes Max a classmate in Madison. Every guess waits in the review queue until a person confirms it.

Nineteen tapes, one timeline

Tapes get reused and recorded over, so tape order isn't time order: Tape 3 is from 2005 and Tape 15 from 2001. tapesplit dates each moment on its own. 100 of the 286 got an exact day: 29 confirmed by a camcorder stamp that Apple Vision read, and 76 from what the video model saw or heard in the moment. The rest borrow a month or year from the moments around them. Above the tapes go the places: where we lived, and 13 trips found from the places in each moment, back to Dad's tapes from before I was born.

_{Teachers' and friends' names are changed, and the faces in the family tree are illustrated stand-ins. The English under the Russian lines was added for this page, except the Journal's, which is tapesplit's own.}

What you get

A local review app that feels like a photo library. Every card links to the second of tape it came from.

A moment plays from the right second. Underneath, How tapesplit knows shows the date stamp it read, boxed where Apple Vision found it, and where each place came from.	The Journal drafts days from the tapes into short entries. Every sentence cites its clip, and quotes are verbatim, Russian included.
Review holds the guesses that are left, each with its evidence. Accept the strong ones in bulk or fix one in a click.	Places puts everywhere the camera went on a map that works offline. Dashed pins are rough guesses.

Under the hood

One command runs everything:

tapesplit auto ~/Tapes

auto is a 21-stage pipeline. Each stage checks what your machine can do and skips with a reason when it can't. Every stage is resumable, cloud stages are priced before they run, and the run ends by auto-accepting only the guesses that clear per-type safety rules.

flowchart TB
  tapes["Digitized tapes · mp4, mov, avi, dv…"]
  subgraph watch["1 · Watch, on your Mac"]
    direction LR
    w1["Scene cuts and<br/>blank-tape removal"] ~~~ w2["Transcripts<br/>whisper.cpp"] ~~~ w3["Speakers<br/>pyannote"] ~~~ w4["On-screen text<br/>and date stamps<br/>Apple Vision"] ~~~ w5["Face tracks<br/>tapesplit"]
  end
  understand["2 · Understand · Gemini watches each tape in chunks · optional, cost-capped"]
  subgraph reconcile["3 · Reconcile, on your Mac"]
    direction LR
    r1["Claims that keep<br/>their evidence"] --> r2["Events, dates, places,<br/>people, relationships"] --> r3["Blind verification<br/>against the clips"]
  end
  review["4 · Review · safe auto-accept, then your queue"]
  archive[("Archive · timeline, people, places, journal, search")]
  tapes --> watch --> understand --> reconcile --> review --> archive
  watch --> reconcile
  review -. "your corrections, replayed on every rebuild" .-> reconcile

Loading

Pass	What happens
Watch	ffmpeg finds scene cuts and throws out blank tape, blue screen and static. whisper.cpp transcribes every word in whatever language was spoken. pyannote separates the voices. Apple Vision reads text in the frame, including the camcorder's burned-in date. tapesplit follows each face through the shot and clusters the tracks into people.
Understand	Gemini on Vertex AI watches each tape in chunks and proposes events with timestamps: what happened, who is there, where and when. Long tapes are chunked, and each chunk is told its own duration so timestamps can't drift. Every batch is estimated first and skipped if it would exceed --max-cloud-usd. Without Gemini, tapesplit still builds a timeline from local signals.
Reconcile	Every signal becomes a claim that keeps its source: a transcript line, a frame, a face track. Events are stitched across chunks and dated from stamps, speech and context. Names are reconciled, so Mom, Katya and Катя become one person, while home in 2003 and home in 2006 stay two places. A blind verifier re-checks a sample of claims against the clips.
Review	Strong guesses are accepted automatically under per-type confidence floors; a relationship also needs both people resolved and agreeing evidence. Everything else waits in the review queue. Corrections live in their own file and are replayed after every rebuild.

The full stage list, state handling and safety policy are in docs/AUTOMATION.md.

Built to be checked

• Guesses look like guesses. Inferred facts read "appears to be…", carry a confidence, and link to their evidence.
• Your fixes stick. Re-run the pipeline as often as you like; corrections replay on top.
• It measures itself. Blind clip verification gives each module a grounded-precision score, and that score decides when new logic ships.
• Plain files. A project is a folder of JSONL files, a SQLite search index and thumbnails. Nothing is locked in the app.

Quickstart

Apple Silicon Macs are the best-tested setup; Linux runs everything except the Apple Vision stages, which fall back to OpenCV. One script sets up either:

git clone https://github.com/bindeman/tapesplit
cd tapesplit
./scripts/setup.sh                     # installs what's missing, then runs tapesplit doctor

.venv/bin/tapesplit auto ~/Tapes --plan    # what would run, without running it
.venv/bin/tapesplit auto ~/Tapes           # tapes in, archive out (~/Tapes.tapesplit)
.venv/bin/tapesplit ui ~/Tapes.tapesplit   # browse and review

setup.sh checks for Python 3.11+, ffmpeg, whisper.cpp and Node 20+ and installs what's missing with Homebrew on macOS or apt, dnf or pacman on Linux (where it builds whisper.cpp from source). Then it creates .venv with the extras for your platform and copies .env.example to .env; every key in it is optional. It's safe to run again, and --check only reports. Add --with-model to download a whisper.cpp model (ggml-large-v3-turbo.bin, about 1.6 GB) and point WHISPER_CPP_MODEL at it, --minimal for the core package without the local AI extras, or --skip-system to leave system packages alone. The review app's own packages install on the first tapesplit ui.

To do it by hand instead:

brew install ffmpeg whisper-cpp node   # on Linux: your package manager, and build whisper.cpp
python3.12 -m venv .venv
.venv/bin/pip install -e '.[local-ai,vision,macos,visual-ai,face-ai,speaker-ai]'   # drop macos on Linux
cp .env.example .env
.venv/bin/tapesplit doctor

Extra	Adds
local-ai	openai-whisper, sentence-transformers and sqlite-vec for dense multilingual search
vision	OpenCV face detection and frame analysis
macos	Apple Vision OCR and face detection
visual-ai	Image captions and visual embeddings
face-ai	Face identity embeddings on ONNX Runtime (Core ML on Apple Silicon)
speaker-ai	pyannote.audio and SpeechBrain diarization

Useful variants:

tapesplit auto ~/Tapes --profile local           # never call a cloud service
tapesplit auto ~/Tapes --max-cloud-usd 25        # raise the cloud budget for this run
tapesplit auto ~/Tapes --language ru             # transcription language hint
tapesplit auto ~/Tapes.tapesplit --force-from scenes   # redo scenes and everything after
tapesplit status ~/Tapes.tapesplit               # stage state and archive metrics
tapesplit search query ~/Tapes.tapesplit "first day of school"

Every stage is also its own command (detect-scenes, transcribe, speakers diarize, gemini analyze-video, detect-faces, cluster-faces, search build, journal generate, …). See tapesplit --help and docs/LOCAL_SETUP.md.

API keys

tapesplit works with no keys at all. Each service you add makes one part of the archive better. Keys go in .env (gitignored); tapesplit doctor reports what it found without printing secrets.

Service	What it adds	Setup	Without it
Google Cloud (Vertex AI · Gemini)	Watches the video itself: rich event titles, summaries, people, places and dates	Create a project with the Vertex AI API enabled and a Cloud Storage bucket, then run gcloud auth application-default login. Set GOOGLE_CLOUD_PROJECT, GOOGLE_CLOUD_LOCATION and GEMINI_GCS_BUCKET.	Low-confidence "recording segment" events from local signals
Hugging Face	pyannote speaker diarization	Create a read token, accept the terms of pyannote/speaker-diarization-community-1, set HF_TOKEN	SpeechBrain fallback
Azure OpenAI	Diarized transcription that names known speakers, the blind verifier, frame descriptions and Journal drafting	Set AZURE_OPENAI_ENDPOINT, AZURE_OPENAI_API_KEY, AZURE_OPENAI_API_VERSION and the deployment names in .env.example. Opt in with TAPESPLIT_DIARIZE_BACKEND=azure-openai and TAPESPLIT_VERIFIER_BACKEND=azure.	Those stages skip with a reason
Google Maps	Sharper geocoding for named places	GOOGLE_MAPS_API_KEY, GOOGLE_MAPS_ENABLED=true and a GOOGLE_MAPS_DAILY_BUDGET_USD cap	OpenStreetMap Nominatim (free; set TAPESPLIT_CONTACT_EMAIL to identify yourself)
TwelveLabs	An optional hosted video index for search experiments	TWELVELABS_API_KEY and the twelvelabs extra	Local hybrid search

What it costs. Gemini analysis of all 31 hours of the sample archive came to about $8 on Vertex AI. Three guards keep it bounded: tapesplit auto estimates each batch against --max-cloud-usd (default $10), GEMINI_PROJECT_BUDGET_USD caps cumulative Gemini spend, and GOOGLE_MAPS_DAILY_BUDGET_USD caps geocoding. Copy cost_rates.example.json to cost_rates.json to price calls with your own rates.

Privacy

• Local by default. Every stage has a local path, and --profile local never makes a network call to an AI service.
• What leaves your machine, when you opt in. Gemini analysis uploads low-resolution proxies of your tapes to a Cloud Storage bucket in your Google Cloud project. The Azure stages send individual frames and short audio clips. Geocoding sends place names.
• Your archive stays out of git. Project folders (*.tapesplit/), media files, .env and cost_rates.json are all gitignored. Facts about one family (who usually held the camera, for example) go in <project>/project_hints.json, never in the code.
• The review app is local-only. Its dev server binds to 127.0.0.1 and refuses requests from other hosts or other websites.

Project layout

src/tapesplit/     Python package and the `tapesplit` CLI: pipeline stages, models, adapters
apps/review-ui/    React + Vite review app (`tapesplit ui`)
docs/              Design notes: automation, entity resolution, relationships, retrieval, …
tests/             pytest suite (runs without the ML extras; heavy tests skip themselves)
site/              The project website, deployed to GitHub Pages

Start with docs/HANDOFF.md for the architecture tour. The other design notes:

• AUTOMATION.md: tapesplit auto stages, state, degradation and safety policies
• REFOUNDATION.md: the claim-based semantic core and how it is verified
• ENTITY_RESOLUTION.md and RELATIONSHIP_INFERENCE.md: people, aliases, roles and family graphs
• TEMPORAL_EVIDENCE_FABRIC.md: the long-term evidence model
• RETRIEVAL_ARCHITECTURE.md: hybrid full-text and vector search
• LOCAL_FACE_STACK.md: face tracks, embeddings and clustering
• JOURNAL.md and DESIGN.md: the Journal's grounding contract and the UI design standard

Status

tapesplit is an alpha. It runs end to end on a real 31-hour archive, and the review queue exists because it still makes mistakes, such as borrowing a place from the wrong era or attributing a quote to the wrong speaker. Next up:

• API-key backends for Gemini (AI Studio) and OpenAI, alongside Vertex AI and Azure
• A packaged desktop app with a native project picker
• A watch mode that picks up new tapes dropped into a folder

Contributing

Issues and pull requests are welcome. See CONTRIBUTING.md for setup, house rules and the terms for contributions, and SECURITY.md to report a vulnerability privately. Please use fictional people and places in tests and bug reports, never real family footage, names or transcripts.

License

tapesplit is source-available under the PolyForm Noncommercial License 1.0.0. You can use, study, change and share it for personal, family, educational, research and other noncommercial purposes. Commercial use, including use inside a business or as part of a paid product or service, needs written permission from the author (phillipbindeman.com).

The footage, stills, clips and screenshots under site/media/ and on this page come from my family's tapes. They are © Phillip Bindeman, all rights reserved, and are not covered by the code license.

---

_{Built by Phillip Bindeman with a box of family tapes.}