Heads-Up

A proactive assistant that interrupts you only when it matters. A small judge model on your Mac reads every new GitHub and calendar event and decides whether it is worth a Telegram message. Everything else is logged and never shown.

By Dev Suthar and the LDS Team. Published by Let's Data Science. Dev created the app and its evaluation; the LDS Team turned it into a build-along with a demo mode, a re-test of the judge's question and a step-by-step guide. See CONTRIBUTORS.md and NOTICE. Inspired by Claude Duet; not affiliated with or endorsed by Anthropic.

Build it step by step: the free guide on Let's Data Science walks through each part, with checks you can run and hints.

The honest result

On 70 held-out synthetic events, the judge scored an F1 of 0.60 with the question this app asks. With Dev's original wording it scored 0.65, and a set of simple rules scored 0.53. Every difference sits inside the 95% interval, so the judge is not proven better than simple rules. The data is synthetic, so real-world numbers may be lower. The details are under Results.

Get the project

• Version 1.0.0 downloads: choose the starter to build the three exercises yourself, or the reference to start from the finished project. The release includes SHA-256 checksums.
• This repository contains the completed reference. Clone it, then follow the steps below. Keep your starter in a separate folder.

git clone https://github.com/lets-data-science/heads-up.git
cd heads-up

What you need

• A Mac with Apple silicon (M1 or newer) and Homebrew. The installer and the judge's model build are Mac-only.
• About 3 GB of free disk space: Python packages, Ollama and the 2 GB judge model.
• For the demo: nothing else. No account, token or API key.
• For live alerts: a Telegram account, and a GitHub account or a Google Calendar (or both). An Anthropic API key is optional: without one, alerts use a plain template.

Heads-Up runs only while your Mac is awake. There is no background service.

Try it in three double-clicks

1. Double-click Install.command (once). It installs Python 3.12 and Ollama with Homebrew if they are missing, creates the project's own Python folder (.venv) with the tested package versions, and downloads the judge model (about 2 GB, with a progress bar). Running it again is safe: every step checks first. It never asks for administrator rights. If Homebrew is missing it stops and says so, because Homebrew's own installer asks for your Mac password.
2. Double-click Demo.command. The judge decides one synthetic day of events and the window shows every score, every decision and whether Dev's label agreed. Nothing is sent anywhere. Try another day with --day (see below).
3. Double-click Heads-Up.command when you want real alerts. Your browser opens the setup page: a Telegram bot token, Find my chat ID, Send test message, a GitHub repository and/or your calendar's secret iCal address, and Test connection. Saved keys are shown only as "set" and are stored in .env on your Mac, readable by you only.

The first time you open each .command file, macOS refuses it ("Install.command" Not Opened), because the scripts are not notarised by Apple. Click Done (not Move to Bin), open System Settings > Privacy & Security, scroll to Security, click Open Anyway next to the file name, enter your Mac password and click Open. Do this once per file. If a double-click does nothing, run chmod +x *.command once in Terminal.

How it works

GitHub + iCal -> one Event shape -> drop repeats (content hash)
  -> the judge: Strands Decider 2B on your Mac (Ollama), asked one yes/no question, answers a probability
  -> policy: probability >= 0.45 and fewer than 5 interruptions today (your computer's time zone)
       yes -> alert text (Haiku if you add a key, otherwise a template) -> Telegram + buttons
       no  -> logged in SQLite only
  -> every step goes to the local decision log: event, probability, verdict, tokens, cost

The judge is asked: "You maintain this repo and are working. Should you be interrupted right now for this event?" It sees only the source, the kind of event, its title and a time phrase; never bodies, attendee lists or descriptions. Claude Haiku 5.5, when you add a key, only writes the alert text for events that already passed: it cannot choose or create buttons. If the judge is down or too slow, the event is logged and never sent.

Buttons can do exactly three things: snooze for an hour, remind in two hours, or re-run a failed CI job. Nothing happens until you tap, taps are accepted only from your own chat, and buttons expire after an hour.

The first check after setup records what each source already holds and alerts on none of it. Only events that appear after that can interrupt you.

Use it from the terminal

From the project folder, after Install.command (or the manual setup below):

PYTHONPATH=src .venv/bin/python -m headsup --demo --day 4    # the demo for any of the 10 synthetic days
PYTHONPATH=src .venv/bin/python -m headsup --once --dry-run  # one pass with your settings: prints what would happen, sends nothing
PYTHONPATH=src .venv/bin/python -m headsup --once            # one real pass: collect, decide, write, send
PYTHONPATH=src .venv/bin/python -m headsup --listen 10       # listen for button taps for 10 minutes
PYTHONPATH=src .venv/bin/python -m headsup --ui              # the dashboard and setup page at http://127.0.0.1:8765/
.venv/bin/python -m pytest                                   # the tests: offline, nothing touches the network

--demo and --ui start Ollama themselves if nothing answers and stop it when they finish. For --once and --listen, start it yourself:

OLLAMA_HOST=127.0.0.1:11434 OLLAMA_MODELS="$PWD/.cache/ollama" OLLAMA_KEEP_ALIVE=60s /opt/homebrew/bin/ollama serve

Manual setup instead of Install.command

python3.12 -m venv .venv
.venv/bin/python -m pip install httpx==0.28.1 icalendar==7.3.0 recurring-ical-events==3.8.2 anthropic==2.3.0 python-dotenv==1.2.4
.venv/bin/python -m pip install pytest==9.1.1 detect-secrets==1.5.0
OLLAMA_HOST=127.0.0.1:11434 OLLAMA_MODELS="$PWD/.cache/ollama" /opt/homebrew/bin/ollama pull strands-decider:2b-mxfp8

Run the pull while Ollama is serving (the command above), in a second terminal.

Accounts and keys for live alerts

The setup page asks for these and writes .env for you. To do it by hand, copy .env.example to .env and fill in the values. Never paste a token into a chat, a command line or a file in git.

• Telegram. Message @BotFather, send /newbot and follow the prompts. That token is TELEGRAM_BOT_TOKEN. Send your new bot any message, then use Find my chat ID on the setup page for TELEGRAM_CHAT_ID.

• GitHub. Two fine-grained personal access tokens: a read token with no extra permissions (GITHUB_TOKEN), and a separate token for one repository with Actions: Read and write (GITHUB_ACTIONS_WRITE_TOKEN). Re-running CI is offered only for repositories listed in GITHUB_WRITE_REPOS. Watched repositories go in GITHUB_REPOS.

• A sandbox repository with a flaky workflow, so you can make CI fail on cue. Add .github/workflows/ci.yml:

  name: ci
  on:
    push:
    workflow_dispatch:
  jobs:
    flaky:
      runs-on: ubuntu-latest
      steps:
        - run: python3 -c "import random, sys; sys.exit(1 if random.random() < 0.5 else 0)"

• Google Calendar. Settings, your calendar, Integrate calendar, "Secret address in iCal format". That is ICAL_URL. Treat it like a password: anyone with it can read your calendar. If it leaks, use Reset in the same place.

• Anthropic (optional). ANTHROPIC_API_KEY, for Haiku-written alert text. Without it, alerts use the template. A workspace spend limit is a good idea.

Leave DECIDER_FORMAT, DECIDER_THRESHOLD and DECIDER_HOST empty to use the evaluated defaults: format F2, threshold 0.45, host 127.0.0.1:11434.

Results

All numbers come from eval/EVAL_REPORT.md (Dev's evaluation) and eval/lds/LDS_EVAL_REPORT.md (the LDS re-test). The data is 100 synthetic events over 10 days, labelled by Dev. Three days (30 events) chose the threshold; the other seven (70 events, 24 of them labelled "interrupt") were run once. Brackets are 95% bootstrap intervals.

decider	precision	recall	F1	alerts per day, mean (max)
simple rules (frozen baseline)	0.483 [0.318, 0.654]	0.583 [0.391, 0.783]	0.528 [0.357, 0.676]	4.1 (5)
the judge, this app's question, at 0.45	0.577 [0.381, 0.750]	0.625 [0.421, 0.815]	0.600 [0.417, 0.744]	3.7 (5)
the judge, Dev's wording, at 0.40	0.640 [0.440, 0.826]	0.667 [0.458, 0.857]	0.653 [0.478, 0.800]	3.6 (5)
Haiku as the judge (evaluation only)	1.000 [1.000, 1.000]	0.458 [0.250, 0.650]	0.629 [0.400, 0.788]	1.6 (3)

• Why two wordings. Dev's question named him: "Dev maintains this repo and is working. Should Dev be interrupted...". This edition asks about "you" instead. A reworded question can be answered differently, so the LDS re-test chose a new threshold on Dev's three tune days (0.45 instead of 0.40) and ran the 70 test events once, with his events, labels and rules (eval/lds/LDS_EVAL_PLAN.md). Rewording alone moved F1 from 0.65 to 0.60, a reminder to measure before trusting a threshold.
• Haiku was the only decider whose difference from the rules excludes zero, and only for precision: it sent 11 alerts and every one was wanted, but it missed more.
• The limit of 5 a day never caused a miss on the test days.
• Reproduced on another Mac. Re-asking Dev's question about all 100 events on an Apple M5 Max gave scores within 0.0094 of his stored ones and the same precision, recall and F1. Check it on yours with PYTHONPATH=src .venv/bin/python -m eval.check_dev_scores.
• Pilot first. Before building, Dev ran 30 synthetic events through a go/no-go rule (eval/PILOT_REPORT.md). It passed.
• Cost. Dev's whole evaluation and live check used $0.0015 of Haiku calls. The judge itself is free and runs on your Mac.

Dev's first live run (27 October 2026)

A real alert from that run. This edition's template no longer repeats the branch name.

One workflow on a feature branch had failed five times overnight while the laptop was off. Heads-Up sent five identical alerts, used the whole daily limit and blocked a more important failure on main. Dev fixed it without touching the judge, the threshold or the limit: repeated failures of the same workflow and branch now give one alert until they recover, failures that already recovered are only logged, and re-runs are tracked by attempt number. He then verified the fix live: one CI failure gave one alert, tapping Re-run failed CI re-ran the job (GitHub shows attempt #2), and a meeting 14 minutes away triggered an alert whose Snooze 1h worked.

The live run also showed what the synthetic data could not: the judge seemed to rate meetings by how soon they start rather than by what they are. That is a handful of data points, so it is a pattern to test, not a result.

Limits

• Synthetic data, one labeller. The events are cleaner than real ones, so these numbers are likely optimistic. Dev wrote the labels as the maintainer of the watched repository, and a rule check against the written rubric led to 27 corrections before any decider ran.
• Mac only. Dev built and ran it on an Apple M2 with 8 GB of memory; the LDS Team ran the tests, the demo and the re-test on an Apple M5 Max. On 8 GB, loading the model can push memory into swap when heavy apps are open.
• The judge is Ollama's MXFP8 build, not the official weights. On 30 test states it differed from the official model by at most 0.0392 in probability.
• Wide intervals. 70 test events are few, and the bootstrap treats events as independent, so the intervals are probably a little narrow.
• Calendar times without a time zone are read as UTC.
• Calendar buttons are the same as CI buttons. Snooze 1h makes little sense for a meeting starting in minutes; that is a good extension.
• Telegram rate limits are not specially handled: an alert that fails to send stays pending and is retried on the next run.
• --dry-run skips Haiku, so a dry run shows template text.

Security

• Fixed actions. Buttons can only snooze, remind or re-run failed CI jobs. The list is in code, not in settings, and no model chooses it. The only write to GitHub is that re-run, and only for repositories in GITHUB_WRITE_REPOS.
• Your chat only. A tap is accepted only when both the chat and the sender match TELEGRAM_CHAT_ID, which happens only in your one-to-one chat with the bot. Other taps are ignored and logged.
• Buttons expire after 1 hour, enforced in the local database. Each button carries a random token, and a second tap on the same action is "already done".
• Event text is untrusted. The judge and Haiku see only the type, title and a time phrase. Buttons come from code based on the event kind.
• Local only. The dashboard and the judge listen on 127.0.0.1. Every dashboard request must carry the right Host, and every button a random per-session token plus a matching Origin, so another website open in your browser cannot press them. This does not stop other programs running as you on the same Mac.
• Secrets stay out of logs. Only src/headsup/config.py reads .env, request logging is turned down so Telegram and calendar addresses never reach a log, and a secrets scan runs on every commit (detect-secrets).

For learners

The starter is this project with three parts blank: the interrupt decision and the daily limit in src/headsup/policy.py, and tap-to-approve in src/headsup/actions.py. Each has guiding tests that fail until you write it. LEARNERS.md has the steps, the test command for each part and a second exercise: label the 30 pilot events yourself and see how the rules, the judge and Haiku score against your labels.

Licence and credits

• Licence: Apache 2.0 (see LICENSE, NOTICE and LICENSING.md). Dev first released Heads-Up under the MIT License; its notice is kept in NOTICE.
• Strands Decider by Strands Agents (Apache 2.0) is the judge model, used through Ollama's build. Ollama (MIT) serves it on your Mac.
• Libraries: httpx, icalendar, recurring-ical-events (LGPL-3.0-or-later, an ordinary installed dependency), the Anthropic SDK and python-dotenv.
• PLAN.md is Dev's original build plan, kept as the design record.