Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions .github/workflows/check.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
name: Sift checks

on:
pull_request:
push:
branches: [main]

permissions:
contents: read

jobs:
verify:
strategy:
fail-fast: false
matrix:
os: [windows-latest, ubuntu-22.04, macos-latest]
runs-on: ${{ matrix.os }}
timeout-minutes: 15
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 24.19.0
- run: npm install --global pnpm@11.19.0
- run: pnpm install --frozen-lockfile
- run: pnpm run notices
- run: pnpm run build
- run: pnpm test
- run: pnpm audit --audit-level=low
- run: pnpm run release
- run: pnpm run verify:package
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# Changelog

## 0.1.2 — 2026-10-02

Updated compatible `fast-uri` and `ip-address` dependencies used by the MCP SDK. The complete dependency audit now reports no known advisories. The release bundles the updated runtime.

Added Windows, Linux and macOS automation for frozen installation, build, the existing test suite, dependency audit, archive creation and installation into a fresh project. Packaged verification exercises the UI service and actual stdio MCP without depending on the source checkout.

The collection header and tabs now wrap on small screens, keeping Add source and Connect reachable at 320 pixels. Notice generation follows the installed dependency graph instead of including obsolete versions left in pnpm's store.

## 0.1.1 — 2026-09-19

Fixed copied MCP configuration for extracted installations, Windows command lookup, and custom workspaces. Connect now includes the absolute Node executable, Sift CLI, database path, and selected collection. A regression check launches the exact rendered configuration with an empty PATH and reads fixture evidence from the intended workspace.
Expand Down
14 changes: 7 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,32 +4,32 @@

Collect the public documentation your project needs, search it locally, and let your coding assistant retrieve passages with source URLs and revision history through MCP.

[Website](https://sift-docs.alx21.chatgpt.site/) · [Download 0.1.1](https://sift-docs.alx21.chatgpt.site/downloads/sift-local-0.1.1.tgz) · [Getting started](docs/GETTING_STARTED.md) · [MCP setup](docs/MCP.md)
[Website](https://sift-docs.alx21.chatgpt.site/) · [Download 0.1.2](https://sift-docs.alx21.chatgpt.site/downloads/sift-local-0.1.2.tgz) · [Getting started](docs/GETTING_STARTED.md) · [MCP setup](docs/MCP.md)

## What Sift does

1. **Collect:** choose public documentation URLs, allowed paths, and optional version labels. Refresh manually when you need new content.
2. **Inspect:** search retained pages, open their original sources, compare refresh outcomes, and read previous revisions.
3. **Connect:** expose selected collections to a local coding assistant through five read only MCP tools.

Sift 0.1.1 is a local application for one user. Your collections live in a SQLite database on your computer. No Sift account, model API key, paid service, or telemetry is required. The public website provides information and downloads; the application runs on your machine.
Sift 0.1.2 is a local application for one user. Your collections live in a SQLite database on your computer. No Sift account, model API key, paid service, or telemetry is required. The public website provides information and downloads; the application runs on your machine.

## Quick start

Requires **Node.js 24.15 or later within the Node 24 release line**. Select Node 24 on the [official download page](https://nodejs.org/en/download), then open a new terminal and check `node --version`. Node 22 and Node 25 or later are not supported by this release. Windows has been tested; macOS and Linux execution has not yet been verified.
Requires **Node.js 24.15 or later within the Node 24 release line**. Select Node 24 on the [official download page](https://nodejs.org/en/download), then open a new terminal and check `node --version`. Node 22 and Node 25 or later are not supported by this release. Clean package installation, CLI, HTTP interface and stdio MCP checks passed on Windows, Ubuntu 22.04 and macOS with Node 24.19.0. The browser walkthrough ran on Windows; see the [verification record](docs/verification-2026-10-02.md) for scope.

1. Download the [Sift archive](https://sift-docs.alx21.chatgpt.site/downloads/sift-local-0.1.1.tgz) and [SHA256 checksum](https://sift-docs.alx21.chatgpt.site/downloads/sift-local-0.1.1.tgz.sha256).
1. Download the [Sift archive](https://sift-docs.alx21.chatgpt.site/downloads/sift-local-0.1.2.tgz) and [SHA256 checksum](https://sift-docs.alx21.chatgpt.site/downloads/sift-local-0.1.2.tgz.sha256).
2. Put the archive in a new folder you will keep, and open a terminal in that folder. See the [installation guide](docs/GETTING_STARTED.md) for checksum commands and Windows instructions.
3. Extract and start Sift:

```sh
tar -xzf sift-local-0.1.1.tgz
tar -xzf sift-local-0.1.2.tgz
node package/dist/cli.js version
node package/dist/cli.js doctor
node package/dist/cli.js start
```

Expect `Sift 0.1.1`, a doctor result with `"ok": true`, and a startup message containing `http://127.0.0.1:4587`.
Expect `Sift 0.1.2`, a doctor result with `"ok": true`, and a startup message containing `http://127.0.0.1:4587`.

4. Open **http://127.0.0.1:4587** in your browser. Keep the terminal running. Press **Ctrl+C** to stop.

Expand Down Expand Up @@ -58,7 +58,7 @@ For a small, repeatable walkthrough, use the [one page first collection](docs/GE
| Find my data, back it up, upgrade, or troubleshoot | [Operations](docs/OPERATIONS.md) |
| Build from source or propose a change | [Development and contributions](CONTRIBUTING.md) |
| Understand crawling, storage, and access boundaries | [Architecture](docs/ARCHITECTURE.md) |
| See what was tested and what remains unverified | [Current acceptance checks](docs/REAL_WORLD_VERIFICATION.md) · [0.1.0 report](docs/VERIFICATION.md) |
| See what was tested and what remains unverified | [October 2 acceptance checks](docs/verification-2026-10-02.md) · [0.1.1 checks](docs/REAL_WORLD_VERIFICATION.md) · [0.1.0 report](docs/VERIFICATION.md) |
| Prepare a release | [Release procedure](docs/RELEASING.md) |
| See release history and dependency notices | [Changelog](CHANGELOG.md) · [Third party notices](THIRD_PARTY_NOTICES.md) |

Expand Down
4 changes: 2 additions & 2 deletions THIRD_PARTY_NOTICES.md
Original file line number Diff line number Diff line change
Expand Up @@ -2951,7 +2951,7 @@ SOFTWARE.



## fast-uri@3.1.7
## fast-uri@3.1.8

Declared license: "BSD-3-Clause"

Expand Down Expand Up @@ -3582,7 +3582,7 @@ THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLI



## ip-address@10.7.0
## ip-address@10.7.2

Declared license: "MIT"

Expand Down
20 changes: 10 additions & 10 deletions docs/GETTING_STARTED.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,36 +12,36 @@ Install **Node.js 24.15 or later within Node 24** from the [official Node.js dow
node --version
```

Expect `v24.15.0` or a newer `v24.x.x`. Other major versions are not supported by Sift 0.1.1. Windows with Node 24.19.0 has been tested. The macOS and Linux instructions have not been executed on those platforms.
Expect `v24.15.0` or a newer `v24.x.x`. Other major versions are not supported by Sift 0.1.2. With Node 24.19.0, clean package installation, CLI, HTTP interface and stdio MCP checks passed on Windows, Ubuntu 22.04 and macOS. The complete browser walkthrough ran on Windows. See the [verification record](verification-2026-10-02.md) for exact scope; the manual checksum commands below have not been executed on macOS/Linux.

## Download and verify

Download both files into the same new folder, such as a folder named `Sift` in your Documents directory:

1. [sift-local-0.1.1.tgz](https://sift-docs.alx21.chatgpt.site/downloads/sift-local-0.1.1.tgz)
2. [sift-local-0.1.1.tgz.sha256](https://sift-docs.alx21.chatgpt.site/downloads/sift-local-0.1.1.tgz.sha256)
1. [sift-local-0.1.2.tgz](https://sift-docs.alx21.chatgpt.site/downloads/sift-local-0.1.2.tgz)
2. [sift-local-0.1.2.tgz.sha256](https://sift-docs.alx21.chatgpt.site/downloads/sift-local-0.1.2.tgz.sha256)

Keep their original filenames. Open a terminal in that folder. On Windows, open the folder in File Explorer, right click an empty area, and choose **Open in Terminal**. Use a fresh folder so extraction will not overwrite another application named `package`.

In **PowerShell**, compare the download with its checksum:

```powershell
$siftExpectedHash = ((Get-Content .\sift-local-0.1.1.tgz.sha256 -Raw).Trim() -split '\s+')[0]
$siftActualHash = (Get-FileHash .\sift-local-0.1.1.tgz -Algorithm SHA256).Hash
$siftExpectedHash = ((Get-Content .\sift-local-0.1.2.tgz.sha256 -Raw).Trim() -split '\s+')[0]
$siftActualHash = (Get-FileHash .\sift-local-0.1.2.tgz -Algorithm SHA256).Hash
if ($siftActualHash -ne $siftExpectedHash) { throw 'Checksum mismatch. Download the archive again.' }
'Checksum matches.'
```

On **macOS**, use:

```sh
shasum -a 256 -c sift-local-0.1.1.tgz.sha256
shasum -a 256 -c sift-local-0.1.2.tgz.sha256
```

On **Linux**, use:

```sh
sha256sum -c sift-local-0.1.1.tgz.sha256
sha256sum -c sift-local-0.1.2.tgz.sha256
```

Expect `Checksum matches.` in PowerShell, or a result ending in `OK` on macOS/Linux. A mismatch means the file does not match the published checksum; do not continue with that copy.
Expand All @@ -51,7 +51,7 @@ Expect `Checksum matches.` in PowerShell, or a result ending in `OK` on macOS/Li
From the same folder:

```sh
tar -xzf sift-local-0.1.1.tgz
tar -xzf sift-local-0.1.2.tgz
node package/dist/cli.js version
node package/dist/cli.js doctor
node package/dist/cli.js start
Expand All @@ -61,7 +61,7 @@ The archive creates a `package` folder. Keep it where you intend to run Sift, es

| Step | Expected result |
| --- | --- |
| `version` | `Sift 0.1.1` |
| `version` | `Sift 0.1.2` |
| `doctor` | JSON containing `"ok": true`, `"integrity": "ok"`, and the database path |
| `start` | A message containing `http://127.0.0.1:4587` and the data path |

Expand Down Expand Up @@ -108,7 +108,7 @@ Version labels describe what you know about the source; Sift does not infer vers
If Node's npm command is available, you can install the downloaded archive globally instead of using its extracted CLI:

```sh
npm install --global ./sift-local-0.1.1.tgz
npm install --global ./sift-local-0.1.2.tgz
sift version
sift doctor
sift start
Expand Down
6 changes: 4 additions & 2 deletions docs/MCP.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Sift provides five read-only tools over **local stdio**. Your assistant starts t

1. [Install and start Sift](GETTING_STARTED.md), create a collection, and refresh a source. Verify that Search finds a passage.
2. Open that collection's **Connect** tab and choose **Test MCP connection**. Expect successful initialization and five tool names. This checks Sift itself, not your assistant's configuration.
3. Choose **Copy configuration**. In Sift 0.1.1, it includes the absolute Node executable, installed CLI, database file, and selected collection ID. Extracted archives do not require a global sift command.
3. Choose **Copy configuration**. It includes the absolute Node executable, installed CLI, database file, and selected collection ID. Extracted archives do not require a global sift command.
4. Add the server to your local assistant's MCP settings, preserving any existing servers. Restart or reconnect the assistant's MCP server.
5. Ask the assistant to use Sift's list_collections, then search_docs for a term you already found in Sift. Check that its answer contains the expected source URL. Use read_document for the complete retained revision.

Expand All @@ -30,11 +30,13 @@ Single quoted TOML strings preserve Windows backslashes. JSON clients use the JS

The management interface can be stopped after collection; MCP reads the persisted database independently. Refreshing sources still requires the management process. No OpenAI API key is needed by Sift; your assistant may have its own account requirements.

On October 2, 2026, Codex app-server 0.159.2 on Windows launched the extracted 0.1.2 archive and successfully called all five tools. Search returned the collected Python tutorial with its source URL; document pagination and collection isolation also passed. This verifies the Codex host's MCP connection and tool execution. It does not establish another assistant's setup or how a model uses retrieved material. See the [verification record](verification-2026-10-02.md).

## If the connection fails

| Symptom | What to check |
| --- | --- |
| Executable not found or ENOENT | Copy configuration from Sift 0.1.1; check that Node and the CLI still exist at those paths. |
| Executable not found or ENOENT | Copy configuration from the installed Sift app; check that Node and the CLI still exist at those paths. |
| Missing database or collection | Confirm the database path matches Backup & restore, and copy the selected collection's actual ID. |
| No search results | Refresh the source, try fewer query words, and confirm Search works in the browser. |
| Built in test passes, assistant fails | The test uses Sift's own paths. Check the assistant's saved configuration, machine, and filesystem access. |
Expand Down
2 changes: 2 additions & 0 deletions docs/REAL_WORLD_VERIFICATION.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Real-world verification: Sift 0.1.1

This is the September 19 record. See [October 2 verification](verification-2026-10-02.md) for the current 0.1.2 packaged workflow, mobile correction and dependency checks.

Checked on September 19, 2026, using Windows, Node 24.19.0, and pnpm 11.19.0. These are operator-run acceptance checks against live public documentation, not a study with recruited users or a claim of universal site compatibility.

## Starting from the public download
Expand Down
2 changes: 1 addition & 1 deletion docs/RELEASING.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ Use a clean source checkout, Node 24.15 or later within Node 24, and pnpm 11.19.
1. Update the package version and the version constant in `src/model.ts`; update the changelog and current download links in README and Getting started. The UI displays the backend's version. Keep historical verification reports and old artifact checksums unchanged.
2. Install with `pnpm install --frozen-lockfile`.
3. Run `pnpm run build` and `pnpm test`.
4. Run `node scripts/notices.mjs` to collect license declarations/text from the installed exact dependency tree; review notices.
4. Run `pnpm run notices` to collect license declarations/text from the installed exact dependency graph; review notices. Old package versions left in pnpm's store are excluded.
5. Run `pnpm run release`. This creates `artifacts/sift-local-VERSION.tgz` and a SHA-256 file. The backend, UI, migration, docs, examples and notices are included. Runtime npm dependencies/install scripts are omitted because the runtime is bundled.
6. Run `pnpm run verify:package`. It installs the archive for the current package version in a fresh temporary project with scripts disabled, then checks version, doctor, the interface, a management mutation, and a real stdio MCP connection. Also extract the archive into a fresh folder and try the README path. Verify the checksum and archive contents before distribution.
7. Update the verification report with actual platform and test evidence. Review `git diff`, scan tracked files for data/secrets, commit the lockfile, and tag the verified commit.
Expand Down
37 changes: 37 additions & 0 deletions docs/verification-2026-10-02.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# October 2, 2026 UTC verification

Local environment: Windows, Node 24.19.0, pnpm 11.19.0 and Playwright Chromium 153.0.8010.12. All acceptance work used a new extracted archive and a separate disposable database. Existing collections were not used.

## Packaged application and live sources

The 0.1.2 archive passed installation into a fresh temporary project, version, doctor, the packaged interface, a management mutation and actual stdio MCP discovery/listing. It runs without development dependencies or the source checkout.

The extracted archive also passed a browser walkthrough: create the first collection, add and refresh three live public sources, search each source, open a retained revision with its original URL and SHA-256, inspect refresh outcomes, view changes and run the built-in MCP diagnostic. Each source used a one-page budget, depth zero and concurrency one:

| Public source | Allowed path | Retained pages | Search |
| ---------------------------------------------------------------- | ------------------ | -------------- | --------------- |
| [Example Domain](https://example.com/) | `/` | 1 | `documentation` |
| [Python tutorial](https://docs.python.org/3/tutorial/index.html) | `/3/tutorial` | 1 | `interpreter` |
| [Node.js errors](https://nodejs.org/api/errors.html) | `/api/errors.html` | 1 | `error code` |

The crawl used the ordinary production transport, including DNS/private-address checks, path scope, robots policy and request limits. These bounded results do not establish complete documentation coverage or general retrieval quality.

## MCP, persistence and recovery

A separate SDK client launched the configuration actually rendered by Connect, with an empty PATH. All five tools returned successful results: `list_collections`, `get_collection_profile`, `search_docs`, `read_document` and `list_changes`. Document pagination returned different successive content pages. A collection outside the explicit allowlist returned `forbidden_collection`.

Codex app-server 0.159.2 then launched that extracted installation through its native stdio MCP configuration, using an ephemeral runtime with the management server stopped. All five tools passed through Codex's `mcpServer/tool/call` API. Search returned the retained Python tutorial and original source URL. Successive document pages differed, and the unexposed collection returned `forbidden_collection`. This exercises the actual Codex host without changing the user's saved server configuration; no model conversation or inference request was run.

The interface created a verified online backup, added a temporary collection, and restored the backup while the MCP reader remained connected. The added collection disappeared and the original retained revision was still retrievable. After stopping the server, CLI doctor passed. Restarting the same extracted installation preserved the restored collections and revision IDs.

## Corrections and automated checks

At 320 pixels, the collection header action and tab row extended beyond the viewport. The header and tabs now wrap. The extracted interface passed Connect layout checks at 1440, 390 and 320 pixels with no horizontal overflow or page errors.

The initial complete dependency audit reported three moderate advisories in `fast-uri` and `ip-address`. Compatible updates reduced the audit to zero known advisories at all severity levels. Notice generation now follows the installed dependency graph, excluding obsolete versions retained in pnpm's store.

Production build and TypeScript checks passed. All 18 existing tests passed, including actual stdio MCP, collection isolation, refresh/recovery, safe rendering and network boundaries. The fixture retrieval benchmark returned 5/5 expected passages in the top three; this is a small deterministic fixture score.

The [verified workflow run](https://github.com/agammann/sift/actions/runs/36968089195) passed frozen installation, notice generation, build, all 18 tests, complete dependency audit, release creation and clean package verification on Windows, Ubuntu 22.04 and macOS with Node 24.19.0. Each platform installed its archive into a fresh temporary project and verified version, persistent database, doctor, the packaged HTTP interface, a management mutation and actual stdio MCP initialization/discovery/listing. The complete rendered browser walkthrough above ran on Windows.

Assistant UI enrollment and model use of retrieved material, independent participant onboarding, large-corpus load, disk exhaustion and power-loss recovery remain unverified. Sift supports public server-rendered HTML only. Its public website is a download/information page, and its management address is not an HTTP MCP endpoint.
Loading
Loading