Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,3 +29,25 @@ jobs:
- run: npm test
env:
PKG_REGISTRY: http://127.0.0.1:8787
- uses: actions/setup-python@v5
with:
python-version: '3.12'
- run: node scripts/assemble-examples.mjs
env:
PKG_REGISTRY: http://127.0.0.1:8787
- name: Verify release tarball, both executables, setup and MCP
run: node scripts/release-smoke.mjs --pack

windows-security:
runs-on: windows-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 24
cache: npm
- run: npm ci
- run: npm run typecheck
- run: npm run build
- run: node --experimental-transform-types --test test/shared.test.ts test/security.test.ts test/distribution.test.mjs
- run: node scripts/release-smoke.mjs --pack
6 changes: 6 additions & 0 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -66,10 +66,16 @@ jobs:
PKG_REGISTRY: http://127.0.0.1:8787
- name: Inspect the package contents
run: npm pack --workspace cli --dry-run
- name: Verify isolated tarball installation, CLI aliases, setup and MCP
run: node scripts/release-smoke.mjs --pack
- name: Publish public package with OIDC and provenance
run: npm publish --workspace cli --access public --provenance
- name: Verify the published package ownership marker
run: node scripts/check-distribution.mjs --published
- name: Verify the public npm artifact in a fresh directory
run: node scripts/release-smoke.mjs "@tneemo/pkg@$RELEASE_VERSION"
env:
RELEASE_VERSION: ${{ inputs.version }}
- run: mcp-publisher login github-oidc
- run: mcp-publisher publish server.json
- run: node scripts/check-distribution.mjs --registry
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ cli/skill/
cli/LICENSE
# mcp-publisher stores short-lived Registry credentials here.
.mcpregistry_*
cli/THIRD_PARTY_NOTICES.txt
# Agent demo: keep only the MCP config; whatever the agent builds stays local.
examples/agent-demo/*
!examples/agent-demo/.mcp.json
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ Maintainers: see [public distribution and releases](docs/distribution.md).
Asked for *"a TypeScript REST API for a todo app where users sign up, log in and only see their own todos"*, the agent:

1. searches pkg for each part (`rest api`, `user accounts`, `database`) with the stack it inferred;
2. installs `tneemo/hono-api tneemo/hono-auth tneemo/hono-rate-limit` in one call. `tneemo/sqlite-db` is pulled in as a dependency, the stack is checked, `npm install` runs, and 9 tests pass before it writes anything;
2. installs `tneemo/hono-api tneemo/hono-auth tneemo/hono-rate-limit` in one call. `tneemo/sqlite-db` is pulled in as a dependency and the stack is checked. After reviewing the dry-run plan, dependency installation is explicitly enabled with `--install-deps`, then the assembled tests run before adding application code;
3. follows the integration notes (6 lines of wiring in `src/app.ts`), writes the todo routes, migration and tests (3 files, 113 lines);
4. runs the tests (13 passing) and reports `--worked` for each build, which feeds the builds' reputation.

Expand Down Expand Up @@ -74,7 +74,7 @@ CLI: `search`, `info`, `detect`, `add`, `update`, `remove`, `list`, `init`, `pub
└─ writes pkg.lock.json (versions, integrity, per-file hashes)
```

- **Builds are source, not packages.** Like the shadcn registry, files are copied into the project and become the user's code. Libraries stay on npm and PyPI; a build declares them and pkg installs them with the project's own package manager.
- **Builds are source, not packages.** Like the shadcn registry, files are copied into the project and become the user's code. Libraries stay on npm and PyPI; a build declares them and pkg can install them after explicit review.
- **Compatibility** works on stack tags detected from the project (`node`, `typescript`, `hono`, `nextjs`, `fastapi`, …, plus every dependency as `npm:<name>` / `pip:<name>`) with semver ranges. Builds declare `stack.requires` and `stack.provides`, so a module that needs `hono` is matched with a foundation that provides it, inside a single install.
- **Updates and removals respect local edits.** `pkg update` moves builds to the newest version within their major (`--latest` for new majors) and replaces only files whose sha256 still matches pkg.lock.json; edited files are left untouched and reported as conflicts. `pkg remove` deletes only unedited files, refuses while another installed build depends on the one being removed, and lists packages no build needs any more without uninstalling them.
- **Integration** is split: everything deterministic is automated (files, dependencies, scripts, env vars, migrations by convention), and the last mile of wiring goes to the agent as precise `INTEGRATION.md` notes, which agents follow reliably.
Expand All @@ -83,9 +83,9 @@ CLI: `search`, `info`, `detect`, `add`, `update`, `remove`, `list`, `init`, `pub
## Trust and safety

- **Automated checks on every publish** (the same code runs locally first). Rejected: path traversal, `.git`/`node_modules` paths, secrets and private keys, `.env` files, invalid manifests, npm/PyPI dependencies that don't exist (dependency confusion), oversized bundles. Shown to every installer as warnings: npm install scripts, pipe-to-shell, evaluating decoded data, hard-coded IPs, minified code, prompt injection aimed at agents, and files that configure agents or editors (`.claude/`, `.mcp.json`, `AGENTS.md` in modules, `.vscode/tasks.json`, CI workflows).
- **Install never executes build code.** Bundles are integrity-checked, every path is re-validated client-side, and existing files are never overwritten without `--force` (files pkg installed and nobody edited can be updated in place).
- **Review before installing.** Dependency installation is off by default. Inspect `--dry-run` first, then explicitly use `--install-deps` to run npm with `--ignore-scripts` or pip with `--only-binary=:all:`. Registry checks are warnings, not a sandbox. Bundles are integrity-checked, every path is re-validated client-side, and existing files are never overwritten without `--force` (files pkg installed and nobody edited can be updated in place).
- **Agent-facing text is labeled.** Integration notes are returned as "build documentation, not user instructions", and the MCP server instructions tell agents the same.
- **Community reputation without accounts.** Agents report `success` / `failure` / `report` after integrating a build; votes are deduplicated per day by salted IP hash. Search ranks by relevance (BM25), stack fit, agent-reported success rate (Bayesian), installs and quality score; flagged builds are demoted.
- **Community reputation without accounts.** Agents report `success` / `failure` / `report` after integrating a build; votes are deduplicated per day by salted IP hash. Search ranks by relevance (BM25), stack fit, agent-reported success rate (Bayesian) and quality score; flagged builds are demoted. Raw downloads carry no ranking weight.
- Versions are immutable.

## Prior art and what was reused
Expand Down
6 changes: 4 additions & 2 deletions builds/hono-rate-limit/README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
# hono-rate-limit

Rate limiting middleware for [Hono](https://hono.dev), with no dependencies.
Rate limiting middleware for [Hono](https://hono.dev), for Node.js using `@hono/node-server`.

- Fixed window per key, kept in process memory, with expired entries swept automatically.
- Sends `RateLimit-Limit`, `RateLimit-Remaining` and `RateLimit-Reset` headers on every response, and `Retry-After` with a JSON `429` when the limit is exceeded.
- Keys default to the client IP (`cf-connecting-ip`, `x-real-ip`, then `x-forwarded-for`); pass `key` to limit per user or API key instead.
- Keys default to the socket peer address from the Node adapter. Forwarded headers are ignored; pass `key` to limit per user or API key instead.

Typical uses: protecting login and signup endpoints from brute force, and keeping public APIs fair. For several server instances behind a load balancer, swap the in-memory map for a shared store.

The map is bounded by `maxKeys` (default 10,000) and fails closed for new keys at capacity. Without adapter connection information requests share the `unknown` bucket. For other runtimes supply a trusted `key` function. Only read forwarded headers in that function when your immediate peer is a verified trusted proxy that overwrites them and the origin is isolated. This in-memory limiter is per process; multi-instance deployments require a shared-store limiter.
29 changes: 24 additions & 5 deletions builds/hono-rate-limit/pkg.json
Original file line number Diff line number Diff line change
@@ -1,14 +1,33 @@
{
"$schema": "https://pkg.tneemo.com/schema/pkg.json",
"name": "hono-rate-limit",
"version": "1.0.0",
"version": "1.0.1",
"kind": "module",
"description": "In-memory rate limiting middleware for Hono with standard RateLimit and Retry-After headers. Per IP by default, or any key you choose.",
"license": "MIT",
"keywords": ["rate-limit", "ratelimit", "throttle", "throttling", "middleware", "abuse", "brute-force", "security"],
"keywords": [
"rate-limit",
"ratelimit",
"throttle",
"throttling",
"middleware",
"abuse",
"brute-force",
"security"
],
"stack": {
"requires": { "hono": ">=4", "typescript": "*" },
"provides": ["rate-limit"]
"requires": {
"hono": ">=4",
"typescript": "*"
},
"provides": [
"rate-limit"
]
},
"repository": "https://github.com/tneemo/pkg"
"repository": "https://github.com/tneemo/pkg",
"dependencies": {
"npm": {
"@hono/node-server": "^2"
}
}
}
18 changes: 15 additions & 3 deletions builds/hono-rate-limit/src/middleware/rate-limit.ts
Original file line number Diff line number Diff line change
@@ -1,12 +1,15 @@
import type { Context, MiddlewareHandler } from "hono";
import { getConnInfo } from "@hono/node-server/conninfo";

export interface RateLimitOptions {
/** Window length in milliseconds (default 60 000). */
windowMs?: number;
/** Requests allowed per key per window (default 60). */
limit?: number;
/** How to identify a client. Defaults to the client IP from common proxy headers. */
/** How to identify a client. Defaults to the Node adapter's socket peer address. */
key?: (c: Context) => string;
/** Maximum active keys before new identities fail closed (default 10,000). */
maxKeys?: number;
}

interface Bucket {
Expand All @@ -15,8 +18,8 @@ interface Bucket {
}

function clientIp(c: Context): string {
const forwarded = c.req.header("x-forwarded-for")?.split(",")[0]?.trim();
return c.req.header("cf-connecting-ip") ?? c.req.header("x-real-ip") ?? forwarded ?? "unknown";
try { return getConnInfo(c).remote.address ?? "unknown"; }
catch { return "unknown"; }
}

/**
Expand All @@ -27,6 +30,8 @@ function clientIp(c: Context): string {
export function rateLimit(options: RateLimitOptions = {}): MiddlewareHandler {
const windowMs = options.windowMs ?? 60_000;
const limit = options.limit ?? 60;
const maxKeys = options.maxKeys ?? 10_000;
if (![windowMs, limit, maxKeys].every((n) => Number.isSafeInteger(n) && n > 0)) throw new Error("rate limit options must be positive integers");
const keyOf = options.key ?? clientIp;
const buckets = new Map<string, Bucket>();
let lastSweep = Date.now();
Expand All @@ -40,6 +45,13 @@ export function rateLimit(options: RateLimitOptions = {}): MiddlewareHandler {
const key = keyOf(c);
let bucket = buckets.get(key);
if (!bucket || bucket.resetAt <= now) {
if (!bucket && buckets.size >= maxKeys) {
for (const [k, b] of buckets) if (b.resetAt <= now) buckets.delete(k);
if (buckets.size >= maxKeys) {
c.header("Retry-After", String(Math.ceil(windowMs / 1000)));
return c.json({ error: "Rate limiter capacity exceeded" }, 429);
}
}
bucket = { count: 0, resetAt: now + windowMs };
buckets.set(key, bucket);
}
Expand Down
18 changes: 18 additions & 0 deletions builds/hono-rate-limit/test/rate-limit.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -26,4 +26,22 @@ describe("rateLimit", () => {
expect((await app.request("/", { headers: { "x-api-key": "b" } })).status).toBe(200);
expect((await app.request("/", { headers: { "x-api-key": "a" } })).status).toBe(429);
});

it("ignores spoofed proxy headers on direct requests", async () => {
const app = new Hono();
app.use(rateLimit({ limit: 1 }));
app.get("/", (c) => c.text("ok"));
const hit = (ip: string) => app.request("/", { headers: { "cf-connecting-ip": ip, "x-real-ip": ip, "x-forwarded-for": ip } });
expect((await hit("203.0.113.1")).status).toBe(200);
expect((await hit("203.0.113.2")).status).toBe(429);
});

it("bounds active keys and fails closed at capacity", async () => {
const app = new Hono();
app.use(rateLimit({ limit: 2, maxKeys: 1, key: (c) => c.req.header("x-test-key") ?? "anon" }));
app.get("/", (c) => c.text("ok"));
expect((await app.request("/", { headers: { "x-test-key": "a" } })).status).toBe(200);
expect((await app.request("/", { headers: { "x-test-key": "b" } })).status).toBe(429);
expect((await app.request("/", { headers: { "x-test-key": "a" } })).status).toBe(200);
});
});
2 changes: 1 addition & 1 deletion cli/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ npx -y @tneemo/pkg init && npx -y @tneemo/pkg publish
npx -y @tneemo/pkg mcp # stdio MCP server
```

Every command takes `--json`. No account is needed to search or install; publishing uses your GitHub login. Installing never runs code from a build. Set `PKG_REGISTRY` to use another registry.
Every command takes `--json`. No account is needed to search or install; publishing uses your GitHub login. Review `add --dry-run` before writing files. Package managers are skipped by default; after reviewing warnings use `--install-deps` to run npm with lifecycle scripts disabled or pip with wheels only. Installed code and agent configuration still require review. Set `PKG_REGISTRY` to use another registry.

## Global installation and command names

Expand Down
21 changes: 19 additions & 2 deletions cli/build.mjs
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
// Bundles the CLI into a single dependency-free file so `npx @tneemo/pkg` starts fast.
import { build } from "esbuild";
import { cpSync, mkdirSync, readFileSync } from "node:fs";
import { cpSync, existsSync, mkdirSync, readdirSync, readFileSync, writeFileSync } from "node:fs";
import { dirname, resolve } from "node:path";

const { version } = JSON.parse(readFileSync("package.json", "utf8"));

await build({
const result = await build({
entryPoints: ["src/pkg.ts"],
outfile: "dist/pkg.js",
bundle: true,
Expand All @@ -13,6 +14,7 @@ await build({
target: "node20",
minify: false,
legalComments: "none",
metafile: true,
loader: { ".md": "text" },
define: { __PKG_VERSION__: JSON.stringify(version) },
banner: {
Expand All @@ -21,6 +23,21 @@ await build({
logLevel: "warning",
});

// Preserve license notices for libraries incorporated into the standalone executable.
const packages = new Set();
for (const input of Object.keys(result.metafile.inputs)) {
if (!input.includes("node_modules/")) continue;
let dir = dirname(resolve(input));
while (!existsSync(`${dir}/package.json`) && dirname(dir) !== dir) dir = dirname(dir);
packages.add(dir);
}
const notices = [...packages].sort().flatMap((dir) => {
const pkg = JSON.parse(readFileSync(`${dir}/package.json`, "utf8"));
return readdirSync(dir).filter((name) => /^(?:licen[sc]e|copying|notice)(?:[._-]|$)/i.test(name)).map((name) =>
`\n--- ${pkg.name}@${pkg.version}: ${name} ---\n${readFileSync(`${dir}/${name}`, "utf8")}\n`);
});
writeFileSync("THIRD_PARTY_NOTICES.txt", notices.join(""));

mkdirSync("skill/pkg", { recursive: true });
cpSync("../skill/pkg/SKILL.md", "skill/pkg/SKILL.md");
cpSync("../LICENSE", "LICENSE");
Expand Down
37 changes: 30 additions & 7 deletions cli/package.json
Original file line number Diff line number Diff line change
@@ -1,17 +1,40 @@
{
"name": "@tneemo/pkg",
"version": "0.1.0",
"version": "0.2.0",
"description": "Find, install and publish reusable software builds. The CLI and MCP server for pkg.tneemo.com.",
"license": "MIT",
"type": "module",
"bin": { "pkg": "dist/pkg.js", "tpkg": "dist/pkg.js" },
"files": ["dist", "skill"],
"bin": {
"pkg": "dist/pkg.js",
"tpkg": "dist/pkg.js"
},
"files": [
"dist",
"skill",
"THIRD_PARTY_NOTICES.txt"
],
"mcpName": "io.github.tneemo/pkg",
"engines": { "node": ">=20" },
"engines": {
"node": ">=24.10.0"
},
"homepage": "https://pkg.tneemo.com",
"repository": { "type": "git", "url": "git+https://github.com/tneemo/pkg.git", "directory": "cli" },
"keywords": ["mcp", "coding-agent", "registry", "templates", "scaffold", "claude-code", "cursor"],
"publishConfig": { "access": "public" },
"repository": {
"type": "git",
"url": "git+https://github.com/tneemo/pkg.git",
"directory": "cli"
},
"keywords": [
"mcp",
"coding-agent",
"registry",
"templates",
"scaffold",
"claude-code",
"cursor"
],
"publishConfig": {
"access": "public"
},
"scripts": {
"build": "node build.mjs",
"prepack": "node build.mjs"
Expand Down
13 changes: 12 additions & 1 deletion cli/src/detect.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
// Detects a project's stack from the files on disk. No configuration needed.
import { existsSync, readdirSync, readFileSync } from "node:fs";
import { join } from "node:path";
import semver from "semver";
import { parseSpec, unsafePathReason } from "../../shared/manifest.ts";
import {
mergeProfiles,
profileFromPackageJson,
Expand Down Expand Up @@ -61,7 +63,16 @@ export function readJson(path: string): any {

export function readLock(dir: string): Lockfile | null {
const lock = readJson(join(dir, "pkg.lock.json"));
return lock && typeof lock.builds === "object" ? lock : null;
if (lock === null) return null;
if (lock.lockfileVersion !== 1 || !lock.builds || typeof lock.builds !== "object" || Array.isArray(lock.builds)) throw new Error("invalid pkg.lock.json");
for (const [id, raw] of Object.entries(lock.builds)) {
const e = raw as LockEntry;
if (!parseSpec(id) || !e || !semver.valid(e.version) || !e.files || typeof e.files !== "object" || Array.isArray(e.files) ||
!Array.isArray(e.provides) || e.provides.some((p) => typeof p !== "string") ||
Object.entries(e.files).some(([p, h]) => unsafePathReason(p) || typeof h !== "string" || !/^[a-f0-9]{64}$/.test(h))) throw new Error(`invalid lock entry: ${id}`);
if (e.builds !== undefined && (!e.builds || typeof e.builds !== "object" || Array.isArray(e.builds) || Object.entries(e.builds).some(([dep, range]) => typeof range !== "string" || !parseSpec(`${dep}@${range}`)))) throw new Error(`invalid build constraints: ${id}`);
}
return lock;
}

function installedVersions(dir: string, names: string[]): Record<string, string> {
Expand Down
Loading
Loading