Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,15 @@

Docusaurus `^3.10.x` on the shared `stackql/docusaurus-config` (vendored to `.shared-config/` by `yarn vendor-config` in `prebuild`/`prestart`). Site-local files only: `provider.js` (`providerName`, `providerTitle`), `docusaurus.config.js` (createConfig + registry logos + `config.presets[0][1].docs.showLastUpdateTime = true;`), `sidebars.js`, `scripts/sanitize-docs.mjs`, `src/` (CopyableCode, SchemaTable, StackqlDeployDropdown, theme overrides), `static/` (CNAME `<provider>-provider.stackql.io`, favicons, manifest, registry logos, `stackql-featured-image.png`, `stackql-<provider>-provider-featured-image.png`). No Docusaurus boilerplate (no undraw images, HomepageFeatures, stock README).

`headerContent1.txt` is the front matter + one-paragraph pitch; `headerContent2.txt` is the getting-started page: See also / Installation (`REGISTRY PULL`) / Scope / Authentication (env vars, bash and PowerShell) / the scoping variable / Rate limit / Beta labelling / Example queries - lead with the queries the provider exists for (estate inventory, posture set, the flagship, provisioning with INSERT/UPDATE/EXEC/DELETE, a cross-provider UNION). Examples must be runnable against the generated surface (snake names, quoted keywords).
`headerContent1.txt` is the front matter + one-paragraph pitch; `headerContent2.txt` is the getting-started page: See also / Installation (`REGISTRY PULL`) / Scope / Authentication (env vars, bash and PowerShell) / the scoping variable / Rate limit / Beta labelling / Example Queries - lead with the queries the provider exists for (estate inventory, posture set, the flagship, provisioning with INSERT/UPDATE/EXEC/DELETE, a cross-provider UNION). Examples must be runnable against the generated surface (snake names, quoted keywords).

The example section has one shape on every provider site, so the table of contents reads the same everywhere:

- The H2 is exactly `## Example Queries` and it is the last section of `headerContent2.txt` (docgen appends `## Services` after it).
- The H2 is followed by one intro sentence, `Try the following queries using \`stackql shell\`, or run them from a script or CI pipeline with \`stackql exec\`.`, so the H2 never sits directly on an H3. No two headings anywhere in the file are back to back.
- Each example is an H3 named for what it answers (`### Estate inventory`), with a one-sentence lead-in ending in a colon and one ` ```sql ` block (two at most). Setup material (scope, auth, casing, rate limits, pagination) stays in its own H2 sections above, never inside the examples.
- Plain prose, `-` rather than em dashes, no ASCII arrows; SQL with uppercase keywords, single-quoted strings, semicolon terminated; placeholders in the same style the generated resource pages use.
- Favicons live at the `static/` root (`favicon.ico`, `favicon-16x16.png`, `favicon-32x32.png`, `apple-touch-icon.png`, `safari-pinned-tab.svg`, `site.webmanifest`); the shared config links them root-relative, so copies under `static/img/` are never served as the site icon.

`make docs` = `generate-docs --snake-case-aliases` + `sanitize-docs.mjs` (escapes MDX-hostile description text, annotates env-var-defaulted parameters "required unless X is set", corrects docgen's landing-page resource count which includes each service index). `make website` = `yarn install && yarn build`. Commit `website/docs` after every regeneration so pages stamp with their regeneration date.

Expand Down
40 changes: 38 additions & 2 deletions .github/workflows/build-and-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,36 +42,61 @@ jobs:
- name: stackql version
run: stackql --version

# A repository just created from the template has no pinned spec yet
# (provider-dev/config/spec_pin.json is written by the first
# `make fetch-spec`). Until it exists there is nothing to build or
# test, so those steps are skipped with a notice and the template
# itself stays green; a derived repository runs every step.
- name: Detect whether a spec is pinned
id: pin
run: |
if [ -f provider-dev/config/spec_pin.json ]; then
echo "pinned=true" >> "$GITHUB_OUTPUT"
else
echo "pinned=false" >> "$GITHUB_OUTPUT"
echo "::notice title=No pinned spec::provider-dev/config/spec_pin.json does not exist yet (template state) - the build, drift check, tests and docs steps were skipped. Run 'make fetch-spec' to pin the upstream spec."
fi

# Drift is reported, not fatal, here: the committed pin is what gets
# built and tested. The spec-drift job below opens the review issue.
- name: Verify the pinned spec against upstream (warns on drift)
if: steps.pin.outputs.pinned == 'true'
run: |
if ! make fetch-spec; then
echo "::warning title=Spec drift::The served spec no longer matches provider-dev/config/spec_pin.json - run 'make refresh-spec && make build && make test' and review the diff. Building from the committed pin."
fi

- name: Build provider from the pinned spec
if: steps.pin.outputs.pinned == 'true'
run: make inventory split mappings pre-normalize normalize generate

# spec_pin.json is excluded: fetch-spec refreshes its "fetched" date on
# every run, and pin drift is reported by the step above and by the
# spec-drift job.
- name: Fail on uncommitted generation drift
if: steps.pin.outputs.pinned == 'true'
run: |
git add -N .
if ! git diff --quiet -- provider-dev/openapi provider-dev/config provider-dev/source; then
if ! git diff --quiet -- provider-dev/openapi provider-dev/config provider-dev/source ':(exclude)provider-dev/config/spec_pin.json'; then
echo "Generated output differs from the committed artifacts - run 'make build' and commit."
git diff --stat -- provider-dev/openapi provider-dev/config provider-dev/source
git diff --stat -- provider-dev/openapi provider-dev/config provider-dev/source ':(exclude)provider-dev/config/spec_pin.json'
exit 1
fi

- name: Offline validation
if: steps.pin.outputs.pinned == 'true'
run: make test-offline

- name: Integration tests (mock API)
if: steps.pin.outputs.pinned == 'true'
run: make test-integration

- name: Meta-route tests
if: steps.pin.outputs.pinned == 'true'
run: make test-meta

- name: Generate docs (sanity - the site build runs on the web workflows)
if: steps.pin.outputs.pinned == 'true'
run: make docs

smoke:
Expand Down Expand Up @@ -128,7 +153,18 @@ jobs:
- name: Install dependencies
run: npm ci

- name: Detect whether a spec is pinned
id: pin
run: |
if [ -f provider-dev/config/spec_pin.json ]; then
echo "pinned=true" >> "$GITHUB_OUTPUT"
else
echo "pinned=false" >> "$GITHUB_OUTPUT"
echo "::notice title=No pinned spec::provider-dev/config/spec_pin.json does not exist yet (template state) - there is no pin to compare against."
fi

- name: Fetch and compare against the pin
if: steps.pin.outputs.pinned == 'true'
id: drift
run: |
set +e
Expand Down
4 changes: 3 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,9 @@ provider-dev/
source/ # split + normalized service specs (committed build artifacts)
source-graphql/ # manifest.yaml + ops/*.yaml GraphQL fragments (optional)
openapi/src/<name>/ # generated provider (committed)
docgen/provider-data # headerContent1.txt / headerContent2.txt (landing + getting-started page)
docgen/provider-data # headerContent1.txt / headerContent2.txt (landing + getting-started page;
# examples close the page under "## Example Queries" - intro sentence, then
# H3 + lead-in + sql block per example, see docs-and-ci.md)
views/<service>/views.yaml # optional provider views (spliced at generate time)
tests/
offline_validation.mjs
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,7 @@ make website # yarn install && yarn build (vendors the shared stackql/docusa
make website-start
```

`provider-dev/docgen/provider-data/headerContent1.txt` is the landing-page front matter and pitch; `headerContent2.txt` is the getting-started page (installation, scope, authentication, the scoping variable, rate limit, labelling, example queries - lead with the queries the provider exists for). `website/provider.js` carries the site identity; `website/static/CNAME` the hostname; add `website/static/img/stackql-<name>-provider-featured-image.png`. Commit `website/docs` after every regeneration.
`provider-dev/docgen/provider-data/headerContent1.txt` is the landing-page front matter and pitch; `headerContent2.txt` is the getting-started page (installation, scope, authentication, the scoping variable, rate limit, labelling, example queries - lead with the queries the provider exists for). The examples close the file under an `## Example Queries` heading, the same table-of-contents entry on every provider site: one intro sentence under the H2 (so no heading sits directly on another), then one H3 per example with a one-sentence lead-in ending in a colon and one `sql` block. `website/provider.js` carries the site identity; `website/static/CNAME` the hostname; add `website/static/img/stackql-<name>-provider-featured-image.png`, and keep the favicon files at the `static/` root (the shared config links them root-relative). Commit `website/docs` after every regeneration.

To publish the site: rename `.github/workflows/prod-web-deploy.yml.disabled` and `test-web-deploy.yml.disabled`, enable GitHub Pages (source: GitHub Actions) and add the DNS record:

Expand Down
31 changes: 24 additions & 7 deletions provider-dev/docgen/provider-data/headerContent2.txt
Original file line number Diff line number Diff line change
Expand Up @@ -75,16 +75,27 @@ TODO(template): how the vendor labels beta / alpha / deprecated endpoints and
how the provider mirrors that labelling in each method description. Delete if
the vendor has no such labelling.

## Example queries

TODO(template): lead with the queries the provider exists for, in this order -
the estate inventory, the posture set (config as rows), the flagship
capability, provisioning with INSERT / UPDATE / EXEC / DELETE, and a
cross-provider UNION. Every example must run against the generated surface
(snake_case names, keywords such as `"database"` quoted).
## Example Queries

Try the following queries using `stackql shell`, or run them from a script or CI pipeline with `stackql exec`.

TODO(template): keep the heading text and the sentence above exactly as they
are - `Example Queries` is the table-of-contents entry every provider site
shares, and the sentence is what stops the H2 from sitting directly on an H3
(never place two headings back to back). This must stay the last section of
the file: docgen appends `## Services` after it. Each example is an H3 with a
one-sentence lead-in ending in a colon and one `sql` block (occasionally two).
Lead with the queries the provider exists for, in this order - the estate
inventory, the posture set (config as rows), the flagship capability,
provisioning with INSERT / UPDATE / EXEC / DELETE, and a cross-provider UNION.
Every example must run against the generated surface (snake_case names,
keywords such as `"database"` quoted); plain prose, `-` rather than em dashes.
Delete this paragraph when the examples are written.

### Estate inventory

TODO(template): every service with the fields an operator scans first:

```sql
SELECT id, name, state, region
FROM myprovider.services.services
Expand All @@ -93,13 +104,17 @@ ORDER BY region, name;

### Posture

TODO(template): a configuration setting read out of a nested column, one row per service:

```sql
SELECT name, json_extract(config, '$.setting') AS setting
FROM myprovider.services.services;
```

### Provisioning

TODO(template): create, change and remove a resource with the corresponding SQL verbs, then a lifecycle `EXEC`:

```sql
INSERT INTO myprovider.secrets.secrets (name, value)
SELECT 'stackql-demo', 'value';
Expand All @@ -119,6 +134,8 @@ WHERE name = 'stackql-demo';

### Cross-provider

TODO(template): the same estate alongside a sibling provider in one result set:

```sql
SELECT 'myprovider' AS platform, name, state
FROM myprovider.services.services
Expand Down
Loading