Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
985a750
Make the API docs and the discovery document match the server
0bserver07 Aug 27, 2026
366410f
Let a static token entry name only token_env
0bserver07 Aug 27, 2026
a21a082
Fix the config check invocation in walgit.example.toml
0bserver07 Aug 27, 2026
6f046e3
Say mode none grants admin as well as write in the README
0bserver07 Aug 27, 2026
386cc95
Read the forwarded client credential only when an edge announced it
0bserver07 Aug 27, 2026
4bb3ce6
Classify transient S3 failures as retryable
ethanstoner Aug 27, 2026
7475139
Install libprotobuf-dev in the Containerfile build stage
0bserver07 Aug 27, 2026
da0b3c0
Ignore local reports
tobi Sep 5, 2026
81dc7cb
Fix strict workspace Clippy diagnostics
tobi Sep 5, 2026
35588da
Merge pull request #29 from 0bserver07/fix/containerfile-libprotobuf-dev
tobi Sep 5, 2026
4f515f5
Merge pull request #34 from 0bserver07/fix/forwarded-authorization-ne…
tobi Sep 5, 2026
dbd7d89
Merge pull request #18 from ethanstoner/fix/s3-retryable-error-classi…
tobi Sep 5, 2026
e9df117
Merge pull request #25 from 0bserver07/fix/api-docs-match-server
tobi Sep 5, 2026
8e1b2c5
Merge pull request #26 from 0bserver07/fix/token-env-without-token
tobi Sep 5, 2026
202ebcc
Require an explicit issuer in oidc mode
0bserver07 Aug 27, 2026
b81b15a
fix(ci): strip ANSI escapes in `just warnings` before matching — gate…
gqf2008 Sep 2, 2026
2ba8318
Run the events, follow and policy suites in the fast tier
0bserver07 Aug 27, 2026
072a96b
Build the web UI on Node 24 in CI
0bserver07 Aug 27, 2026
a412a3b
List just clippy in the README developing block
0bserver07 Aug 27, 2026
db28a8b
Guard the dev-store podman socket bootstrap to Linux
0bserver07 Aug 27, 2026
f509b50
Guard the dev-local SPA build on web/dist/repos.js
0bserver07 Aug 27, 2026
1964583
Survive empty array expansion in e2e.sh under bash 3.2
0bserver07 Aug 27, 2026
9f53c7d
Stop e2e synth from reading a local-mode-only config
0bserver07 Aug 27, 2026
924f019
Fix S3 retry classification after the #18 merge
tobi Sep 5, 2026
2c5af1a
Make the #18/#25 merges clippy-clean
tobi Sep 5, 2026
e1589bc
Use assert_eq! in the bundle verify test
tobi Sep 5, 2026
a9da38f
Stop reading past the git bundle header in tests
tobi Sep 5, 2026
e5295e6
Give the integration cargo test invocations 10 minutes
tobi Sep 5, 2026
d5e75ca
Check pushed tips even when the pack is empty
0bserver07 Sep 6, 2026
41faf55
Let just clippy pass on macOS
0bserver07 Sep 7, 2026
9911cb1
Write faulted objects off the async worker
0bserver07 Sep 7, 2026
4149de9
Resolve the GCE machine type at startup, not in a handler
0bserver07 Sep 7, 2026
2fdedc7
Spend the maintenance interval between passes, not before the first
zaeku Sep 10, 2026
66e3e14
Merge pull request #42 from 0bserver07/fix/empty-pack-push-tip-check
tobi Sep 11, 2026
0c8d4b6
Merge pull request #43 from 0bserver07/fix/clippy-clean-on-macos
tobi Sep 11, 2026
8b438d4
Merge pull request #44 from 0bserver07/fix/machine-type-probe-off-the…
tobi Sep 11, 2026
f599feb
Merge pull request #45 from 0bserver07/fix/fault-writes-off-the-worker
tobi Sep 11, 2026
6465bf5
Merge pull request #49 from zaeku/maintain-interval-between-passes
tobi Sep 11, 2026
c0cbaae
chore: merge upstream/main into the fork
yordis Sep 12, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ jobs:
version: 10
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version: 22
node-version: 24
cache: pnpm
cache-dependency-path: web/pnpm-lock.yaml

Expand Down Expand Up @@ -82,7 +82,7 @@ jobs:
version: 10
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version: 22
node-version: 24
cache: pnpm
cache-dependency-path: web/pnpm-lock.yaml
- name: Build the SPA and the SDK
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ target/

# local review dumps (findings, smoketests, notes — not for the repo)
.review/
reports/

# built web assets (embedded into the binary at build time; `just web-build`)
web/dist/
Expand Down
5 changes: 0 additions & 5 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -60,11 +60,6 @@ must_use_candidate = "allow"
similar_names = "allow"
too_many_lines = "allow"

# Every site is an integer widened to f64 for a ratio, a gauge, or a human-readable
# size. There is no lossless spelling to migrate to, so the lint only ever asks for
# an #[allow] at the cast.
cast_precision_loss = "allow"

[workspace.dependencies]
walgit-proto = { path = "crates/walgit-proto" }
walgit-store = { path = "crates/walgit-store" }
Expand Down
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,7 @@ each repository one maintainer (placement globs) and you are done.

| mode | who gets in | how git authenticates |
|---|---|---|
| `none` | everyone is `anon` with write — loopback experiments | nothing |
| `none` | everyone is `anon` with write and admin — loopback experiments | nothing |
| `token` | static `tokens` in the config (`token_env` reads the secret from the environment) | `Authorization: Bearer <token>`, or the token as an HTTP Basic password |
| `oidc` | any OpenID Connect issuer (`issuer`, `oauth_client_id/secret`, `allowed_domains`/`allowed_emails`): Google, Entra, Okta, Auth0, Keycloak, Dex, GitLab… | a **walgit access token**: sign in once in the browser, create one at `/_auth/tokens`, paste it into the installer. Stateless (HMAC with `session_secret`, `access_token_ttl`); rotating the secret revokes all. ID tokens from the issuer (`audiences`) and static `tokens` work too. |

Expand All @@ -156,7 +156,8 @@ and turns on `transfer.bundleURI`. `?repo=owner/name` clones right after.
just test # fast hermetic tier (< 1 min): unit + quick integration, in-memory store, real git
just e2e # real git against the server (~20 s)
just warnings # zero rustc warnings across all targets
just ci # all of the above
just clippy # the [workspace.lints] set across all targets, warnings are errors
just ci # warnings, clippy, test, e2e: everything that must be green before a merge
cargo test -p walgit-server --test sim # fault-injection simulation (crashes, partitions, stale reads)
just test-s3 # store contract against local rustfs
```
Expand Down
21 changes: 15 additions & 6 deletions crates/walgit-bundle/src/lib.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
#![allow(clippy::unused_self, clippy::doc_lazy_continuation)]
//! bundle-uri: scheduled full/incremental bundle strategies, bundle list.
//! See AGENTS.md Phase 5 and docs/CONTRACT.md `walgit-bundle`.
//!
Expand All @@ -6,9 +7,9 @@
//! The [`Bundler`] is the public entry point. It depends on a [`BundleSource`]
//! trait that provides repo-scoped access (local git repo + [`Prefixed`] store
//! + `head_seq`). When `walgit_wal::Registry` lands it will implement
//! `BundleSource` (impl lives in this crate) and the `new` signature will
//! accept `Arc<Registry>` directly. Until then, [`Bundler::new_with_source`]
//! accepts any `BundleSource` impl (used by tests).
//! `BundleSource` (impl lives in this crate) and the `new` signature will
//! accept `Arc<Registry>` directly. Until then, [`Bundler::new_with_source`]
//! accepts any `BundleSource` impl (used by tests).
//!
//! The core operations in [`ops`] take a [`walgit_git::LocalRepo`] + [`Prefixed`]
//! store so they are unit-testable with upstream `git` + [`MemoryStore`] without
Expand Down Expand Up @@ -190,6 +191,7 @@ impl Bundler {
}

fn find_strategy<'a>(
&self,
cfg: &'a Config,
name: &str,
) -> Result<&'a walgit_config::BundleStrategy, BundleError> {
Expand Down Expand Up @@ -234,7 +236,7 @@ impl Bundler {
cut: &ops::Cut,
) -> Result<BundleEntry, BundleError> {
let cfg = self.cfg_for(handle);
let strat = Self::find_strategy(cfg, strategy_name)?;
let strat = self.find_strategy(cfg, strategy_name)?;
let store = &handle.store;
let refs = slots::default_refs(&cfg.bundles, strat);

Expand Down Expand Up @@ -299,7 +301,7 @@ impl Bundler {
.map(|t| t.oid.clone())
.collect();
let commits = ops::count_commits(&handle.local, &tip_oids, &prerequisites).await?;
metrics::histogram!("walgit_bundle_commits", "strategy" => strategy_name.to_string()).record(commits as f64);
metrics::histogram!("walgit_bundle_commits", "strategy" => strategy_name.to_string()).record(metric_u64(commits));
tracing::info!(
strategy = strategy_name,
slot = cut.slot,
Expand Down Expand Up @@ -636,7 +638,7 @@ impl Bundler {
) -> Result<Option<BundleEntry>, BundleError> {
let mut handle = self.source.open_repo(id).await?;
let cfg = self.cfg_for(&handle).clone();
let strat = Self::find_strategy(&cfg, strategy)?.clone();
let strat = self.find_strategy(&cfg, strategy)?.clone();
let strat = &strat;
let store = handle.store.clone();
let Some(lease) = ops::try_acquire_lease(&store, &strat.name, self.lease_ttl).await? else {
Expand Down Expand Up @@ -824,6 +826,13 @@ impl Bundler {
}
}

/// Metrics use `f64`; values beyond its exact integer range are still useful as
/// approximate counters.
#[allow(clippy::cast_precision_loss)]
fn metric_u64(value: u64) -> f64 {
value as f64
}

// ---------------------------------------------------------------------------
// BundleSource impl for walgit_wal::Registry (behind 'wal' feature)
// ---------------------------------------------------------------------------
Expand Down
37 changes: 25 additions & 12 deletions crates/walgit-bundle/src/ops.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
#![allow(clippy::needless_continue, clippy::too_many_arguments)]
//! Core bundling operations: ref resolution, bundle creation, store upload,
//! bundle-list CAS management, pruning, and per-strategy leasing.
//!
Expand Down Expand Up @@ -70,7 +71,7 @@ pub(crate) fn filter_refs(snap: &RefSnapshotData, patterns: &[String]) -> (Vec<S
let effective: Vec<&str> = if patterns.is_empty() {
vec!["refs/heads/*", "refs/tags/*", "HEAD"]
} else {
patterns.iter().map(String::as_str).collect()
patterns.iter().map(std::string::String::as_str).collect()
};

let mut ref_names = Vec::new();
Expand Down Expand Up @@ -230,12 +231,12 @@ pub async fn create_bundle(
"--stdout",
]
.iter()
.map(ToString::to_string)
.map(std::string::ToString::to_string)
.collect();
if let Some(f) = filter {
po_args.push(format!("--filter={f}"));
}
let po_args: Vec<&str> = po_args.iter().map(String::as_str).collect();
let po_args: Vec<&str> = po_args.iter().map(std::string::String::as_str).collect();
let mut child = git(&po_args)
.stdin(std::process::Stdio::piped())
.stdout(std::process::Stdio::piped())
Expand All @@ -247,7 +248,7 @@ pub async fn create_bundle(
let mut stdin = child
.stdin
.take()
.ok_or_else(|| BundleError::Io("git pack-objects stdin".into()))?;
.ok_or_else(|| BundleError::Io("git pack-objects stdin was not piped".into()))?;
stdin
.write_all(revs.as_bytes())
.await
Expand All @@ -268,12 +269,16 @@ pub async fn create_bundle(
let mut stdout = child
.stdout
.take()
.ok_or_else(|| BundleError::Io("git pack-objects stdout".into()))?;
.ok_or_else(|| BundleError::Io("git pack-objects stdout was not piped".into()))?;
let mut first = [0u8; 12];
tokio::io::AsyncReadExt::read_exact(&mut stdout, &mut first)
.await
.map_err(|e| BundleError::Io(format!("pack header: {e}")))?;
let objects = u32::from_be_bytes([first[8], first[9], first[10], first[11]]);
let count_bytes: [u8; 4] = first
.get(8..12)
.and_then(|bytes| bytes.try_into().ok())
.ok_or_else(|| BundleError::Other("pack header lacks an object count".into()))?;
let objects = u32::from_be_bytes(count_bytes);
{
use tokio::io::AsyncWriteExt;
file.write_all(&first)
Expand Down Expand Up @@ -332,7 +337,10 @@ pub fn bundle_checksum_file(path: &std::path::Path) -> std::io::Result<String> {
if n == 0 {
break;
}
hasher.update(buf.get(..n).unwrap_or_default());
let chunk = buf
.get(..n)
.ok_or_else(|| std::io::Error::other("read exceeded checksum buffer"))?;
hasher.update(chunk);
}
Ok(hex::encode(hasher.finalize()))
}
Expand Down Expand Up @@ -460,6 +468,7 @@ where
Ok(meta) => return Ok(Some((meta.version, new_list))),
Err(StoreError::PreconditionFailed { .. }) => {
debug!(attempt, "cas retry: list created by another writer");
continue;
}
Err(e) => return Err(e.into()),
}
Expand All @@ -482,6 +491,7 @@ where
Ok(new_meta) => return Ok(Some((new_meta.version, new_list))),
Err(StoreError::PreconditionFailed { .. }) => {
debug!(attempt, "cas retry: list changed by another writer");
continue;
}
Err(e) => return Err(e.into()),
}
Expand Down Expand Up @@ -800,7 +810,7 @@ pub async fn build_and_upload(
build_span.record("bytes", s);
build_span.record("outcome", "ok");
metrics::histogram!("walgit_bundle_build_seconds", "strategy" => strategy_name.to_string(), "kind" => match kind { BundleKind::Full => "full", BundleKind::Incremental => "incremental" }).record(t_build.elapsed().as_secs_f64());
metrics::histogram!("walgit_bundle_build_bytes", "strategy" => strategy_name.to_string()).record(s as f64);
metrics::histogram!("walgit_bundle_build_bytes", "strategy" => strategy_name.to_string()).record(metric_u64(s));
s
}
Err(BundleError::Git(GitError::Subprocess { stderr, .. }))
Expand Down Expand Up @@ -884,6 +894,13 @@ pub async fn build_and_upload(
Ok(entry)
}

/// Metrics use `f64`; values beyond its exact integer range are still useful as
/// approximate byte counts.
#[allow(clippy::cast_precision_loss)]
fn metric_u64(value: u64) -> f64 {
value as f64
}

/// Find the most recent bundle entry for `strategy` in `list`.
pub fn last_for_strategy<'a>(list: &'a BundleList, strategy: &str) -> Option<&'a BundleEntry> {
list.bundles
Expand Down Expand Up @@ -1012,10 +1029,6 @@ pub fn full_bundle_header(
(h, tips)
}

#[allow(
clippy::too_many_arguments,
reason = "one parameter per input the compose needs; a wrapper struct would only be built and destructured at the call sites"
)]
/// Publish `bundles/<strategy>/<stamp>-<pack>.bundle` = header ∘ `wal/<pack>.pack`
/// by compose (falls back to streaming header + `pack_path` when the store
/// cannot compose; then `pack_path` must be a local file) and return the entry
Expand Down
36 changes: 24 additions & 12 deletions crates/walgit-bundle/src/render.rs
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
#![allow(clippy::too_many_arguments)]
//! Render the bundle list in git's bundle-list config format and protocol v2
//! key=value lines.
//!
//! See: <https://git-scm.com/docs/bundle-uri> and
//! <https://git-scm.com/docs/gitprotocol-v2> (bundle-uri command).

use std::fmt::Write as _;
use std::time::Duration;

use walgit_config::{BundleServe, BundlesConfig};
Expand Down Expand Up @@ -54,10 +54,11 @@ static SIGNING_WARNED: std::sync::LazyLock<std::sync::Mutex<std::collections::Ha

fn warn_signing_once(owner: &str, repo: &str, e: &dyn std::fmt::Display) {
let key = format!("{owner}/{repo}");
let mut seen = SIGNING_WARNED
if SIGNING_WARNED
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner);
if seen.insert(key.clone()) {
.unwrap_or_else(std::sync::PoisonError::into_inner)
.insert(key.clone())
{
tracing::warn!(repo = %key, error = %e, "signed bundle URL failed; serving proxy URIs instead (check the store signing permissions)");
}
}
Expand All @@ -81,10 +82,6 @@ fn proxy_uri(entry: &BundleEntry, owner: &str, repo: &str, base_url: &str) -> St
)
}

#[allow(
clippy::too_many_arguments,
reason = "one parameter per input the render needs; a wrapper struct would only be built and destructured at the call sites"
)]
/// Render the bundle list as git config text (bundle-list format).
///
/// ```ini
Expand Down Expand Up @@ -140,11 +137,26 @@ pub async fn render_list_text(
)
.await?;
out.push('\n');
let _ = writeln!(out, "[bundle \"{}\"]", entry.id);
let _ = writeln!(out, " uri = {uri}");
let _ = writeln!(out, " creationToken = {}", entry.creation_token);
{
let _ =
std::fmt::Write::write_fmt(&mut out, format_args!("[bundle \"{}\"]\n", entry.id));
};
{
let _ = std::fmt::Write::write_fmt(&mut out, format_args!(" uri = {uri}\n"));
};
{
let _ = std::fmt::Write::write_fmt(
&mut out,
format_args!(" creationToken = {}\n", entry.creation_token),
);
};
if !entry.filter.is_empty() {
let _ = writeln!(out, " filter = {}", entry.filter);
{
let _ = std::fmt::Write::write_fmt(
&mut out,
format_args!(" filter = {}\n", entry.filter),
);
};
}
}

Expand Down
2 changes: 1 addition & 1 deletion crates/walgit-bundle/src/schedule.rs
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ fn to_chrono(t: SystemTime) -> DateTime<Utc> {

/// Convert a chrono UTC datetime back to [`SystemTime`].
fn to_system(dt: DateTime<Utc>) -> SystemTime {
UNIX_EPOCH + Duration::from_secs(u64::try_from(dt.timestamp().max(0)).unwrap_or(0))
UNIX_EPOCH + Duration::from_secs(dt.timestamp().max(0).unsigned_abs())
}

/// Next fire time of `schedule` strictly after `after`, or `None` if the
Expand Down
5 changes: 3 additions & 2 deletions crates/walgit-bundle/src/slots.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
#![allow(clippy::doc_lazy_continuation)]
//! Calendar-slot scheduling with backfill.
//!
//! A strategy's cron expression defines **slots** (its fire times). Each
Expand Down Expand Up @@ -181,7 +182,7 @@ pub fn base_for_slot_chain<'a>(
/// * `chain = true`: this strategy's own newest bundle before the slot, **if it is newer than
/// that base** (dailies chain from the weekly onwards; hourlies restart from every new daily
/// instead of chaining across it); else the base.
/// `slot = 0` (a manual cut, "now"): the same with the newest bundles overall.
/// `slot = 0` (a manual cut, "now"): the same with the newest bundles overall.
pub fn base_for_incremental<'a>(
cfg: &BundlesConfig,
list: &'a BundleList,
Expand Down Expand Up @@ -598,7 +599,7 @@ mod tests {
}
fn t(s: &str) -> SystemTime {
let dt = chrono::DateTime::parse_from_rfc3339(s).unwrap();
from_epoch(u64::try_from(dt.timestamp()).unwrap_or(0))
from_epoch(dt.timestamp().max(0).unsigned_abs())
}
fn entry(strategy: &str, slot: u64, base_id: &str) -> BundleEntry {
BundleEntry {
Expand Down
Loading
Loading