BE-829: Port over initial durable-kernel crate - #9703
thehabbos007 wants to merge 5 commits into
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
3 Skipped Deployments
|
PR SummaryMedium Risk Overview The workspace registers the crate and pulls in Ships documentation (README), AGPL-3 license, Reviewed by Cursor Bugbot for commit 24042a1. Bugbot is set up for automated code reviews on this repo. Configure here. |
github-actions
Bot
added
area/deps
![semgrep-code-hashintel[bot]](https://avatars.githubusercontent.com/u/22116112?s=60&v=4){kind=small}
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #9703 +/- ##
==========================================
+ Coverage 65.89% 66.35% +0.45%
==========================================
Files 1887 1899 +12
Lines 198443 203334 +4891
Branches 8248 8396 +148
==========================================
+ Hits 130773 134927 +4154
- Misses 66140 66818 +678
- Partials 1530 1589 +59
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
|
You are seeing this message because GitHub Code Scanning has recently been set up for this repository, or this pull request contains the workflow file for the Code Scanning tool. What Enabling Code Scanning Means:
For more information about GitHub Code Scanning, check out the documentation. |
There was a problem hiding this comment.
🟡 Changes recommended
Unbounded effect and snapshot behavior, unenforced codec identity, and an unbounded storage flush affect correctness and operability.
Get a fresh assessment by requesting another Copilot review.
Pull request overview
Adds the initial durable-kernel Rust crate for embedded durable execution, including event journaling, recovery, snapshots, effect orchestration, and deterministic simulation testing.
Changes:
- Implements the public domain, runtime, storage, routing, and compatibility APIs.
- Adds fault-injection, property-based, recovery, and record-size tests.
- Adds examples, documentation, licensing, and workspace integration.
File summaries
| File | Description |
|---|---|
Cargo.toml |
Registers the crate in the workspace. |
libs/@local/durable-kernel/.gitignore |
Excludes local specifications. |
libs/@local/durable-kernel/Cargo.toml |
Defines crate dependencies and features. |
libs/@local/durable-kernel/LICENSE.md |
Adds the AGPL-3 license. |
libs/@local/durable-kernel/README.md |
Documents architecture, usage, and recovery. |
libs/@local/durable-kernel/package.json |
Adds Turborepo package metadata. |
libs/@local/durable-kernel/examples/customer_sync.rs |
Demonstrates durable CRM synchronization. |
libs/@local/durable-kernel/examples/webhook_relay.rs |
Demonstrates durable webhook delivery. |
libs/@local/durable-kernel/src/lib.rs |
Exposes the crate modules. |
libs/@local/durable-kernel/src/domain.rs |
Defines events, folds, executors, and snapshots. |
libs/@local/durable-kernel/src/ids.rs |
Implements content-derived identifiers. |
libs/@local/durable-kernel/src/keyspace.rs |
Defines validated storage paths. |
libs/@local/durable-kernel/src/port.rs |
Defines the low-level domain interface. |
libs/@local/durable-kernel/src/properties.rs |
Catalogs simulation safety properties. |
libs/@local/durable-kernel/src/registry.rs |
Registers durable record declarations. |
libs/@local/durable-kernel/src/routing.rs |
Implements fixed shard routing identifiers. |
libs/@local/durable-kernel/src/runtime.rs |
Runs shards and external effects. |
libs/@local/durable-kernel/src/shard_log/mod.rs |
Implements journal storage and recovery. |
libs/@local/durable-kernel/src/shard_log/command_loop.rs |
Serializes shard commands and state updates. |
libs/@local/durable-kernel/src/sim/mod.rs |
Provides an in-memory fault-injection journal. |
libs/@local/durable-kernel/src/sim/harness.rs |
Generates and verifies deterministic schedules. |
libs/@local/durable-kernel/tests/event_size.rs |
Tests record-size boundaries and recovery. |
Review details
- Files reviewed: 22/23 changed files
- Comments generated: 6
- Review effort level: Balanced
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Merging this PR will not alter performance
|
| Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|
as_constant |
< 1 ns | < 1 ns | N/A | |
constant_equal |
< 1 ns | < 1 ns | N/A | |
constant_not_equal |
< 1 ns | < 1 ns | N/A | |
access |
< 1 ns | < 1 ns | N/A | |
runtime_equal |
< 1 ns | < 1 ns | N/A | |
runtime_not_equal |
< 1 ns | < 1 ns | N/A |
Comparing asa/durable-execution (24042a1) with main (67f60d5)1
Footnotes
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit afca874. Configure here.
GitHub Actions
Error
GitHub Actions
Error
🌟 What is the purpose of this PR?
This PR contains the initial durable execution crate used by the integration framework we want for the HASH Graph. The integrations framework needs resiliency, durability and simple operations, and this should tick off those three requirements.
Most of this code has been reviewed by me over several passes, but obviously most of it is LLM generated. The general flow of the code generation progress has been through rigorous testing under a certain set of properties we wish for the system. The
src/properties.rsmodule contains a list of properties we expect the system to pass on at every step of running (at checkpoints in the execution flow). These properties could also form the foundation for a more formal proof using systems a la TLA+.At a high level, the library is used to orchestrate business logic in an event stream in a durable, resilient fashion. Think of Temporal/Restate/etc. without needing to operate a separate orchestration cluster. Instead this library embeds within programs that need durability, while expecting certain shapes of how the work is laid out.
The
READMEgoes over the details more in depth. For the public API, one supplies trait implementations ofDomainEvent,Fold,SimpleDomain, andExecutoras well as configuringKernelConfig. Two example programs are provided in the directory, see thecustomer_sync.rsexample for a nice introduction.Apologies for the huge PR, any suggestion for how we can split it up/make it more palatable would be appreciated
🔗 Related links
🚫 Blocked by
🔍 What does this change?
@localcratedurable-kernelunder AGPL-3Pre-Merge Checklist 🚀
🚢 Has this modified a publishable library?
This PR:
📜 Does this require a change to the docs?
The changes in this PR:
🕸️ Does this require a change to the Turbo Graph?
The changes in this PR:
🐾 Next steps
🛡 What tests cover this?
Coverage:
❓ How to test this?
cargo test -p durable-kernel --all-features sim::)📹 Demo
Hard to showcase, not very flashy :(