Add the PAM access-audit event store - #8230
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #8230 +/- ##
==========================================
+ Coverage 69.56% 69.66% +0.10%
==========================================
Files 2471 2479 +8
Lines 105932 106289 +357
Branches 9601 9615 +14
==========================================
+ Hits 73689 74045 +356
Misses 29779 29779
- Partials 2464 2465 +1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
60f33d0 to
a3135ff
Compare
🤖 Bitwarden Claude Code ReviewOverall Assessment: APPROVE Re-reviewed the net-new Code Review Details
PR Metadata Assessment
|
|
What's the plan here for non-relational storage of these events? We don't store events in the relational database ourselves and it's a self-host fallback. We cannot launch with the assumption that we can use this for our cloud-hosted deployments. |
|
Should we include the rotation daemon stuff https://github.com/bitwarden/server/blob/5640b443833c6f027ac5ef4c472f84199a319d85/src/Sql/dbo/Pam/Tables/AccessAuditEvent.sql |
a3135ff to
ca60268
Compare
|
@withinfocus the plan is to start off storing PAM Audit logs in the relational db; and whenever a larger generic audit solution is in place we will start writing there instead. Feel free to schedule time with me for a higher bandwidth discussion. |
@abergs @withinfocus Bitwarden already has an activity log (Azure table storage). Years ago they decided: for our cloud service, to not keep that in the main database — it's too much writing, too fast, and it would slow down the database that holds everyone's actual vaults. So in cloud, event logs get shipped off to separate cheap storage (Azure Table Storage). The SQL version only gets switched on for customers who host Bitwarden themselves. |
withinfocus
left a comment
There was a problem hiding this comment.
Using this relational table for cloud is just not something we can do. It's not a lot of extra coding to set up the Table Storage writer as @rkac-bw described and that's where I also expected it to go.
|
Also, as I take a step back I don't understand why this is not just using the existing event storage rails. We do not need a new table and must move forward with an approach that works for all events, albeit in your case with additional data points you wish to store. |
After discussion with the team on their plans, we're accepting an interim state while they explore additional storage options. The team accepts the risks of this interim state.
7bd4bb8 to
ac1f870
Compare
b60d271 to
eb84453
Compare
Append-only store for PAM access and rotation audit events, with Dapper and EF repositories, the SSDT table and stored procedures, and migrations for all four databases. Display names are snapshotted at write time so an event survives a later delete or rename of what it references.
Bring the new files in line with the repository's standard formatting so the automated style check passes.
The DbScripts migration was dated 2026-08-18, which is before 2026-08-25_00_AddPamCollectionReads.sql on main, so the "Validate new migration naming and order" gate rejected it. Re-date to 2026-08-31 so DbUp applies it after everything already on main.
.claude/rules/database-dapper.md requires datetime columns to end in Date.
The audit store had a single read that returned an organization's whole trail unfiltered. It is replaced by AccessAuditEvent_ReadPageByOrganizationId, which takes a date range, filters on kind, actor, requester, cipher and rule, and returns one page at a time; each action's attempt/outcome pair is now collapsed in the store rather than by the caller, so a pair split across a page boundary still reads as one entry. A second procedure, AccessAuditEvent_ReadItemsByOrganizationId, lists the ciphers and rules the trail names in a range, which is what the Item filter's menu is built from. The trail index gains included columns covering both reads, and a new index on CorrelationId serves the collapse. Ported from c362bb8 on pam/uat and folded into this branch's existing migration, since the store has not shipped.
eb84453 to
df9a0b1
Compare
|
@rkac-bw Resolved the last discussion point. Would appreciate a new review if and when you have time. Thanks! |
| [SyncState] TINYINT NULL, | ||
| CONSTRAINT [PK_AccessAuditEvent] PRIMARY KEY CLUSTERED ([Id] ASC), | ||
| CONSTRAINT [FK_AccessAuditEvent_Organization] FOREIGN KEY ([OrganizationId]) | ||
| REFERENCES [dbo].[Organization] ([Id]) ON DELETE CASCADE |
There was a problem hiding this comment.
Just an FYI: Our last DB migration with an ON DELETE CASCADE to Organization caused a brief period of blocking (~30 seconds). That doesn't necessarily mean this will cause it again, but we should be aware that it could occur.
mkincaid-bw
left a comment
There was a problem hiding this comment.
Just some minor formatting nitpicks.
| FROM (SELECT 1 AS [X]) Seed | ||
| LEFT JOIN [dbo].[User] AU ON AU.[Id] = @ActorId | ||
| LEFT JOIN [dbo].[User] RU ON RU.[Id] = @RequesterId |
There was a problem hiding this comment.
⛏️ Minor formatting nitpick but most SQL keywords should be on their own line
| FROM (SELECT 1 AS [X]) Seed | |
| LEFT JOIN [dbo].[User] AU ON AU.[Id] = @ActorId | |
| LEFT JOIN [dbo].[User] RU ON RU.[Id] = @RequesterId | |
| FROM | |
| (SELECT 1 AS [X]) Seed | |
| LEFT JOIN | |
| [dbo].[User] AU ON AU.[Id] = @ActorId | |
| LEFT JOIN | |
| [dbo].[User] RU ON RU.[Id] = @RequesterId |
See https://contributing.bitwarden.com/contributing/code-style/sql/#select-statements
| FROM [dbo].[AccessAuditEvent] | ||
| WHERE [OrganizationId] = @OrganizationId | ||
| AND [OccurredDate] >= @StartDate | ||
| AND [OccurredDate] <= @EndDate | ||
| AND [CipherId] IS NOT NULL |
There was a problem hiding this comment.
⛏️ Same formatting nitpick (and throughout the rest of the SQL code).
| FROM [dbo].[AccessAuditEvent] | |
| WHERE [OrganizationId] = @OrganizationId | |
| AND [OccurredDate] >= @StartDate | |
| AND [OccurredDate] <= @EndDate | |
| AND [CipherId] IS NOT NULL | |
| FROM | |
| [dbo].[AccessAuditEvent] | |
| WHERE | |
| [OrganizationId] = @OrganizationId | |
| AND [OccurredDate] >= @StartDate | |
| AND [OccurredDate] <= @EndDate | |
| AND [CipherId] IS NOT NULL |
| -- LeaseActivationRejected, so filtering before the collapse would answer "activated" with an action that was | ||
| -- turned down. | ||
| AND ( | ||
| @Kinds IS NULL |
There was a problem hiding this comment.
I realize these procs are temporary while the team explores other storage options, but I wanted to point out that this type of catch-all dynamic search pattern is a known SQL Server anti-pattern. We won't see performance issues until the table grows but depending on how long it takes for the new storage options, this stored proc could eventually have real performance issues.
🎟️ Tracking
PM-39047
📔 Objective
Adds the append-only store behind the PAM access-audit trail: the
AccessAuditEventtable, its three storedprocedures, migrations for MSSQL and the three EF providers, and the Dapper and EF implementations of
IAccessAuditEventRepository.Nothing calls it yet, and that's deliberate. The emitter that writes events and the endpoint that reads the trail
back are separate PRs. Landing the persistence layer on its own keeps the schema reviewable without a feature's
worth of code wrapped around it. The parts DB Ops care about are the whole diff here, not a corner of it.
The MSSQL side is one consolidated net-new script (
2026-08-31_00_AddAccessAuditEvent.sql) rather than theincremental steps the store went through in development, because the feature has not shipped and there is nothing
to roll forward from.
Four design decisions look like mistakes if you don't know the intent:
Rows are self-contained.
AccessAuditEvent_Createsnapshots the actor, requester, rule, target system, anddaemon display names into the row at write time. Reading the trail then touches no other table, and a later rename
can't rewrite history. Actor and requester names are resolved by id from
[User]in the procedure; the rule, targetsystem, and daemon names come from the caller instead of a join, because those entities can be deleted or renamed by
the very action being recorded. The subject ids are deliberately not foreign keyed for the same reason: an audit
event has to outlive what it references. Only
OrganizationIdis, so the rows are removed with the organization.No vault data lands here.
Every snapshotted name is plaintext, so the subject cipher and collection are recorded
by id only, and there are no
CipherName/CollectionNamecolumns. Labelling those subjects is the caller's job,from the vault it has already decrypted.
RuleNameis stored because an access rule's name is plaintextorganization configuration, not vault data.
The read collapses each action to one row, then filters.
An action writes an
Attemptbefore its point of noreturn and an
Outcomeafter, sharing aCorrelationId. The read returns theOutcomewhere the action landed andthe lone
Attemptwhere it didn't, which the caller flags as in-doubt. The collapse happens in the store rather thanin the caller, because a caller holding one page cannot tell an
AttemptwhoseOutcomesits on the next page fromone that never landed. It is scoped to the filter's own range, so an action straddling a bound reads as in-doubt at
that edge instead of vanishing. The filters then apply to whichever row survived, because the two halves need not
agree, and a refused activation writes
LeaseActivatedthenLeaseActivationRejected, so filtering first wouldanswer "activated" with an action that was turned down.
Paging is keyset, not offset.
Before/BeforeIdcarry the last row of the previous page. The store isappend-only and read newest-first, so an offset would re-serve rows whenever an event was written between two
requests, and would get slower with depth.
Idis the third key column of the covering index purely so that ordercomes straight off the index:
OccurredDatealone is not unique, since an action'sAttemptandOutcomeshare atimestamp, and without a tiebreaker a page boundary landing among events that share an instant cannot be resumed
exactly.