Skip to content

Add the PAM access-audit event store - #8230

Open
patriksvensson wants to merge 6 commits into
mainfrom
psvensson/pam-audit-log-scaffolding
Open

Add the PAM access-audit event store#8230
patriksvensson wants to merge 6 commits into
mainfrom
psvensson/pam-audit-log-scaffolding

Conversation

@patriksvensson

@patriksvensson patriksvensson commented Aug 19, 2026

Copy link
Copy Markdown
Member

🎟️ Tracking

PM-39047

📔 Objective

Adds the append-only store behind the PAM access-audit trail: the AccessAuditEvent table, its three stored
procedures, migrations for MSSQL and the three EF providers, and the Dapper and EF implementations of
IAccessAuditEventRepository.

Nothing calls it yet, and that's deliberate. The emitter that writes events and the endpoint that reads the trail
back are separate PRs. Landing the persistence layer on its own keeps the schema reviewable without a feature's
worth of code wrapped around it. The parts DB Ops care about are the whole diff here, not a corner of it.

The MSSQL side is one consolidated net-new script (2026-08-31_00_AddAccessAuditEvent.sql) rather than the
incremental steps the store went through in development, because the feature has not shipped and there is nothing
to roll forward from.

Four design decisions look like mistakes if you don't know the intent:

Rows are self-contained.

AccessAuditEvent_Create snapshots the actor, requester, rule, target system, and
daemon display names into the row at write time. Reading the trail then touches no other table, and a later rename
can't rewrite history. Actor and requester names are resolved by id from [User] in the procedure; the rule, target
system, and daemon names come from the caller instead of a join, because those entities can be deleted or renamed by
the very action being recorded. The subject ids are deliberately not foreign keyed for the same reason: an audit
event has to outlive what it references. Only OrganizationId is, so the rows are removed with the organization.

No vault data lands here.

Every snapshotted name is plaintext, so the subject cipher and collection are recorded
by id only, and there are no CipherName/CollectionName columns. Labelling those subjects is the caller's job,
from the vault it has already decrypted. RuleName is stored because an access rule's name is plaintext
organization configuration, not vault data.

The read collapses each action to one row, then filters.

An action writes an Attempt before its point of no
return and an Outcome after, sharing a CorrelationId. The read returns the Outcome where the action landed and
the lone Attempt where it didn't, which the caller flags as in-doubt. The collapse happens in the store rather than
in the caller, because a caller holding one page cannot tell an Attempt whose Outcome sits on the next page from
one that never landed. It is scoped to the filter's own range, so an action straddling a bound reads as in-doubt at
that edge instead of vanishing. The filters then apply to whichever row survived, because the two halves need not
agree, and a refused activation writes LeaseActivated then LeaseActivationRejected, so filtering first would
answer "activated" with an action that was turned down.

Paging is keyset, not offset.

Before/BeforeId carry the last row of the previous page. The store is
append-only and read newest-first, so an offset would re-serve rows whenever an event was written between two
requests, and would get slower with depth. Id is the third key column of the covering index purely so that order
comes straight off the index: OccurredDate alone is not unique, since an action's Attempt and Outcome share a
timestamp, and without a tiebreaker a page boundary landing among events that share an instant cannot be resumed
exactly.

@codecov

codecov Bot commented Aug 19, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.87955% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 69.66%. Comparing base (4c664a9) to head (7db72ab).

Files with missing lines Patch % Lines
...ork/Pam/Repositories/AccessAuditEventRepository.cs 98.84% 0 Missing and 2 partials ⚠️
...ure.EntityFramework/Pam/Models/AccessAuditEvent.cs 96.66% 1 Missing ⚠️
src/Pam.Domain/Models/AccessAuditEvent.cs 96.66% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8230      +/-   ##
==========================================
+ Coverage   69.56%   69.66%   +0.10%     
==========================================
  Files        2471     2479       +8     
  Lines      105932   106289     +357     
  Branches     9601     9615      +14     
==========================================
+ Hits        73689    74045     +356     
  Misses      29779    29779              
- Partials     2464     2465       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Comment thread src/Sql/dbo/Pam/Stored Procedures/AccessAuditEvent_ReadManyByOrganizationId.sql Outdated
Comment thread src/Sql/dbo/Pam/Stored Procedures/AccessAuditEvent_Create.sql
Comment thread src/Infrastructure.EntityFramework/Pam/Models/AccessAuditEvent.cs Outdated
@patriksvensson
patriksvensson force-pushed the psvensson/pam-audit-log-scaffolding branch 2 times, most recently from 60f33d0 to a3135ff Compare August 24, 2026 15:41
@patriksvensson
patriksvensson marked this pull request as ready for review August 24, 2026 15:49
@patriksvensson
patriksvensson requested review from a team as code owners August 24, 2026 15:49
@github-actions

github-actions Bot commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

🤖 Bitwarden Claude Code Review

Overall Assessment: APPROVE

Re-reviewed the net-new AccessAuditEvent persistence layer after the paging rework: the SSDT table and two stored procedures, the consolidated MSSQL migration, the three generated EF provider migrations, the Dapper and EF repositories, the domain models/enums, and the eight DatabaseTheory integration tests. Dual-ORM parity holds — the same 29 columns are written and read on both paths, and the write-time name snapshot is equivalent between the LEFT JOIN in AccessAuditEvent_Create and the two EF lookups. The earlier OFFSET @Skip finding is addressed: paging is now keyset on (OccurredAt, Id), the index key order matches the ORDER BY on all four providers, the Dapper cursor parameters are pinned to DATETIME2(7) so the boundary round-trips at full precision, and two tests cover the partition and the append-between-pages case.

Code Review Details
  • ♻️ : OccurredAt breaks the *Date datetime column naming convention on a net-new table
    • src/Sql/dbo/Pam/Tables/AccessAuditEvent.sql:6

PR Metadata Assessment

  • QUESTION: The description no longer matches the code. It says AccessAuditEvent_Create snapshots cipher and collection display names via JSON_VALUE over the encrypted Data document; the procedure records those subjects by id only and holds no vault data. It also says five integration tests where there are eight, and does not mention the keyset paging that is now the store's central design point.

Comment thread src/Sql/dbo/Pam/Stored Procedures/AccessAuditEvent_ReadManyByOrganizationId.sql Outdated
@withinfocus

Copy link
Copy Markdown
Contributor

What's the plan here for non-relational storage of these events? We don't store events in the relational database ourselves and it's a self-host fallback. We cannot launch with the assumption that we can use this for our cloud-hosted deployments.

@patriksvensson
patriksvensson marked this pull request as draft August 24, 2026 15:58
@Hinton

Hinton commented Aug 24, 2026

Copy link
Copy Markdown
Member

@patriksvensson
patriksvensson force-pushed the psvensson/pam-audit-log-scaffolding branch from a3135ff to ca60268 Compare August 25, 2026 06:51
@abergs

abergs commented Aug 25, 2026

Copy link
Copy Markdown
Member

@withinfocus the plan is to start off storing PAM Audit logs in the relational db; and whenever a larger generic audit solution is in place we will start writing there instead.

Feel free to schedule time with me for a higher bandwidth discussion.

@rkac-bw

rkac-bw commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

@withinfocus the plan is to start off storing PAM Audit logs in the relational db; and whenever a larger generic audit solution is in place we will start writing there instead.

Feel free to schedule time with me for a higher bandwidth discussion.

@abergs @withinfocus Bitwarden already has an activity log (Azure table storage). Years ago they decided: for our cloud service, to not keep that in the main database — it's too much writing, too fast, and it would slow down the database that holds everyone's actual vaults. So in cloud, event logs get shipped off to separate cheap storage (Azure Table Storage). The SQL version only gets switched on for customers who host Bitwarden themselves.

@withinfocus withinfocus left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using this relational table for cloud is just not something we can do. It's not a lot of extra coding to set up the Table Storage writer as @rkac-bw described and that's where I also expected it to go.

@withinfocus

Copy link
Copy Markdown
Contributor

Also, as I take a step back I don't understand why this is not just using the existing event storage rails. We do not need a new table and must move forward with an approach that works for all events, albeit in your case with additional data points you wish to store.

@withinfocus
withinfocus dismissed their stale review August 25, 2026 17:16

After discussion with the team on their plans, we're accepting an interim state while they explore additional storage options. The team accepts the risks of this interim state.

@patriksvensson
patriksvensson force-pushed the psvensson/pam-audit-log-scaffolding branch 5 times, most recently from 7bd4bb8 to ac1f870 Compare August 31, 2026 08:40
@patriksvensson patriksvensson added the t:feature Change Type - Feature Development label Aug 31, 2026
@patriksvensson
patriksvensson marked this pull request as ready for review August 31, 2026 09:31
@patriksvensson
patriksvensson requested a review from abergs August 31, 2026 09:34
Comment thread src/Sql/dbo/Pam/Tables/AccessAuditEvent.sql Outdated
Comment thread util/Migrator/DbScripts/2026-08-31_00_AddAccessAuditEvent.sql
@patriksvensson
patriksvensson force-pushed the psvensson/pam-audit-log-scaffolding branch 5 times, most recently from b60d271 to eb84453 Compare September 7, 2026 11:57
Append-only store for PAM access and rotation audit events, with Dapper and
EF repositories, the SSDT table and stored procedures, and migrations for all
four databases. Display names are snapshotted at write time so an event
survives a later delete or rename of what it references.
Bring the new files in line with the repository's standard formatting so the
automated style check passes.
The DbScripts migration was dated 2026-08-18, which is before
2026-08-25_00_AddPamCollectionReads.sql on main, so the "Validate new
migration naming and order" gate rejected it. Re-date to 2026-08-31 so
DbUp applies it after everything already on main.
.claude/rules/database-dapper.md requires datetime columns to end in
Date.
The audit store had a single read that returned an organization's whole trail
unfiltered. It is replaced by AccessAuditEvent_ReadPageByOrganizationId, which
takes a date range, filters on kind, actor, requester, cipher and rule, and
returns one page at a time; each action's attempt/outcome pair is now collapsed
in the store rather than by the caller, so a pair split across a page boundary
still reads as one entry. A second procedure,
AccessAuditEvent_ReadItemsByOrganizationId, lists the ciphers and rules the
trail names in a range, which is what the Item filter's menu is built from. The
trail index gains included columns covering both reads, and a new index on
CorrelationId serves the collapse. Ported from c362bb8 on pam/uat and folded
into this branch's existing migration, since the store has not shipped.
@patriksvensson
patriksvensson force-pushed the psvensson/pam-audit-log-scaffolding branch from eb84453 to df9a0b1 Compare September 8, 2026 07:32
@patriksvensson

Copy link
Copy Markdown
Member Author

@rkac-bw Resolved the last discussion point. Would appreciate a new review if and when you have time. Thanks!

[SyncState] TINYINT NULL,
CONSTRAINT [PK_AccessAuditEvent] PRIMARY KEY CLUSTERED ([Id] ASC),
CONSTRAINT [FK_AccessAuditEvent_Organization] FOREIGN KEY ([OrganizationId])
REFERENCES [dbo].[Organization] ([Id]) ON DELETE CASCADE

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just an FYI: Our last DB migration with an ON DELETE CASCADE to Organization caused a brief period of blocking (~30 seconds). That doesn't necessarily mean this will cause it again, but we should be aware that it could occur.

@mkincaid-bw mkincaid-bw left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some minor formatting nitpicks.

Comment on lines +98 to +100
FROM (SELECT 1 AS [X]) Seed
LEFT JOIN [dbo].[User] AU ON AU.[Id] = @ActorId
LEFT JOIN [dbo].[User] RU ON RU.[Id] = @RequesterId

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⛏️ Minor formatting nitpick but most SQL keywords should be on their own line

Suggested change
FROM (SELECT 1 AS [X]) Seed
LEFT JOIN [dbo].[User] AU ON AU.[Id] = @ActorId
LEFT JOIN [dbo].[User] RU ON RU.[Id] = @RequesterId
FROM
(SELECT 1 AS [X]) Seed
LEFT JOIN
[dbo].[User] AU ON AU.[Id] = @ActorId
LEFT JOIN
[dbo].[User] RU ON RU.[Id] = @RequesterId

See https://contributing.bitwarden.com/contributing/code-style/sql/#select-statements

Comment on lines +26 to +30
FROM [dbo].[AccessAuditEvent]
WHERE [OrganizationId] = @OrganizationId
AND [OccurredDate] >= @StartDate
AND [OccurredDate] <= @EndDate
AND [CipherId] IS NOT NULL

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⛏️ Same formatting nitpick (and throughout the rest of the SQL code).

Suggested change
FROM [dbo].[AccessAuditEvent]
WHERE [OrganizationId] = @OrganizationId
AND [OccurredDate] >= @StartDate
AND [OccurredDate] <= @EndDate
AND [CipherId] IS NOT NULL
FROM
[dbo].[AccessAuditEvent]
WHERE
[OrganizationId] = @OrganizationId
AND [OccurredDate] >= @StartDate
AND [OccurredDate] <= @EndDate
AND [CipherId] IS NOT NULL

-- LeaseActivationRejected, so filtering before the collapse would answer "activated" with an action that was
-- turned down.
AND (
@Kinds IS NULL

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realize these procs are temporary while the team explores other storage options, but I wanted to point out that this type of catch-all dynamic search pattern is a known SQL Server anti-pattern. We won't see performance issues until the table grows but depending on how long it takes for the new storage options, this stored proc could eventually have real performance issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

t:feature Change Type - Feature Development

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants