Skip to content

Merge shell history across concurrent processes - #228

Merged
Mike Krüger (mkrueger) merged 13 commits into
mainfrom
dev/mkrueger/fix-history-multi-process
Oct 2, 2026
Merged

Mike Krüger (mkrueger) merged 13 commits into
mainfrom
dev/mkrueger/fix-history-multi-process

Conversation

@mkrueger

@mkrueger Mike Krüger (mkrueger) commented Sep 29, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

Shell history was lost when more than one shell process used the same config directory, for example two terminals, or a -c script while an interactive shell or MCP server is running. Each process loaded cmd_history once at startup. SaveHistory then rewrote the whole file from that process's in-memory list (FileMode.Create), guarded only by an in-process lock. A save from one process therefore discarded every entry another process had saved since the first one started.

A related issue: --clear-history deleted the file but left the already-loaded entries in memory, so the next save wrote the old history back.

Changes

  • Each interpreter tracks the newest 60 pending entries recorded since its last successful save.
  • Saves and clears coordinate through a separate cmd_history.lock file (FileShare.None, short bounded retry). Saving reads the current entries, applies pending entries with the existing remove-and-append de-duplication, keeps the newest MAXHISTORYITEMS, and writes the complete encoded result to a private temporary file. The temporary file is flushed before atomically replacing cmd_history.
  • The in-memory history of each process stays per-process; only persistence merges.
  • --clear-history atomically replaces the history file under the same lock and clears in-memory and pending entries only after replacement succeeds. Failed clears report an error.
  • Owner-only permissions on Linux/macOS, encoded multi-line entries, and best-effort ordinary history saving are preserved. Failed writes leave the saved history intact.
  • README.md and docs/navigation.md describe concurrent merging and atomic history persistence.

Tests

SerializedExecutionTests cover synchronized independent processes, partial-write failures, shared-lock failures, bounded pending entries, and two interpreters on the same config directory: merging entries from both, de-duplication across processes, trimming to the newest entries, clear-history not resurrecting saved entries, and multi-line entries surviving a merge.

Merge pending per-process history entries with the on-disk file under an exclusive file lock so concurrent shell instances do not overwrite each other. Keep clear-history best-effort and add focused coverage for merge, de-duplication, trimming, clear, and multi-line entries.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The core cross-process locking behavior lacks a process-level concurrency regression test.

Review effort: Balanced
Findings: 1 Medium severity

Open (1)
What changed in this PR

Adds concurrent-safe shell history persistence across processes.

Changes:

  • Merges pending history entries under an exclusive file lock.
  • Clears persisted and in-memory history together.
  • Adds merge, deduplication, trimming, and multiline tests and documentation.
File Description
docs/​navigation.md Documents concurrent history merging.
CosmosDBShell/​Program.cs Routes history clearing through the interpreter.
CosmosDBShell/​Azure.Data.Cosmos.Shell.Core/​ShellInterpreter.cs Implements locked history merging and clearing.
CosmosDBShell.Tests/​Runtime/​SerializedExecutionTests.cs Tests history persistence behavior.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread CosmosDBShell.Tests/Runtime/SerializedExecutionTests.cs
@github-code-quality

github-code-quality Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

Code Coverage Overview

Languages: C#

C# / code-coverage/dotnet

The overall line coverage in commit ab20889 in the dev/mkrueger/fix-his... branch remains at 66%, unchanged from commit 6992beb in the main branch.

Show a line coverage summary of the most impacted files.
File main 6992beb dev/mkrueger/fix-his... ab20889 +/-
D:\a\CosmosDBSh...lInterpreter.cs 72% 73% +1%
D:\a\CosmosDBSh...hell\Program.cs 79% 80% +1%

Updated October 02, 2026 08:50 UTC

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings September 30, 2026 08:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Startup can race an exclusively locked history file, and failed clear operations incorrectly report success.

Review effort: Balanced
Findings: 1 High severity

Open (1)
Resolved since last review (1)
Previously missed (1)

In code that hasn't changed since last review

Medium severity Clear-history reports success even when file truncation fails

CosmosDBShell/​Azure.Data.Cosmos.Shell.Core/​ShellInterpreter.cs:2345

This suppresses a failed clear after already clearing only the in-memory list, so Program.cs unconditionally prints “History deleted.” and exits successfully even when the persisted file remains (for example, on permission failure or lock timeout). Unlike ordinary history saving, --clear-history has no useful outcome if truncation fails; propagate/return the failure so the caller reports it instead of claiming success.

Comment thread CosmosDBShell/Azure.Data.Cosmos.Shell.Core/ShellInterpreter.cs Outdated
@mkrueger

Copy link
Copy Markdown
Collaborator Author

Copilot review

Co-authored-by: mkrueger <341098+mkrueger@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings September 30, 2026 08:57

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The locking, merge behavior, clearing semantics, and relevant edge cases are implemented consistently and adequately tested.

Review effort: Balanced
Findings: None

Resolved since last review (1)

Co-authored-by: mkrueger <341098+mkrueger@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings September 30, 2026 09:03

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Clear-history can truncate the file before a permission failure, allowing retained pending entries to restore cleared history.

Review effort: Balanced
Findings: 1 Medium severity

Open (1)

Comment thread CosmosDBShell/Azure.Data.Cosmos.Shell.Core/ShellInterpreter.cs Outdated
Co-authored-by: mkrueger <341098+mkrueger@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings September 30, 2026 09:06

Copilot AI commented Sep 30, 2026

Copy link
Copy Markdown
Contributor

Copilot review

Reviewed and addressed the remaining history issues in 381001f, cbc415c, and af4ba85: startup now retries the exclusive history lock, and --clear-history reports persistence failures without clearing in-memory state prematurely. Focused tests and builds pass.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The shared open helper prevents loading readable but unwritable history files and cannot repair their Unix permissions.

Review effort: Balanced
Findings: None

Resolved since last review (1)
Previously missed (1)

In code that hasn't changed since last review

Medium severity Constructor requires write access and loses readable history

CosmosDBShell/​Azure.Data.Cosmos.Shell.Core/​ShellInterpreter.cs:2398

This helper now requires write access even when the constructor only needs to read history. A history file that remains readable but is temporarily unwritable (for example, an owner-read-only file or read-only config mount) previously loaded via File.ReadAllLines; now the open fails, the constructor swallows the exception, and the session starts with empty history. On Unix, save also cannot repair missing write bits because RestrictHistoryFileToOwner runs only after this read/write open succeeds. Please use a read-only, exclusive open for constructor loading and reserve the read/write OpenOrCreate path for save/clear, applying any existing-file mode repair before requesting write access.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings September 30, 2026 09:17
@mkrueger

Copy link
Copy Markdown
Collaborator Author

Copilot review

Co-authored-by: mkrueger <341098+mkrueger@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings September 30, 2026 09:36

Copilot AI commented Sep 30, 2026

Copy link
Copy Markdown
Contributor

Copilot review

Review found that startup opened history read/write, so a readable but non-writable history file could not be loaded. Startup now uses the exclusive lock in read-only mode, with Unix permission coverage. I also synchronized the process regression so both processes load before writes. Fixed in 2a9d454.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The concurrency and clear-history behavior are consistently implemented, documented, localized, and covered by focused tests.

Review effort: Balanced
Findings: None

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…rocess' into dev/mkrueger/fix-history-multi-process
Copilot AI balanced review requested due to automatic review settings September 30, 2026 10:10

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The locking and pending-entry design addresses the data-loss scenarios with comprehensive focused coverage.

Review effort: Balanced
Findings: None

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings September 30, 2026 14:22

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

In-place truncation can permanently lose history when a transient write failure produces a partial file.

Review effort: Balanced
Findings: 1 High severity

Open (1)

Comment thread CosmosDBShell/Azure.Data.Cosmos.Shell.Core/ShellInterpreter.cs Outdated
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 2, 2026 08:17

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Atomic replacement can weaken protected Windows history-file ACLs and expose sensitive history.

Review effort: Balanced
Findings: 1 High severity

Open (1)
Resolved since last review (1)

Comment thread CosmosDBShell/Azure.Data.Cosmos.Shell.Core/ShellInterpreter.cs Outdated
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 2, 2026 08:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The implementation addresses the concurrency and recovery scenarios with comprehensive regression coverage.

Review effort: Balanced
Findings: None

Resolved since last review (1)

@mkrueger
Mike Krüger (mkrueger) merged commit 074b19b into main Oct 2, 2026
12 checks passed
@mkrueger
Mike Krüger (mkrueger) deleted the dev/mkrueger/fix-history-multi-process branch October 2, 2026 10:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants