Skip to content

Add non-transactional bulk operations - #245

Draft
Mike Krüger (mkrueger) wants to merge 1 commit into
mainfrom
dev/mkrueger/bulk-operations
Draft

Mike Krüger (mkrueger) wants to merge 1 commit into
mainfrom
dev/mkrueger/bulk-operations

Conversation

@mkrueger

Copy link
Copy Markdown
Collaborator

Summary

Work-in-progress snapshot of non-transactional bulk operations. This draft preserves the current implementation for further work and is not ready to merge.

  • Add independent create, upsert, replace, delete, and patch operations across partition keys with bounded concurrency.
  • Add predicate-based patch/delete selection, saved plans, dry runs, observed-RU/item limits, and ETag safeguards.
  • Add journals for resumable execution and explicit handling of uncertain writes.
  • Add interactive bulk state and prompt indicators, plus stateless MCP subcommands with confirmation handling.
  • Include command coverage, localized help, and documentation.

Related to #107.

Add bounded concurrent writes, selection plans, journaling, interactive bulk state, and MCP integration. Related to #107.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Comment on lines +770 to +773
var response = new ResponseMessage(HttpStatusCode.OK)
{
Content = new MemoryStream(Encoding.UTF8.GetBytes(JsonSerializer.Serialize(new { _rid = this.Rid, partitionKey = new { paths = SinglePath } }))),
};
if (ann != null && ann.Restricted)
{
helpJson["isRestricted"] = "This tool can't be run via MCP. User only.";
helpJson["isRestricted"] = cmd?.CommandName == "bulk"
Comment on lines +118 to +121
catch (Exception ex)
{
failure ??= ExceptionDispatchInfo.Capture(ex);
}
Comment on lines +131 to +134
catch (Exception ex)
{
failure ??= ExceptionDispatchInfo.Capture(ex);
}
Comment on lines +37 to +42
catch (Exception ex)
{
// The request may or may not have reached the service.
failure ??= ExceptionDispatchInfo.Capture(ex);
result = BulkOutcome.Create(operation, BulkOutcome.Uncertain, error: ex.Message);
}
Comment on lines +281 to +288
foreach (var segment in path.Split('/', StringSplitOptions.RemoveEmptyEntries))
{
if (current.ValueKind != JsonValueKind.Object || !current.TryGetProperty(segment, out current))
{
found = false;
break;
}
}
options.UnixCreateMode = UnixFileMode.UserRead | UnixFileMode.UserWrite;
}

return new BulkSpool(new FileStream(Path.Combine(Path.GetTempPath(), $"cosmos-bulk-{Guid.NewGuid():N}.jsonl"), options));
{
private readonly DirectoryInfo directory = System.IO.Directory.CreateTempSubdirectory("cosmos-bulk-test-");

public string PathFor(string name) => Path.Combine(this.directory.FullName, name);
@github-code-quality

Copy link
Copy Markdown

Code Coverage Overview

Languages: C#

C# / code-coverage/dotnet

The overall line coverage in commit f0a9b79 in the dev/mkrueger/bulk-op... branch is 67%. The line coverage in commit 3a9dc25 in the main branch is 66%.

Show a line coverage summary of the most impacted files.
File main 3a9dc25 dev/mkrueger/bulk-op... f0a9b79 +/-
D:\a\CosmosDBSh...ationFactory.cs 37% 53% +16%
D:\a\CosmosDBSh...\BulkCommand.cs 0% 82% +82%
D:\a\CosmosDBSh...onNormalizer.cs 0% 92% +92%
D:\a\CosmosDBSh...\BulkJournal.cs 0% 93% +93%
D:\a\CosmosDBSh...BulkExecutor.cs 0% 95% +95%
D:\a\CosmosDBSh...ds\BulkSpool.cs 0% 98% +98%
D:\a\CosmosDBSh...ingBulkState.cs 0% 100% +100%
D:\a\CosmosDBSh...\BulkSummary.cs 0% 100% +100%
D:\a\CosmosDBSh...\BulkOutcome.cs 0% 100% +100%
D:\a\CosmosDBSh...ulkOperation.cs 0% 100% +100%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant