FEAT: Add remove_seeds_from_memory_async to memory by blahdeblahde · Pull Request #2243 · microsoft/PyRIT

blahdeblahde · 2026-07-21T22:22:57Z

Description

PyRIT memory can add seed prompt datasets (via add_seed_datasets_to_memory_async / add_seeds_to_memory_async) and query them with get_seeds, but there is no counterpart for removing seeds. Today users have to drop into raw SQL, which is error-prone and differs across the supported backends (DuckDB, SQLite, Azure SQL).

This PR adds remove_seeds_from_memory_async, which accepts the same filter parameters as get_seeds (dataset_name, dataset_name_pattern, added_by, harm_categories, authors, groups, source, value_sha256, data_types, seed_type, parameters, metadata, prompt_group_ids) and returns the number of seeds removed.

Recommended workflow — preview, then delete with the same filters:

# See what matches before deleting
to_delete = memory.get_seeds(dataset_name="illegal")

# Remove them; returns the count deleted
removed = await memory.remove_seeds_from_memory_async(dataset_name="illegal")

Implementation

Extracted the filter-building logic out of get_seeds into a shared private helper (_build_seed_filter_conditions) so both query and removal build conditions from a single source and cannot drift.
remove_seeds_from_memory_async deletes matching rows in a single transaction using the SQLAlchemy ORM (rollback on error), so it behaves consistently across DuckDB, SQLite, and Azure SQL.

Safety

At least one filter must be provided; a no-filter call raises ValueError to prevent accidentally wiping the entire seed database.

Tests and Documentation

Tests — tests/unit/memory/memory_interface/test_interface_remove_seeds.py covers:

exact dataset_name match
dataset_name_pattern (SQL LIKE)
multi-filter narrowing (dataset_name + added_by)
no-filter call raises ValueError
zero-match returns 0 and deletes nothing
harm_categories (list field) and source filters

Documentation — added a "Removing Seeds from the Database" section to doc/code/memory/8_seed_database.py (and the synced .ipynb) demonstrating the preview-then-remove workflow.

Copilot

Pull request overview

This PR adds first-class support for deleting seed prompts from PyRIT memory by introducing remove_seeds_from_memory_async, mirroring the existing get_seeds filter surface so users no longer need backend-specific SQL for seed cleanup.

Changes:

Extracts seed filter construction into a shared private helper (_build_seed_filter_conditions) used by both seed querying and deletion.
Adds remove_seeds_from_memory_async to delete matching SeedEntry rows transactionally and return the number removed (with a no-filter safety guard).
Adds unit tests and documentation demonstrating a “preview via get_seeds, then delete via remove_seeds_from_memory_async” workflow.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 13 comments.

File	Description
`pyrit/memory/memory_interface.py`	Adds shared seed-filter builder and new async seed removal API implemented via SQLAlchemy.
`tests/unit/memory/memory_interface/test_interface_remove_seeds.py`	Adds unit coverage for common filter cases and safety behavior.
`doc/code/memory/8_seed_database.py`	Documents seed removal usage and recommended preview-then-delete workflow.
`doc/code/memory/8_seed_database.ipynb`	Syncs the same documentation updates into the notebook format.

+@pytest.mark.asyncio
+async def test_remove_seeds_by_dataset_name(sqlite_instance: MemoryInterface):


+@pytest.mark.asyncio
+async def test_remove_seeds_by_dataset_name_pattern(sqlite_instance: MemoryInterface):


+@pytest.mark.asyncio
+async def test_remove_seeds_by_added_by(sqlite_instance: MemoryInterface):


+@pytest.mark.asyncio
+async def test_remove_seeds_multi_filter_narrowing(sqlite_instance: MemoryInterface):


+@pytest.mark.asyncio
+async def test_remove_seeds_no_filter_raises_value_error(sqlite_instance: MemoryInterface):


+        remove_seeds_from_memory_async stay in sync and cannot drift.

        Args:
            value (str): The value to match by substring. If None, all values are returned.


+
+        Args:
+            value (str): The value to match by substring. If None, all values are returned.
+            value_sha256 (str): The SHA256 hash of the value to match. If None, all values are returned.


+            prompt_group_ids (Sequence[uuid.UUID]): A list of prompt group IDs to filter by.
+
+        Returns:
+            Sequence[SeedPrompt]: A list of prompts matching the criteria.


+
+        Args:
+            value (str): The value to match by substring. If None, all values are considered.
+            value_sha256 (str): The SHA256 hash of the value to match. If None, all values are considered.


+                query = session.query(SeedEntry).filter(and_(*conditions))
+                count = query.delete(synchronize_session="fetch")
+                session.commit()


Adds an API to remove seed prompts from memory using the same filter parameters as get_seeds (dataset_name, dataset_name_pattern, added_by, harm_categories, authors, groups, source, value_sha256, etc.). Implementation: - Extract filter-building logic from get_seeds into shared _build_seed_filter_conditions helper to prevent drift - Add remove_seeds_from_memory_async that uses the shared helper - Require at least one filter (raises ValueError if none provided) - Single transaction with rollback on failure - Works across DuckDB, SQLite, Azure SQL (uses SQLAlchemy ORM) Unit tests cover: exact name filter, LIKE pattern, multi-filter narrowing, no-filter ValueError, zero-match return, harm_categories, and source filter. Docs: add a 'Removing Seeds from the Database' section to the seed database notebook (8_seed_database.py/.ipynb) demonstrating the preview-then-remove workflow. Resolves AB#5688 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: f9e3cc10-5c77-4384-92d9-d27dad9baea2

blahdeblahde · 2026-07-21T22:38:57Z

Thanks for the review! Addressed all points in the latest push:

Removed the @pytest.mark.asyncio decorators from the new tests (asyncio_mode = auto is set globally).
Fixed the value_sha256 docstring type to Sequence[str] | None in the shared helper, get_seeds, and remove_seeds_from_memory_async.
Corrected the get_seeds return docstring to Sequence[Seed] (not just SeedPrompt).
Changed the delete to synchronize_session=False to avoid the extra SELECT on a short-lived session.

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

+        # Delete matching rows in a single transaction so it is atomic across backends.
+        with closing(self.get_session()) as session:
+            try:
+                query = session.query(SeedEntry).filter(and_(*conditions))
+                count = query.delete(synchronize_session=False)
+                session.commit()
+                return count
+            except SQLAlchemyError as e:
+                session.rollback()
+                logger.exception(f"Failed to remove seeds from memory: {e}")
+                raise


blahdeblahde · 2026-07-21T22:47:17Z

@microsoft-github-policy-service agree company="Microsoft"

Copilot AI review requested due to automatic review settings July 21, 2026 22:22

Copilot started reviewing on behalf of blahdeblahde July 21, 2026 22:23 View session

Copilot AI reviewed Jul 21, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings July 21, 2026 22:38

blahdeblahde force-pushed the blahdeblahde-remove-seeds-from-memory-api branch from 2d45b57 to 0e3ae02 Compare July 21, 2026 22:38

Copilot started reviewing on behalf of blahdeblahde July 21, 2026 22:39 View session

Copilot AI reviewed Jul 21, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

FEAT: Add remove_seeds_from_memory_async to memory#2243

FEAT: Add remove_seeds_from_memory_async to memory#2243
blahdeblahde wants to merge 1 commit into
microsoft:mainfrom
blahdeblahde:blahdeblahde-remove-seeds-from-memory-api

blahdeblahde commented Jul 21, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

blahdeblahde commented Jul 21, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

blahdeblahde commented Jul 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		@pytest.mark.asyncio
		async def test_remove_seeds_by_dataset_name(sqlite_instance: MemoryInterface):

		@pytest.mark.asyncio
		async def test_remove_seeds_by_added_by(sqlite_instance: MemoryInterface):

		@pytest.mark.asyncio
		async def test_remove_seeds_multi_filter_narrowing(sqlite_instance: MemoryInterface):

		@pytest.mark.asyncio
		async def test_remove_seeds_no_filter_raises_value_error(sqlite_instance: MemoryInterface):

Uh oh!

Conversation

blahdeblahde commented Jul 21, 2026

Description

Implementation

Safety

Tests and Documentation

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

blahdeblahde commented Jul 21, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

blahdeblahde commented Jul 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants