feat: expand SupportedContentType enum with common media types#2233
feat: expand SupportedContentType enum with common media types#2233lxcxjxhx wants to merge 4 commits into
Conversation
Add support for image, audio, video, and document content types to address TODO comment in storage.py. This enables PyRIT to handle multimodal data (images, audio, video, PDFs) when uploading to Azure Blob Storage. Added types: - Text: HTML, JSON, XML, CSV, Markdown - Image: PNG, JPEG, GIF, WebP, SVG, BMP - Audio: WAV, MP3, OGG, FLAC, M4A - Video: MP4, WebM, OGG, AVI - Document: PDF, ZIP, TAR, GZIP Includes unit tests to verify all enum values.
- Modify AzureBlobStorageIO.write_file_async to accept content_type parameter - Infer content_type from file extension using mimetypes.guess_type - Pass content_type through serializers (save_data_async, save_b64_image_async, save_formatted_audio_async) - Consolidate SupportedContentType enum imports - Add unit tests for content_type inference and propagation Addresses maintainer feedback on PR microsoft#2233
|
感谢 @romanlutz 的详细反馈!我已经按照您的建议进行了修改:
这些修改应该能够正确支持广告中声明的媒体类型,通过将 MIME 选择逻辑集成到文件上传工作流中。如果需要进一步修改,请告诉我! |
|
@microsoft-github-policy-service agree |
|
For context, I'm adding a translation here (I can't read Chinese):
|
The Jupyter notebook still imported SupportedContentType from pyrit.prompt_target.azure_blob_storage_target, which no longer defines it (consolidated into pyrit.memory.storage.storage). Co-Authored-By: Claude <noreply@anthropic.com>
|
I have addressed all the review feedback:
@romanlutz could you please re-review when you have a chance? |
| @abstractmethod | ||
| async def write_file_async(self, path: Path | str, data: bytes) -> None: | ||
| async def write_file_async( | ||
| self, path: Path | str, data: bytes, content_type: str | None = None |
There was a problem hiding this comment.
Serializers now always pass content_type=, so third-party StorageIO implementations using the previous (path, data) contract remain concrete but fail at runtime with an unexpected keyword argument. Please preserve the generic interface/call sites, or add a backward-compatible metadata-aware API with a default implementation.
| from azure.storage.blob.aio import ContainerClient as AsyncContainerClient | ||
|
|
||
| from pyrit.common import default_values | ||
| from pyrit.memory.storage.storage import SupportedContentType |
There was a problem hiding this comment.
Consolidating these enums makes every new value valid for AzureBlobStorageTarget, but this target still accepts text/url input and always uploads str.encode(request.converted_value). Please keep this parameter restricted to textual types, or add actual binary message handling before exposing PNG, PDF, audio, and video choices.
| if content_type is None: | ||
| from mimetypes import guess_type | ||
|
|
||
| inferred_type, _ = guess_type(blob_name) |
There was a problem hiding this comment.
mimetypes.guess_type is platform-dependent and does not reliably produce the enum values added here. For example, .gz still falls back to text/plain, while Windows maps .csv, .flac, .avi, and .zip to different legacy/vendor types. Please use a deterministic extension-to-SupportedContentType mapping and test all advertised extensions.
| "self._memory.results_storage_io is not initialized" | ||
| ) | ||
| # Infer content type from file extension | ||
| content_type = self.get_mime_type(str(file_path)) |
There was a problem hiding this comment.
save_formatted_audio_async always produces RIFF/WAVE bytes, but AudioPathDataTypeSerializer defaults to an .mp3 filename, so normal callers now upload WAV data as audio/mpeg. Please force a .wav path and audio/wav here, or encode the bytes according to the requested extension.
|
I've addressed all the review feedback:
@romanlutz could you please re-review when you have a chance? |
|
There hasn't been an update since my last comments. @lxcxjxhx |
Motivation
The
SupportedContentTypeenum inpyrit/memory/storage/storage.pyhad a TODO comment indicating that more media types should be added. Currently onlyPLAIN_TEXTwas supported, which limits PyRIT's ability to handle multimodal data (images, audio, video, PDFs) when uploading to Azure Blob Storage.This is particularly relevant since the existing doc example
doc/code/executor/5_workflow.pyalready referencesSupportedContentType.HTMLwhich doesn't exist in the enum yet.Changes
Enum expansion
Expanded
SupportedContentTypeenum with commonly used media types:Enum consolidation
Removed the duplicate
SupportedContentTypeenum fromazure_blob_storage_target.pyand unified all imports to use the canonical definition inpyrit.memory.storage.storage.MIME type integration
content_typeparameter toStorageIO.write_file_asyncand its implementationsmimetypes.guess_typewhen not explicitly providedsave_data_async,save_b64_image_async,save_formatted_audio_asyncserializers to pass inferred content_typeblob_content_typeTesting
test_supported_content_type_has_expected_valuesfor all enum valuescontent_typeparameter