Is your feature request related to a problem? Please describe.
PyRIT's built-in seed datasets cover multilingual safety, jailbreaks, and over-refusal, but do not currently provide a Turkish prompt-injection dataset with paired benign boundary cases. This makes it difficult to test whether a defense distinguishes adversarial intent from legitimate Turkish requests that share the same topics and security-sensitive vocabulary.
Describe the solution you'd like
Add a remote loader for the public, CC BY 4.0 licensed Turkish Conversation Prompt-Injection Dataset.
The dataset contains 750 curated Turkish examples:
- 600 legitimate user requests
- 150 prompt-injection attacks
- 150 benign boundary cases paired with related attacks
- train, validation, and test splits
- 10 attack families, including direct instruction override, system-prompt extraction, role-play jailbreaks, tool/agent abuse, indirect injection, RAG poisoning, memory poisoning, and obfuscation/code-switching
The proposed loader would:
- return literal
SeedPrompt objects
- default to attack examples for red-team workflows
- provide typed filters for attack/benign/all labels, dataset split, and attack family
- preserve row-level provenance in seed metadata
- expose attack families as harm categories for attack rows
- support Hugging Face tokens consistently with existing remote loaders
- include unit tests, built-in-dataset documentation, and a real-dataset smoke test
Describe alternatives you've considered, if relevant
Users can call datasets.load_dataset() directly and manually translate records into PyRIT seeds. That loses PyRIT dataset discovery, consistent metadata, typed filtering, and reproducible loading behavior.
A second option is to expose only attack rows. Keeping the paired benign boundary cases available is more useful because it supports both adversarial testing and over-refusal / false-positive analysis.
Additional context
Dataset source: https://huggingface.co/datasets/3nesdeniz/turkish-conversation-prompt-injection
Source repository: https://github.com/3nesdeniz/turkish-conversation-prompt-injection
The implementation will not publish or claim model benchmark results; it only integrates the documented dataset structure and provenance.
Is your feature request related to a problem? Please describe.
PyRIT's built-in seed datasets cover multilingual safety, jailbreaks, and over-refusal, but do not currently provide a Turkish prompt-injection dataset with paired benign boundary cases. This makes it difficult to test whether a defense distinguishes adversarial intent from legitimate Turkish requests that share the same topics and security-sensitive vocabulary.
Describe the solution you'd like
Add a remote loader for the public, CC BY 4.0 licensed Turkish Conversation Prompt-Injection Dataset.
The dataset contains 750 curated Turkish examples:
The proposed loader would:
SeedPromptobjectsDescribe alternatives you've considered, if relevant
Users can call
datasets.load_dataset()directly and manually translate records into PyRIT seeds. That loses PyRIT dataset discovery, consistent metadata, typed filtering, and reproducible loading behavior.A second option is to expose only attack rows. Keeping the paired benign boundary cases available is more useful because it supports both adversarial testing and over-refusal / false-positive analysis.
Additional context
Dataset source: https://huggingface.co/datasets/3nesdeniz/turkish-conversation-prompt-injection
Source repository: https://github.com/3nesdeniz/turkish-conversation-prompt-injection
The implementation will not publish or claim model benchmark results; it only integrates the documented dataset structure and provenance.