feat(airt): multimodal prompt_matrix (text×media cross-product) + agent transparency (1.6.10)#82
Merged
Conversation
…nt transparency (1.6.10) Multimodal attacks could only pair prompts to media 1:1 (prompts / prompts_csv by index/basename), so "4 text prompts across 5 images" produced 5 trials, not the 20 a user expects. Adds a cross-product mode — a common pattern. - generate_multimodal_attack gains prompt_matrix=[...]: every prompt runs against every media item → N×M trials (4 prompts × 5 images = 20). Configured TRANSFORMS apply to each combination. The generated workflow builds the media-set list (matrix or 1:1) and logs "Running 20 multimodal set(s) (matrix: 4 prompts x 5 media)". Validated live: 20 trials from 4×5. - Agent guidance: always call execute_workflow in the same turn as generate_* (never stop after "workflow generated" → 0 trials), and NARRATE each step to the user (what's running, how many trials, and any target error) — no black box. Loop variables renamed to descriptive names (no bare `i`). Bumps capability to 1.6.10.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Multimodal attacks could only pair prompts to media 1:1 (
prompts/prompts_csvby index/basename). So "I have 4 text prompts and 5 images, run all combinations" produced 5 trials, not the 20 a user expects — a common pattern that wasn't supported. This adds a cross-product mode.generate_multimodal_attackgainsprompt_matrix=[...]: every prompt runs against every media item → N×M trials (4 prompts × 5 images = 20). Configuredtransformsapply to each combination. Distinct fromprompts/prompts_csv(which remain 1:1).Running 20 multimodal set(s) (matrix: 4 prompts x 5 media)then[1/20] … [20/20].execute_workflowin the same turn asgenerate_*— never stop after "workflow generated" (that leaves 0 trials and looks like a silent failure), and (2) instructs the agent to narrate every step (what's running, how many trials, any target error) — no black box.i).Bumps capability to 1.6.10.
Validation
generate_only: generated workflow compiles, emitsPROMPT_MATRIX = [...], uses the cross-product loop (for matrix_prompt in PROMPT_MATRIX), appliestransforms=TRANSFORMSper set, and uses no barei.Running 20 multimodal set(s) (matrix: 4 prompts x 5 media),[1/20]…[20/20],Assessment complete(assessmentefe16775). 20 trials as expected.py_compileclean onattack_runner.py+tools/attacks.py.Notes