Skip to content

feat: simplify attack logging with nested attempt_history - #131

Merged
kyuz0 merged 3 commits into
ReversecLabs:developfrom
kyuz0:feat/attack-attempt-history
Sep 10, 2026
Merged

kyuz0 merged 3 commits into
ReversecLabs:developfrom
kyuz0:feat/attack-attempt-history

Conversation

@kyuz0

@kyuz0 kyuz0 commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator

Dynamic attacks now keep one <id>-attack result with their representative input/response, overall verdict, and total attempt count. Supporting single-turn attacks attach intermediate candidates as nested attempt_history, simplifying logging and keeping result statistics tied to the attack's reported count.

  • Collect history explicitly inside supporting attack modules, including Best-of-N, both LLM jailbreakers, and the workspace sample attack. Keep the existing four-element return tuple and legacy module signatures.
  • Enable collection by default, with SPIKEE_ATTACK_HISTORY=false available to reduce result size. Remove the CLI flag, viewer checkbox, extra attack parameter, and alternative list return.
  • Restore Crescendo, GOAT, and Echo Chamber's original conversation logging; remove duplicate candidate logging from multi_turn.
  • Combine supplied candidate histories across repeated invocations and display them within the result detail. Preserve target responses when judging fails, and keep valid outcomes/counts when optional history is malformed.
  • Continue reading legacy expanded result files. Update testing, custom-attack, and results documentation and the workspace environment example.

Validation on the rebased commit:

  • All 479 functional tests passed on Python 3.11 with isolated package installation, covering legacy modules/results, counts, resume, extraction, rejudge, error paths, and history enabled/disabled.
  • Ruff 0.16.6 lint and formatting checks passed for spikee/ and the three changed/new test files.
  • Viewer checks passed on desktop and mobile.

@kyuz0 kyuz0 changed the title Simplify attack logging with nested attempt_history feat: simplify attack logging with nested attempt_history Sep 10, 2026
@kyuz0
kyuz0 force-pushed the feat/attack-attempt-history branch from 9216964 to 1330fab Compare September 10, 2026 15:37
@kyuz0
kyuz0 merged commit c14ff2a into ReversecLabs:develop Sep 10, 2026
1 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant