Trustabl Agent scanner - #110
Conversation
WalkthroughThe pull request adds a ChangesTrustabl workflow
Estimated code review effort: 2 (Simple) | ~10 minutes Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment Warning |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/trustabl.yml:
- Line 16: Update the actions/checkout@v4 step in the Trustabl workflow to set
persist-credentials to false, preventing the checkout token from being stored in
local Git configuration while preserving the existing github.token usage by the
Trustabl action.
- Around line 7-10: Add inline comments to the security-events and pull-requests
entries in the workflow permissions block, documenting that Trustabl requires
them for SARIF uploads and sticky PR comments. Preserve both write permissions
and the existing contents permission unchanged.
- Around line 2-5: Add workflow-level concurrency settings near the top-level
triggers in the Trustabl workflow, using a group keyed by the workflow identity
and pull request number, with cancel-in-progress enabled. Ensure the group also
handles non-pull-request runs without disrupting existing push and pull_request
triggers.
- Around line 16-17: Update the workflow steps using actions/checkout and
trustabl/trustabl-action to reference approved immutable commit SHAs instead of
mutable tags. Configure trustabl/trustabl-action with an explicit approved
scanner release via its with.version input, avoiding the v0 tag and latest
default while preserving the existing workflow behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: b8b01d9f-79f1-446a-83a6-4c3028d29c2d
📒 Files selected for processing (1)
.github/workflows/trustabl.yml
| on: | ||
| push: | ||
| branches: [main] | ||
| pull_request: |
There was a problem hiding this comment.
🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick win
Cancel superseded scans.
Multiple commits to one pull request can run this workflow at the same time. Older Trustabl runs can continue after a newer commit and can publish obsolete annotations or PR comments. Add a workflow-level concurrency group keyed by this workflow and the pull request number. Set cancel-in-progress: true. GitHub documents this pattern for cancelling stale runs, and Trustabl publishes PR annotations and sticky comments by default. (docs.github.com)
Suggested change
on:
push:
branches: [main]
pull_request:
+concurrency:
+ group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
+ cancel-in-progress: true📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| on: | |
| push: | |
| branches: [main] | |
| pull_request: | |
| on: | |
| push: | |
| branches: [main] | |
| pull_request: | |
| concurrency: | |
| group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }} | |
| cancel-in-progress: true |
🧰 Tools
🪛 zizmor (1.28.0)
[warning] 2-5: insufficient job-level concurrency limits (concurrency-limits): workflow is missing concurrency setting
(concurrency-limits)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/trustabl.yml around lines 2 - 5, Add workflow-level
concurrency settings near the top-level triggers in the Trustabl workflow, using
a group keyed by the workflow identity and pull request number, with
cancel-in-progress enabled. Ensure the group also handles non-pull-request runs
without disrupting existing push and pull_request triggers.
Source: Linters/SAST tools
| permissions: | ||
| contents: read | ||
| security-events: write | ||
| pull-requests: write |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win
Document the required write permissions.
Keep security-events: write and pull-requests: write because Trustabl uses them for SARIF upload and sticky PR comments. Add inline comments so a future least-privilege change does not silently disable these output surfaces. (github.com)
Suggested change
permissions:
contents: read
- security-events: write
- pull-requests: write
+ security-events: write # Trustabl SARIF upload.
+ pull-requests: write # Trustabl sticky PR comment.📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| permissions: | |
| contents: read | |
| security-events: write | |
| pull-requests: write | |
| permissions: | |
| contents: read | |
| security-events: write # Trustabl SARIF upload. | |
| pull-requests: write # Trustabl sticky PR comment. |
🧰 Tools
🪛 zizmor (1.28.0)
[warning] 9-9: overly broad permissions (excessive-permissions): security-events: write is overly broad at the workflow level
(excessive-permissions)
[error] 10-10: overly broad permissions (excessive-permissions): pull-requests: write is overly broad at the workflow level
(excessive-permissions)
[warning] 9-9: permissions without explanatory comments (undocumented-permissions): needs an explanatory comment
(undocumented-permissions)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/trustabl.yml around lines 7 - 10, Add inline comments to
the security-events and pull-requests entries in the workflow permissions block,
documenting that Trustabl requires them for SARIF uploads and sticky PR
comments. Preserve both write permissions and the existing contents permission
unchanged.
Source: Linters/SAST tools
| scan: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 |
There was a problem hiding this comment.
🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick win
Avoid persisting the checkout token.
actions/checkout@v4 writes GITHUB_TOKEN to the local Git configuration unless persist-credentials is false. The Trustabl action already receives github.token for release lookup, SARIF upload, and PR comments, so this change is defense in depth. Set persist-credentials: false unless authenticated Git commands are required. (github.com)
Suggested change
- uses: actions/checkout@v4
+ with:
+ persist-credentials: false📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| - uses: actions/checkout@v4 | |
| - uses: actions/checkout@v4 | |
| with: | |
| persist-credentials: false |
🧰 Tools
🪛 zizmor (1.28.0)
[warning] 16-16: credential persistence through GitHub Actions artifacts (artipacked): does not set persist-credentials: false
(artipacked)
[error] 16-16: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)
(unpinned-uses)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/trustabl.yml at line 16, Update the actions/checkout@v4
step in the Trustabl workflow to set persist-credentials to false, preventing
the checkout token from being stored in local Git configuration while preserving
the existing github.token usage by the Trustabl action.
Source: Linters/SAST tools
| - uses: actions/checkout@v4 | ||
| - uses: trustabl/trustabl-action@v0 |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/usr/bin/env bash
set -euo pipefail
resolve_tag() {
local repo="$1" tag="$2" sha
sha="$(git ls-remote "https://github.com/${repo}.git" "refs/tags/${tag}^{}" | awk 'NR == 1 { print $1 }')"
if [[ -z "$sha" ]]; then
sha="$(git ls-remote "https://github.com/${repo}.git" "refs/tags/${tag}" | awk 'NR == 1 { print $1 }')"
fi
test "${`#sha`}" -eq 40
printf '%s@%s -> %s\n' "$repo" "$tag" "$sha"
}
resolve_tag actions/checkout v4
resolve_tag trustabl/trustabl-action v0.3.1Repository: Abstract-Data/mediaite-ghostink
Length of output: 309
🏁 Script executed:
#!/usr/bin/env bash
set -euo pipefail
echo "== workflow files =="
git ls-files '.github/workflows/**' || true
echo
echo "== trustabl workflow =="
if [ -f .github/workflows/trustabl.yml ]; then
nl -ba .github/workflows/trustabl.yml | sed -n '1,120p'
else
echo "missing .github/workflows/trustabl.yml"
fi
echo
echo "== trustabl action usages =="
rg -n "trustabl/trustabl-action|actions/checkout|with:|version:" .github/workflows || trueRepository: Abstract-Data/mediaite-ghostink
Length of output: 519
🌐 Web query:
Trustabl action GitHub v0 moving tag version input latest default
💡 Result:
For the Trustabl GitHub action (trustabl/trustabl-action), the v0 tag is a moving major version tag that automatically points to the latest stable release within the v0 major version range [1][2]. GitHub Actions do not automatically resolve semantic version ranges; instead, they pull the exact reference you specify [3]. When maintainers use a "moving tag" like v0, they are responsible for force-pushing that tag to the commit of the latest v0.x.x release [4][5][6]. By using @v0 in your workflow file, you benefit from receiving non-breaking updates (such as bug fixes and minor features) automatically [4][7][5]. If you prefer stability over automatic updates, you should pin your workflow to a specific immutable tag (e.g., @v0.5.0) or a full commit SHA [1][2][4][5]. Regarding the version input for the action itself: - The version input allows you to specify which version of the underlying trustabl CLI tool to run [1][2]. - This input defaults to latest if not specified, which instructs the action to use the most recent version of the CLI [2]. - For production environments, it is recommended to explicitly set the version input to a specific CLI version (e.g., version: v0.5.0) to ensure your CI runs remain reproducible [1][2]. In summary, the @v0 tag controls the version of the action wrapper itself, while the version input controls the version of the Trustabl binary executed by that action [1][2].
Citations:
- 1: https://github.com/trustabl/trustabl-action
- 2: https://claudeers.com/trustabl-action
- 3: https://xebia.com/blog/how-github-actions-versioning-system-works/
- 4: https://github.com/actions/toolkit/blob/master/docs/action-versioning.md
- 5: https://github.com/github/docs/blob/main/content/actions/how-tos/create-and-publish-actions/manage-custom-actions.md
- 6: https://github.com/jessehouwing/actions-semver-checker/blob/main/docs/versioning-guidance.md
- 7: https://docs.github.com/en/actions/how-tos/create-and-publish-actions/release-and-maintain-actions
🌐 Web query:
GitHub Actions security actions checkout recommend pin to commit SHA
💡 Result:
Yes, GitHub strongly recommends pinning actions, including actions/checkout, to a full-length 40-character commit SHA to ensure security and immutability [1][2][3]. While tags (like @v7) and branch references (like @main) are convenient, they are mutable, meaning an action maintainer or an attacker who gains access to the repository could move the tag or branch to point to malicious code [4][2][5]. Pinning to a full-length commit SHA creates an immutable reference, ensuring that your workflow always executes the exact code you have reviewed [2][6][7]. To maintain security while still benefiting from updates, the recommended best practice is to use the full commit SHA while keeping the version number as a trailing comment [4][8][7]. This allows automated tools like Dependabot and Renovate to continue tracking new releases and creating pull requests to update your actions [4][8][7]. Example of recommended pinning syntax: - uses: actions/checkout@9c091bb # v7.0.0 GitHub also provides organization and repository-level policies that administrators can enable to enforce SHA pinning, causing any workflow that uses an unpinned action to fail [1][2][9]. While GitHub-owned actions are generally considered lower risk, pinning them remains a recommended consistency practice for security-hardened environments [5][7].
Citations:
- 1: https://github.blog/changelog/2025-08-15-github-actions-policy-now-supports-blocking-and-sha-pinning-actions/
- 2: https://docs.github.com/en/actions/reference/security/secure-use
- 3: https://github.com/github/docs/blob/962a1c8dccb8c0f66548b324e5b921b5e4fbc3d6/content/actions/security-for-github-actions/security-guides/security-hardening-for-github-actions.md
- 4: https://starsling.dev/best-practices/github-actions/pin-action-shas
- 5: https://safeguard.sh/resources/blog/how-to-pin-github-actions-to-shas-correctly
- 6: https://microsoft.github.io/zerotrustassessment/docs/workshop-guidance/devsecops/DS_032
- 7: https://tomodahinata.com/en/blog/dependabot-github-actions-sha-pinning-supply-chain-security-guide
- 8: https://carlosbecker.com/posts/pinning-github-actions/
- 9: https://github.com/github/docs/blob/main/content/actions/reference/security/secure-use.md
Pin the workflow actions and Trustabl scanner release.
Use immutable commit SHAs for actions/checkout and trustabl/trustabl-action. Set trustabl/trustabl-action with.version to an approved release instead of relying on the v0 action tag or scanner latest default. Mutable action tags and latest CLI versions can change executed code or scan results without a repository change.
Suggested change
- - uses: actions/checkout@v4
+ - uses: actions/checkout@<verified-40-character-SHA> # v4
with:
persist-credentials: false
- - uses: trustabl/trustabl-action@v0
+ - uses: trustabl/trustabl-action@<verified-40-character-SHA> # approved action release
+ with:
+ version: <approved-trustabl-release>🧰 Tools
🪛 zizmor (1.28.0)
[warning] 16-16: credential persistence through GitHub Actions artifacts (artipacked): does not set persist-credentials: false
(artipacked)
[error] 16-16: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)
(unpinned-uses)
[error] 17-17: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)
(unpinned-uses)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/trustabl.yml around lines 16 - 17, Update the workflow
steps using actions/checkout and trustabl/trustabl-action to reference approved
immutable commit SHAs instead of mutable tags. Configure
trustabl/trustabl-action with an explicit approved scanner release via its
with.version input, avoiding the v0 tag and latest default while preserving the
existing workflow behavior.
Source: Linters/SAST tools
1 participant
We came across your repo and we like the unique approach you're taking to combine statistical stylometry with embedding drift analysis for AI writing forensics. We scanned the repo, and noticed agent runtime reliability findings that might be worth reviewing.
[MEDIUM] Skill fetches untrusted external content
File: .claude/skills/gitbutler-workflow/SKILL.md
What it means: This skill's body references an external http(s) URL.
[MEDIUM] Skill fetches untrusted external content
File: website/.claude/skills/abstract-data-docs-author/SKILL.md
What it means: This skill's body references an external http(s) URL.
[MEDIUM] Skill fetches untrusted external content
File: website/.claude/skills/abstract-data-setup/SKILL.md
What it means: This skill's body references an external http(s) URL.
Recommendations are based on our understanding of agent runtime reliability, some findings may be intentional. Please let us know if this was intentional or if our findings are helpful so we can improve the accuracy of the scanner.
Best,
Trustabl.ai
Open-source AI agent reliability scanner (runs locally, GitHub Action)
Summary by CodeRabbit