Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
9d884d4
tools/safety: add tool script safety guard
Wsp030914 Jul 26, 2026
9a7af9c
fix(safety): close scanner fail-open paths
Wsp030914 Jul 26, 2026
6ae981a
test(safety): improve rule coverage
Wsp030914 Jul 26, 2026
9bc9de6
fix(safety): guard MCP command execution
Wsp030914 Jul 26, 2026
312bc25
fix(safety): close review edge cases
Wsp030914 Jul 26, 2026
127e053
fix(safety): harden recursive delete detection
Wsp030914 Jul 26, 2026
5d9985d
fix(safety): harden static loop analysis
Wsp030914 Jul 27, 2026
63bac6d
fix(safety): close audit integration gaps
Wsp030914 Jul 27, 2026
a907f6d
fix(safety): stabilize review edge cases
Wsp030914 Jul 27, 2026
e196644
fix(safety): close static deletion gaps
Wsp030914 Jul 27, 2026
8d685ad
fix(safety): harden output and audit handling
Wsp030914 Jul 27, 2026
75e8e0c
fix(safety): harden MCP timeout recovery
Wsp030914 Jul 27, 2026
9a90c50
fix(safety): preserve truncation and timeout types
Wsp030914 Jul 27, 2026
a3596af
fix(safety): tighten audit and shell checks
Wsp030914 Jul 27, 2026
e3d7256
fix(safety): make truncation and audit checks observable
Wsp030914 Jul 27, 2026
7bfa862
fix(safety): avoid literal loop condition materialization
Wsp030914 Jul 27, 2026
0a0a149
fix(safety): truncate formatted bash output
Wsp030914 Jul 27, 2026
e64d987
fix(safety): tighten output and truthiness checks
Wsp030914 Jul 27, 2026
f88eb98
fix(safety): harden audit failure handling
Wsp030914 Jul 27, 2026
ac271dc
fix(safety): restore audit parent permissions
Wsp030914 Jul 27, 2026
6ede31a
fix(safety): bound static loop truthiness
Wsp030914 Jul 27, 2026
9ae8931
fix(safety): detect nested truthy negations
Wsp030914 Jul 27, 2026
ea5ee82
fix(safety): harden audit and timeout cleanup
Wsp030914 Jul 27, 2026
cd055ed
fix(safety): redact mcp command output
Wsp030914 Jul 27, 2026
d55bdfa
fix(safety): avoid inherited env and parent chmod
Wsp030914 Jul 27, 2026
58cdb6a
fix(safety): fsync audit records
Wsp030914 Jul 27, 2026
eacda0f
fix(safety): create audit parents privately
Wsp030914 Jul 27, 2026
8668369
fix(safety): cover process and http call variants
Wsp030914 Jul 27, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
149 changes: 149 additions & 0 deletions examples/tool_safety_guard/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
# Tool Script Safety Guard

该示例展示 Python/Bash 静态扫描、Tool Filter、CodeExecutor wrapper、策略、
报告、审计和 OpenTelemetry 接入。

## 交付物

- `README.md`:规则、接入、扩展方式和安全边界。
- `tool_safety_policy.yaml`:可修改的策略示例。
- `tool_safety_report.json`:结构化扫描报告示例。
- `tool_safety_audit.jsonl`:审计事件示例。
- `manifest.yaml` 与 `samples/`:12 个公开验收样本及预期决策。
- `real_agent.py`、`mcp_server.py` 与 `skills/`:真实 Agent 执行示例。

`tool_safety_report.json` 和 `tool_safety_audit.jsonl` 是由 CLI 生成的示例产物,
仅用于展示格式,不是固定契约;规则或归一化逻辑变更后可删除并按 CLI 命令重新生成。

## CLI

```bash
python scripts/tool_safety_check.py \
--file examples/tool_safety_guard/samples/danger_delete.py \
--language python \
--policy examples/tool_safety_guard/tool_safety_policy.yaml \
--report tool_safety_report.json \
--audit tool_safety_audit.jsonl
```

退出码:`0=allow`、`2=needs_human_review`、`3=deny`、`1=CLI/config error`。

## Tool Filter

```python
from trpc_agent_sdk.tools import BashTool
from trpc_agent_sdk.tools.safety import JsonlAuditSink
from trpc_agent_sdk.tools.safety import ToolSafetyFilter

audit = JsonlAuditSink("tool_safety_audit.jsonl")
safety_filter = ToolSafetyFilter.from_policy(
"examples/tool_safety_guard/tool_safety_policy.yaml",
audit,
)
tool = BashTool(cwd=".")
tool.add_one_filter(safety_filter)
```

Filter 在 `_run_async_impl()` 前扫描。`deny` 和 `needs_human_review` 不执行
handler;`allow` 把有效 timeout 注入 Tool 参数,并在 `_after` 阶段限制返回给
Agent 的 output 大小。安全 Filter 应放在所有参数改写 Filter 之后,后续 Filter
也不应扩展其已截断的输出。CodeExecutor wrapper 另外使用协作式 deadline。
已知 Tool 使用固定 adapter;未知 Tool 只要提供非空 `command`、`code` 或
`script` 字段,也会按 Bash/Python 保守扫描,避免自定义执行 Tool 静默绕过。

## CodeExecutor

```python
from trpc_agent_sdk.tools.safety import SafetyGuardedCodeExecutor
from trpc_agent_sdk.tools.safety import ToolScriptSafetyGuard

safe_executor = SafetyGuardedCodeExecutor(
delegate=executor,
guard=ToolScriptSafetyGuard.from_policy(
"examples/tool_safety_guard/tool_safety_policy.yaml",
),
audit_sink=audit,
)
```

wrapper 统一执行扫描、审计、阻断、wall-clock timeout 和返回 output 截断;
超时返回 `outcome=DEADLINE_EXCEEDED`,并在 output 中包含超时提示。

## 真实模型 Agent

`real_agent.py` 创建一个真正的 `LlmAgent`,同时接入:

- `BashTool`
- `SkillToolSet` 的 `skill_run`/`skill_exec`/`workspace_exec`
- 本地 stdio MCP Tool `execute_command`
- `SafetyGuardedCodeExecutor`

MCP 示例使用 argv-only 的 `create_subprocess_exec`,不提供 Shell 管道、重定向或
命令拼接语义;此类输入会在执行前进入人工审核。`mcp-review` 使用未加入命令
白名单的 `uname -a` 演示审核路径。
MCP handler 独立运行时只负责扫描和阻断,不写 Agent 侧审计事件;通过
`ToolSafetyFilter` 接入 Agent 时,审计由 Filter 的 `AuditSink` 统一记录,避免重复事件。

每个入口都提供 `allow`、`review`、`deny` 场景:

```bash
export TRPC_AGENT_API_KEY='<your-key>'
export TRPC_AGENT_BASE_URL='https://api.deepseek.com'
export TRPC_AGENT_MODEL_NAME='deepseek-v4-flash'

python examples/tool_safety_guard/real_agent.py --list-scenarios
python examples/tool_safety_guard/real_agent.py all
python examples/tool_safety_guard/real_agent.py tool-allow
python examples/tool_safety_guard/real_agent.py mcp-review
python examples/tool_safety_guard/real_agent.py skill-deny
python examples/tool_safety_guard/real_agent.py executor-allow
```

场景共 12 个,命名为
`{tool|mcp|skill|executor}-{allow|review|deny}`。终端输出模型实际发出的 `CALL`
和框架返回的 `RESULT`,audit 默认写入 `real_agent_audit.jsonl`。

MCP 子进程只继承运行所需的 `PATH`/Python/系统环境,不继承 API key、token 等
父进程密钥。

review/deny 使用演示目录内的有限副作用命令,但本示例仍包含真实本地执行器。
不要在生产主机运行;生产必须换成容器/沙箱执行器。密钥只通过环境变量传入,
禁止写入代码、策略、prompt 或 audit。

### 真实运行结果

使用 `deepseek-v4-flash` 实际运行四类入口后的结果:

| 入口 | allow | review | deny |
|---|---|---|---|
| `BashTool` | handler 执行,stdout=`tool-allow` | `PROC001/medium`,未执行 | `FILE001/critical`,未执行 |
| Skill `skill_run` | workspace 执行,stdout=`skill-allow` | `PROC001/medium`,未执行 | `FILE001/critical`,未执行 |
| MCP `execute_command` | MCP server 收到调用,stdout=`mcp-allow` | `PROC001/medium`,MCP server 未收到调用 | `FILE001/critical`,MCP server 未收到调用 |
| CodeExecutor | `Outcome.OUTCOME_OK`,输出 `executor-allow` | `PROC001/high`,delegate 未执行 | `FILE001/critical`,delegate 未执行 |

模型在 `skill-review` 的自然语言总结中曾错误描述为“没有阻断”,但实际
`RESULT` 是 `needs_human_review`,且 handler 未运行。这说明安全验收必须以
结构化 Tool result 和 audit 为准,不能信任模型对安全结果的二次转述。
真实模型验收依赖付费外部 API 且输出非确定,因此不进入默认 pytest;上述
`all` 命令是可重复的显式验收入口,默认测试继续覆盖确定性的 handler 未调用
断言。

## 扩展规则

在 `_python_rules.py`、`_bash_rules.py` 或 `_common_rules.py` 中增加小型规则,
使用 `_common_rules.py` 的 `RuleSpec` 和 `make_finding()`,并补充 rule id、decision、
evidence、recommendation 测试。策略字段由 Pydantic 严格校验。

## 安全边界

该机制是执行前静态检查,不是沙箱。动态拼接、反射、编码混淆、运行时下载、
符号链接、DNS 重绑定和未知解释器语义可能绕过规则。它也不能强制 CPU、内存、
PID、磁盘、网络或子进程内核输出配额。
不响应 Python 取消的第三方 handler 也可能延迟返回,必须由执行器或容器提供
进程级硬超时和清理。

生产环境仍需容器/沙箱、最小权限、只读挂载、网络白名单和运行时资源限制。
Filter 只保护明确挂载它的 Tool;必须清点所有 Tool、MCP Tool、Skill 和
CodeExecutor 执行入口。JSONL 仅为本地示例,多进程生产部署应使用集中审计。
POSIX 上 JSONL 文件强制为 `0600`;自定义 audit sink 通过 `emit_report` 串行
调用。
13 changes: 13 additions & 0 deletions examples/tool_safety_guard/manifest.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
samples:
- {file: samples/safe_python.py, language: python, expected: allow, safe: true}
- {file: samples/safe_allowed_request.py, language: python, expected: allow, safe: true}
- {file: samples/danger_delete.py, language: python, expected: deny, category: file}
- {file: samples/danger_ssh.py, language: python, expected: deny, category: file}
- {file: samples/danger_network.py, language: python, expected: deny, category: network}
- {file: samples/review_dynamic_network.py, language: python, expected: needs_human_review, category: network}
- {file: samples/review_subprocess.py, language: python, expected: needs_human_review, category: process}
- {file: samples/danger_shell_injection.sh, language: bash, expected: deny, category: process}
- {file: samples/review_dependency.sh, language: bash, expected: needs_human_review, category: dependency}
- {file: samples/danger_loop.py, language: python, expected: deny, category: resource}
- {file: samples/danger_secret.py, language: python, expected: deny, category: secret}
- {file: samples/review_pipeline.sh, language: bash, expected: needs_human_review, category: process}
148 changes: 148 additions & 0 deletions examples/tool_safety_guard/mcp_server.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
# Tencent is pleased to support the open source community by making tRPC-Agent-Python available.
#
# Copyright (C) 2026 Tencent. All rights reserved.
#
# tRPC-Agent-Python is licensed under Apache-2.0.
"""Local stdio MCP command server used by the real-agent safety demo."""

from __future__ import annotations

import asyncio
import math
import os
import shlex
from pathlib import Path

from mcp.server import FastMCP
from trpc_agent_sdk.tools.safety import ScriptLanguage
from trpc_agent_sdk.tools.safety import ScriptPayload
from trpc_agent_sdk.tools.safety import ScriptScanRequest
from trpc_agent_sdk.tools.safety import SafetyDecision
from trpc_agent_sdk.tools.safety import ToolMetadata
from trpc_agent_sdk.tools.safety import ToolScriptSafetyGuard

APP = FastMCP("tool-safety-demo")
WORK_DIR = Path(__file__).resolve().parent
POLICY_PATH = WORK_DIR / "tool_safety_policy.yaml"
GUARD = ToolScriptSafetyGuard.from_policy(POLICY_PATH)
MAX_OUTPUT_CHARS = 4096
PROCESS_REAP_TIMEOUT_SECONDS = 1.0
SUBPROCESS_ENV_KEYS = ("PATH", "SYSTEMROOT", "WINDIR")


def _subprocess_env() -> dict[str, str]:
"""Pass only runtime loader essentials to approved demo commands."""
return {key: os.environ[key] for key in SUBPROCESS_ENV_KEYS if key in os.environ}


def _safe_response(response: dict) -> dict:
"""Redact secret-looking output before returning it to the Agent."""
safe = dict(response)
redacted = bool(safe.get("redacted", False))
for key in ("stdout", "stderr", "output", "formatted_output"):
item = safe.get(key)
if not isinstance(item, str):
continue
safe_text, changed = GUARD.sanitizer.sanitize(item)
safe[key] = safe_text
redacted = redacted or changed
if redacted:
safe["redacted"] = True
return GUARD.limit_output(safe)


@APP.tool()
async def execute_command(command: str, timeout: float | None = None) -> dict:
"""Execute an approved shell command in the disposable example directory."""
requested_timeout = float(timeout) if isinstance(timeout, (int, float)) else None
if requested_timeout is not None and not math.isfinite(requested_timeout):
return {
"decision": SafetyDecision.NEEDS_HUMAN_REVIEW.value,
"summary": "needs_human_review: timeout must be finite.",
"execution_blocked": True,
}
if requested_timeout is not None and requested_timeout <= 0:
requested_timeout = None
timeout_limit = float(GUARD.policy.max_timeout_seconds)
requested_or_default = requested_timeout if requested_timeout is not None else timeout_limit
effective_timeout = min(
requested_or_default,
timeout_limit,
)
request = ScriptScanRequest(
payloads=[ScriptPayload(
language=ScriptLanguage.BASH,
content=command,
source="mcp.execute_command",
)],
cwd=str(WORK_DIR),
execution_root=str(WORK_DIR.anchor),
metadata=ToolMetadata(name="execute_command"),
requested_timeout_seconds=requested_timeout,
effective_timeout_seconds=effective_timeout,
max_output_bytes=GUARD.policy.max_output_bytes,
)
report = GUARD.scan(request)
if report.decision != SafetyDecision.ALLOW:
return {
**report.as_dict(),
"execution_blocked": True,
}
try:
argv = shlex.split(command, posix=True)
except ValueError as error:
return {
"decision": SafetyDecision.NEEDS_HUMAN_REVIEW.value,
"summary": "needs_human_review: malformed shell command.",
"error": str(error),
"execution_blocked": True,
}
if not argv:
return {
"decision": SafetyDecision.NEEDS_HUMAN_REVIEW.value,
"summary": "needs_human_review: empty shell command.",
"execution_blocked": True,
}
process = await asyncio.create_subprocess_exec(
*argv,
cwd=WORK_DIR,
env=_subprocess_env(),
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
)
try:
stdout, stderr = await asyncio.wait_for(
process.communicate(),
timeout=request.effective_timeout_seconds,
)
except asyncio.TimeoutError:
reap_timed_out = False
try:
process.kill()
except ProcessLookupError:
pass
try:
await asyncio.wait_for(
process.communicate(),
timeout=PROCESS_REAP_TIMEOUT_SECONDS,
)
except Exception:
reap_timed_out = True
response = {
"return_code": None,
"stdout": "",
"stderr": "Command exceeded the tool safety timeout.",
"timed_out": True,
"reap_timed_out": reap_timed_out,
}
return _safe_response(response)
response = {
"return_code": process.returncode,
"stdout": stdout.decode(errors="replace")[:MAX_OUTPUT_CHARS],
"stderr": stderr.decode(errors="replace")[:MAX_OUTPUT_CHARS],
}
return _safe_response(response)


if __name__ == "__main__":
APP.run(transport="stdio")
Loading
Loading