Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
80 changes: 80 additions & 0 deletions docs/mkdocs/en/tool_safety.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
# Tool Script Safety

`trpc_agent_sdk.tools.safety` provides pre-execution static scanning for Python
and Bash, policy decisions, Tool Filter and CodeExecutor integrations, audit
events, and OpenTelemetry attributes.

## Architecture

The guard has four independent parts:

1. `ToolSafetyScanner` converts `SafetyScanRequest` into a structured
`SafetyReport`.
2. `ToolScriptSafetyFilter` blocks unsafe Tool, MCP Tool, or Skill arguments
before their implementation runs.
3. `SafetyGuardedCodeExecutor` scans code blocks before delegating to an
existing local, container, or remote executor.
4. audit sinks and span attributes expose decisions to monitoring systems.

The default decision precedence is `deny` over `needs_human_review` over
`allow`. Human-review findings fail closed unless an application explicitly
continues after reviewing the exact script hash and report.

## Configuration

Load a strict YAML policy:

```python
from trpc_agent_sdk.tools.safety import ToolSafetyPolicy
from trpc_agent_sdk.tools.safety import ToolSafetyScanner

policy = ToolSafetyPolicy.from_yaml("tool_safety_policy.yaml")
scanner = ToolSafetyScanner(policy)
```

The policy configures allowed domains and commands, denied paths, timeout and
output limits, resource heuristics, disabled rules, and per-rule decision or
risk overrides. See
[`examples/tool_safety_guard/tool_safety_policy.yaml`](https://github.com/trpc-group/trpc-agent-python/tree/main/examples/tool_safety_guard/tool_safety_policy.yaml).

## Integration

Attach `ToolScriptSafetyFilter` through a Tool's `filters` list. Its default
extractor recognizes common execution fields such as `script`, `code`,
`command`, `cwd`, `env`, and `timeout`. Supply a custom extractor for another
schema.

Wrap a CodeExecutor with `SafetyGuardedCodeExecutor`. The wrapper preserves the
delegate's executor configuration, rejects disallowed code before delegation,
emits an event for every scanned block, and limits returned output.

The full runnable examples, 12 public scan samples, generated report, and audit
log are under
[`examples/tool_safety_guard`](https://github.com/trpc-group/trpc-agent-python/tree/main/examples/tool_safety_guard).

## Acceptance verification

Run the public scanner and focused suite from the repository root:

```bash
python examples/tool_safety_guard/tool_safety_check.py \
--report /tmp/tool_safety_report.json \
--audit /tmp/tool_safety_audit.jsonl
pytest -q tests/tools/safety
```

The public samples produce 2 `allow`, 7 `deny`, and 3
`needs_human_review` decisions. The focused suite contains 47 tests, including
the 500-line scan performance assertion.

## Security boundary

Static analysis is not a sandbox. The guard can be bypassed by obfuscation,
generated code, runtime downloads, native binaries, complex shell expansion,
symlink races, DNS rebinding, or behavior hidden in dependencies. It can also
produce false positives.

Production deployments must still isolate network access, mounts, credentials,
process identity, syscalls, CPU, memory, disk, PIDs, time, and output. The guard
reduces obvious unsafe execution and improves explainability and auditability;
the sandbox contains behavior that is only visible at runtime.
68 changes: 68 additions & 0 deletions docs/mkdocs/zh/tool_safety.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# Tool 脚本安全

`trpc_agent_sdk.tools.safety` 为 Python 和 Bash 提供执行前静态扫描、策略决策、
Tool Filter 与 CodeExecutor 接入、审计事件和 OpenTelemetry 属性。

## 架构

安全守卫由四个独立部分组成:

1. `ToolSafetyScanner` 将 `SafetyScanRequest` 转换为结构化 `SafetyReport`。
2. `ToolScriptSafetyFilter` 在 Tool、MCP Tool 或 Skill 的实现运行前拦截危险参数。
3. `SafetyGuardedCodeExecutor` 在委托现有本地、容器或远程执行器前扫描代码块。
4. 审计 Sink 与 span attributes 将决策提供给监控系统。

默认决策优先级为 `deny`、`needs_human_review`、`allow`。人工复核结果默认阻断;
应用只有在核对准确的脚本哈希和报告后,才能显式继续执行。

## 配置

加载严格校验的 YAML 策略:

```python
from trpc_agent_sdk.tools.safety import ToolSafetyPolicy
from trpc_agent_sdk.tools.safety import ToolSafetyScanner

policy = ToolSafetyPolicy.from_yaml("tool_safety_policy.yaml")
scanner = ToolSafetyScanner(policy)
```

策略可配置白名单域名与命令、禁止路径、超时与输出限制、资源风险阈值、禁用规则,
以及逐规则的决策和风险等级覆盖。完整示例见
[`examples/tool_safety_guard/tool_safety_policy.yaml`](https://github.com/trpc-group/trpc-agent-python/tree/main/examples/tool_safety_guard/tool_safety_policy.yaml)。

## 接入

通过 Tool 的 `filters` 列表挂载 `ToolScriptSafetyFilter`。默认提取器识别
`script`、`code`、`command`、`cwd`、`env`、`timeout` 等常见执行参数;
其他参数结构可提供自定义提取器。

使用 `SafetyGuardedCodeExecutor` 包装 CodeExecutor。wrapper 会保留委托执行器的
配置,在委托前拒绝不允许的代码,为每个扫描代码块输出事件,并限制返回内容大小。

完整可运行示例、12 个公开扫描样本、结构化报告与审计日志位于
[`examples/tool_safety_guard`](https://github.com/trpc-group/trpc-agent-python/tree/main/examples/tool_safety_guard)。

## 验收验证

在仓库根目录运行公开扫描器与专项测试:

```bash
python examples/tool_safety_guard/tool_safety_check.py \
--report /tmp/tool_safety_report.json \
--audit /tmp/tool_safety_audit.jsonl
pytest -q tests/tools/safety
```

12 个公开样本的结果为 2 个 `allow`、7 个 `deny` 和 3 个
`needs_human_review`。专项测试共 47 条,其中包含 500 行脚本的扫描性能断言。

## 安全边界

静态扫描不能替代沙箱。混淆、动态生成代码、运行时下载、原生二进制、复杂 shell
展开、符号链接竞态、DNS rebinding 或依赖内部行为都可能绕过规则,规则也可能产生
误报。

生产环境仍必须隔离网络、挂载目录、凭据、进程身份和系统调用,并限制 CPU、内存、
磁盘、PID、时间和输出。安全守卫用于提前减少明显危险执行并提高可解释性与可审计性;
沙箱负责约束只能在运行时观察到的行为。
154 changes: 154 additions & 0 deletions examples/tool_safety_guard/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,154 @@
# Tool Script Safety Guard

This example adds a policy-driven safety boundary before script-capable Tools,
MCP Tools, Skills, and CodeExecutors. It scans Python source and Bash commands,
then returns one of:

- `allow`: no configured rule requires intervention.
- `deny`: a high-confidence prohibited pattern was found.
- `needs_human_review`: the scanner cannot prove the operation is safe.

The scanner is deterministic and does not execute the submitted script.

## Run the public samples

From the repository root:

```bash
python examples/tool_safety_guard/tool_safety_check.py \
--report /tmp/tool_safety_report.json \
--audit /tmp/tool_safety_audit.jsonl
```

The command scans all 12 files under `samples/`, prints a decision table, and
writes:

- `tool_safety_report.json`: complete findings with rule IDs, redacted evidence,
recommendations, risk levels, and decisions.
- `tool_safety_audit.jsonl`: compact monitoring events without script content.

Omit `--report` and `--audit` for a read-only scan. The committed JSON and
JSONL files are example snapshots and are not overwritten by default.

Scan one file with:

```bash
python examples/tool_safety_guard/tool_safety_check.py path/to/script.py \
--language python
```

No sample is executed by this command.

## Policy

`tool_safety_policy.yaml` controls:

- `allowed_domains`: exact hosts or `*.example.org` subdomain patterns.
- `allowed_commands`: Bash executables that do not require human review.
- `denied_paths`: protected absolute prefixes and sensitive basenames.
- `max_timeout_seconds`, `max_output_bytes`, `max_script_bytes`,
`max_file_write_bytes`, `max_sleep_seconds`, and `max_concurrent_tasks`.
- `rule_decisions` and `rule_risk_levels`: reviewed per-rule overrides.
- `disabled_rules`: emergency suppression for a known false positive.

Changing these fields requires no code change. Invalid or unknown YAML fields
fail policy loading rather than being silently ignored.

## Tool, MCP Tool, and Skill integration

Attach a filter instance to a script-capable Tool:

```python
from trpc_agent_sdk.tools import FunctionTool
from trpc_agent_sdk.tools.safety import JsonlAuditSink
from trpc_agent_sdk.tools.safety import ToolSafetyPolicy
from trpc_agent_sdk.tools.safety import ToolSafetyScanner
from trpc_agent_sdk.tools.safety import ToolScriptSafetyFilter

policy = ToolSafetyPolicy.from_yaml("tool_safety_policy.yaml")
guard = ToolScriptSafetyFilter(
scanner=ToolSafetyScanner(policy),
audit_sink=JsonlAuditSink("tool_safety_audit.jsonl"),
)
tool = FunctionTool(execute_script, filters=[guard])
```

The default extractor recognizes `script`, `code`, or `command`, plus
`language`, `args`/`command_args`, `cwd`/`working_directory`,
`env`/`environment`, and `timeout`/`timeout_seconds`. MCP and Skill tools with
different schemas can pass a custom `request_extractor`. See
`filter_example.py`.

`needs_human_review` is blocked by default. A reviewed application can create a
separate filter with `allow_human_review=True` only after its human approval
workflow has verified the exact report and script hash.

## CodeExecutor integration

Wrap an existing executor:

```python
guarded = SafetyGuardedCodeExecutor(
executor=container_executor,
scanner=ToolSafetyScanner(policy),
audit_sink=JsonlAuditSink("tool_safety_audit.jsonl"),
)
```

Every code block is scanned before delegation. A blocked block prevents the
delegate from running. Allowed output is truncated to `max_output_bytes`. See
`executor_example.py`.

## Rules and decisions

| Category | Representative rules | Default decision |
| --- | --- | --- |
| File access | recursive deletion, protected credentials, file deletion | deny/review |
| Network | non-allowlisted or dynamic targets | deny/review |
| Process | subprocess, shell injection, pipelines, sudo, unknown commands | deny/review |
| Dependencies | pip/npm/apt and similar runtime installs | deny |
| Resources | unbounded loops, fork bomb, long sleep, excessive workers/write | deny/review |
| Secrets | credentials written to logs, files, or network calls | deny |

Reports never contain the complete script or environment map. They include a
SHA-256 digest and bounded evidence snippets. Known environment secret values
and credential-shaped literals are replaced with `[REDACTED]`.

Audit events contain `tool_name`, `decision`, `risk_level`, primary `rule_id`,
all `rule_ids`, `duration_ms`, `redacted`, `blocked`, the script digest, and the
policy version. The JSONL sink intentionally omits script and environment
content.

OpenTelemetry attributes are attached to the active span:

- `tool.safety.decision`
- `tool.safety.risk_level`
- `tool.safety.rule_id`
- `tool.safety.rule_ids`
- `tool.safety.duration_ms`
- `tool.safety.redacted`
- `tool.safety.blocked`
- `tool.safety.policy_version`

## Relationship to sandboxing

This guard complements, but cannot replace, a sandbox. Static scanning can stop
obvious unsafe intent before paying execution cost and provides explainable
policy decisions. A sandbox must still enforce runtime network policy,
filesystem mounts, process identity, syscall restrictions, PID limits,
CPU/memory/disk quotas, timeouts, and output limits.

Known bypasses and limitations include obfuscated or encoded payloads, runtime
downloads, reflection, aliases, generated code, unusual shell expansion,
native binaries, symlink and time-of-check/time-of-use races, DNS rebinding, and
semantic behavior hidden behind apparently safe libraries. AST data-flow
tracking is intentionally shallow. Unknown or dynamic behavior is generally
sent to human review, but false positives and false negatives remain possible.

## Extending rules

Add Python AST checks in `_python_scanner.py`, Bash checks in
`_bash_scanner.py`, and a stable rule ID plus a focused test. Evidence must be
bounded and redacted. Use `deny` only for high-confidence prohibited behavior;
use `needs_human_review` for uncertainty. Keep runtime enforcement in the
sandbox rather than trying to emulate it in static rules.
30 changes: 30 additions & 0 deletions examples/tool_safety_guard/executor_example.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Tencent is pleased to support the open source community by making tRPC-Agent-Python available.
#
# Copyright (C) 2026 Tencent. All rights reserved.
#
# tRPC-Agent-Python is licensed under the Apache License Version 2.0.
"""Minimal CodeExecutor wrapper integration example."""

from pathlib import Path

from trpc_agent_sdk.code_executors import ContainerCodeExecutor
from trpc_agent_sdk.tools.safety import JsonlAuditSink
from trpc_agent_sdk.tools.safety import SafetyGuardedCodeExecutor
from trpc_agent_sdk.tools.safety import ToolSafetyPolicy
from trpc_agent_sdk.tools.safety import ToolSafetyScanner

EXAMPLE_DIR = Path(__file__).resolve().parent
policy = ToolSafetyPolicy.from_yaml(EXAMPLE_DIR / "tool_safety_policy.yaml")

# Static scanning is an additional boundary, not a sandbox replacement. The
# delegated executor should still enforce network, filesystem, process, and
# resource isolation.
sandbox_executor = ContainerCodeExecutor(
image="python:3.12-slim",
timeout=policy.max_timeout_seconds,
)
guarded_executor = SafetyGuardedCodeExecutor(
executor=sandbox_executor,
scanner=ToolSafetyScanner(policy),
audit_sink=JsonlAuditSink(EXAMPLE_DIR / "tool_safety_audit.jsonl"),
)
31 changes: 31 additions & 0 deletions examples/tool_safety_guard/filter_example.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# Tencent is pleased to support the open source community by making tRPC-Agent-Python available.
#
# Copyright (C) 2026 Tencent. All rights reserved.
#
# tRPC-Agent-Python is licensed under the Apache License Version 2.0.
"""Minimal Tool Filter integration example."""

from pathlib import Path

from trpc_agent_sdk.tools import FunctionTool
from trpc_agent_sdk.tools.safety import JsonlAuditSink
from trpc_agent_sdk.tools.safety import ToolSafetyPolicy
from trpc_agent_sdk.tools.safety import ToolSafetyScanner
from trpc_agent_sdk.tools.safety import ToolScriptSafetyFilter

EXAMPLE_DIR = Path(__file__).resolve().parent
policy = ToolSafetyPolicy.from_yaml(EXAMPLE_DIR / "tool_safety_policy.yaml")
guard = ToolScriptSafetyFilter(
scanner=ToolSafetyScanner(policy),
audit_sink=JsonlAuditSink(EXAMPLE_DIR / "tool_safety_audit.jsonl"),
)


def execute_script(script: str, language: str) -> dict:
"""Placeholder script-capable Tool implementation."""
return {"executed": True, "language": language, "script_size": len(script)}


# The guard runs before execute_script. It can also be attached to MCP or Skill
# tools that expose script/code/command fields in their argument dictionaries.
safe_tool = FunctionTool(execute_script, filters=[guard])
4 changes: 4 additions & 0 deletions examples/tool_safety_guard/samples/01_safe_python.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
"""A safe Python sample used by the tool safety guard demo."""

values = [2, 4, 6]
print(sum(values))
3 changes: 3 additions & 0 deletions examples/tool_safety_guard/samples/02_dangerous_delete.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/usr/bin/env bash

rm -rf /var/lib/application
6 changes: 6 additions & 0 deletions examples/tool_safety_guard/samples/03_read_ssh_key.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
"""A dangerous credential access sample."""

from pathlib import Path

private_key = Path("~/.ssh/id_rsa").expanduser().read_text()
print(len(private_key))
6 changes: 6 additions & 0 deletions examples/tool_safety_guard/samples/04_external_network.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
"""A non-allowlisted network request sample."""

import requests

response = requests.get("https://collector.example.net/upload", timeout=10)
print(response.status_code)
6 changes: 6 additions & 0 deletions examples/tool_safety_guard/samples/05_allowed_network.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
"""An allowlisted network request sample."""

import requests

response = requests.get("https://api.example.com/health", timeout=5)
print(response.status_code)
Loading
Loading