BoundBench

PraisonAI

Python multi-agent framework (praisonaiagents SDK plus praisonai wrapper/CLI) for building tool-using, memory-enabled agents and teams.

github.com/mervinpraison/praisonai · 2026-10-04 · e49b254

Defense-in-depth score

3.0 / 10

Minimal

PraisonAI's core SDK has real safety work in it: a bare Agent hard-denies shell/code execution when not on a terminal, prompts per call on a terminal, and its file tools are confined to the working directory. But the gate keys on a fixed list of built-in tool names, so developer tools and MCP tools run unapproved, and the approval API is not a complete gate. The dominant risk is the working directory: project configuration and instruction files are not integrity-protected, and AGENTS.md is auto-loaded into the system prompt. The shell tool runs on the host with the full environment.

Key gaps (3)

  1. execute_command runs model-generated commands on the host with the full environment. C4 · Code-execution isolation
  2. With developer and MCP tools ungated and unfenced, a prompt injection can exfiltrate data and take irreversible actions without a human in the default configuration. C5 · Untrusted input blast radius
  3. Plugins are executed in-process with every credential the agent holds. C7 · Third-party extensions

Criteria

C1 Identity & least privilege

Minimal 0.17 / 1.00

The framework runs with whatever credentials the developer's process holds. Provider API keys come from environment variables, and the built-in shell tool hands every child process a full copy of the environment, while the Python code tool and MCP servers get a scrubbed one. There is no per-tool or per-request authorization layer and no scoped identity; multi-user isolation is left to the application.

C2 Approval gates

Minimal 0.33 / 1.00

A bare Agent on an interactive terminal installs a console prompt that asks before each call to a fixed list of built-in dangerous tools (shell, code, file write/edit/delete, SQL, crawl), showing the arguments (truncated to about 100 characters) or a diff; off a terminal, shell/code/delete are hard-denied. The gate keys on tool names, so developer-written tools and MCP tools run without approval unless individually decorated. The `approval=True` API is not a complete gate, and environment variables (PRAISONAI_AUTO_APPROVE, PRAISONAI_TOOL_SAFETY=off) silently disable the gate. There is no default checkpoint or rollback.

C3 Tool & action scoping

Minimal 0.45 / 1.00

A bare Agent gets no tools, which is a good least-agency default. The bundled file tools resolve symlinks and confine paths to the working directory, and the web tools have SSRF checks. But the bundled shell tool accepts any program and arguments (with environment-variable expansion), and there is no central validation layer: developer tools and MCP tools receive whatever arguments the model produces.

C4 Code-execution isolation

Minimal 0.38 / 1.00

The built-in shell tool runs programs directly on the host as the user, with the full environment. The Python code tool is better by default: a separate process with an AST blocklist, a clean environment and resource limits, but that is filtering, not isolation, and selection of the execution mode is not tamper-resistant. Real sandboxes (Docker, E2B, Modal and others) exist through praisonai-sandbox and tools_run_on=, but are opt-in; the Docker one is a stock container with networking off by default.

C5 Untrusted input blast radius

Minimal 0.25 / 1.00

Results from a fixed list of web search and scrape tools are wrapped in an <external_tool_result> fence, which is a delimiter the model is asked to respect, not an enforced limit. MCP results and developer-tool results are not fenced, and nothing changes what the agent can do after it reads untrusted content. Because developer and MCP tools are ungated and the framework's normal use combines web input, private data and outbound tools, a successful injection can exfiltrate and act without a human.

C6 Memory, context & configuration integrity

Minimal 0.17 / 1.00

By default the Agent auto-loads instruction files (AGENTS.md, CLAUDE.md, .cursorrules and others) from the working directory into the system prompt without asking the user to trust the folder. Neither @imports nor project configuration files are integrity-protected. Long-term memory is off by default and its user_id namespace is optional.

C7 Third-party extensions

Minimal 0.25 / 1.00

MCP servers are launched from whatever command the developer writes (the README uses `npx -y`, which is unpinned) with no pinning or integrity check, though stdio servers do get a scrubbed environment. Project-local plugin files need an environment variable before they load, but the remaining plugin gates do not cover every path. Plugins run in-process with the agent's full access.

C8 Secrets & sensitive-data protection

Minimal 0.25 / 1.00

Keys come from environment variables. Telemetry is opt-in and content-free, and artifact storage and status output redact secrets. But the shell tool expands $VARIABLES in model-chosen arguments and passes the full environment to child processes, so the model can read any key; instruction-file import handling is not confined either.

C9 Audit & traceability

Minimal 0.35 / 1.00

Tool calls are logged only at debug level by default. A structured trace emitter records the start and end of each tool call with arguments and timing, but its default sink discards everything, so nothing is kept unless the developer wires one up. Approvals are not part of that trace.

C10 Limits & kill switch

Minimal 0.40 / 1.00

The agent loop caps iterations at 20 and tool calls per turn at 10 by default, the shell and code tools have per-call timeouts, and sub-agents spawned through the subagent tool are limited to depth 3. There is no default wall-clock limit or spend cap. The shell tool's timeout is a model-chosen argument with no ceiling.