Khoj

Khoj runs every tool with the server's own identity: one Postgres login (the database superuser in the shipped compose file), the operator's model and search API keys, and the server's network position. Built-in document, file and memory tools do scope their database queries to the requesting user, which is a real per-user check. Admin-configured MCP servers and the web reader are shared by all users and run with the server's full authority. The shipped docker-compose deployment's access-control and credential defaults are not locked down.

There is no human approval step anywhere in Khoj's tool path. The model picks tools (web search, arbitrary web page reads, sandboxed Python, file and note search, admin-configured MCP tools, and the opt-in computer operator) and they run immediately. In the default configuration the built-in tools are mostly read-only, so the main unattended effects are outbound requests to model-chosen URLs and automatic memory writes. Any mutating MCP tool an admin adds would also run with no confirmation.

Khoj's file tools are narrow: view, list and regex-search operate only on the requesting user's indexed documents in the database, and regex patterns are compiled before use. The web reader is the opposite: it fetches any model-chosen URL, and its handling of internal destinations is not a strict boundary. The prompt tells the model to read only one page, but the code reads every URL it is given. All tools are enabled by default, though custom agents can be limited to a subset.

Model-written Python never runs inside the Khoj server process: it is posted over HTTP to a separate sandbox container (the Terrarium service in the shipped compose) or to E2B if an E2B key is set. If no sandbox is configured, the code tool is hidden rather than falling back to the host, which fails closed. The compose service is a stock container with no hardening flags, no resource limits and no network restriction, and it sits on the same Docker network as the Postgres database. Terrarium's WebAssembly isolation is described upstream but lives in an external, unpinned image, so this review could only credit the container boundary.

Khoj reads untrusted web pages, search results, uploaded documents, MCP tool output and MCP tool descriptions, and in multi-user deployments other users' public agent personas. The only defenses are XML-style wrappers around tool results and an LLM 'safety' check on public agent personas that is not a strict boundary. A hijacked session holds the user's notes and memories and can send them out by asking the server to fetch an attacker URL; a further unattended egress path also exists. Nothing in default Khoj can take irreversible actions on external systems, so the worst case is unattended data leakage rather than destructive action.

Long-term memory is on by default. After every chat turn an LLM pulls 'facts' from the latest exchange, which can include text echoed from web pages, and saves them with no validation or user confirmation. Saved memories are injected into later chats, including tool selection, as a user-role message about the user. Memories are stored per user and every query filters by the requesting user, and users can list, edit and delete them through the memory API. The server has no workspace-loaded config files.

No third-party extension is enabled by default. An admin can add MCP servers in the admin panel; a bare package name is launched with npx and resolved to whatever version is current, with no pinning, hash check or re-approval when tool definitions change. Stdio servers run as child processes inside the Khoj server container as the same user. Only admins can add servers, and the repo has no workspace that could add them. Embedding models are downloaded from Hugging Face without trust_remote_code by default.

The shipped deployment's credential defaults are not locked down. Provider API keys are stored as plaintext database fields. There is no redaction anywhere: the file log handler always records DEBUG output, compose starts with -vv, and every conversation turn and every executed code snippet with its output is logged at INFO. Usage telemetry is on by default and sends content-free metadata (user UUID, host, referrer, command, agent slug) to a Khoj-run endpoint. Secrets are not routinely placed in model context.

Khoj stores each chat turn in its database with the user and agent, and keeps code and online-search context for completed turns. Research-mode tool calls, including MCP calls with their arguments, appear only as INFO log lines: the structured research and operator context is saved only when a turn is interrupted and dropped when it completes. Records are written by the server, outside anything the model can touch, but only at the end of a turn. There are no approvals to record, no actor chain beyond user and agent, and no tamper evidence.

Research mode stops after 5 iterations by default (KHOJ_RESEARCH_ITERATIONS), and individual tools have timeouts: 30 seconds for sandboxed code, 60 seconds for page fetches, and HTTP timeouts on model calls. There is no token or cost cap, and per-user chat rate limits are skipped whenever billing is not configured, which is the self-hosted default. A single iteration can fan out to any number of parallel tool calls. Stopping a chat cancels the asyncio task, and a code timeout calls the sandbox's stop endpoint. User-created scheduled automations keep running until deleted.