inspect_sandbox_tools

Imported from UKGovernmentBEIS/inspect_ai (src/inspect_sandbox_tools/AGENTS.md). Install upstream with npx skills add UKGovernmentBEIS/inspect_ai --skill inspect_sandbox_tools. Copyright stays with the author.

This package provides tool support for inspect_ai without requiring custom Docker images or Dockerfiles. It uses an executable injection approach to deploy tool functionality directly into running containers.

Stateful Tool Design Pattern

diagram

Some tools can be implemented without the need for any in-process state. For those tools, the tool code will be executed within the inspect-sandbox-tools process.

For tools that require the maintenance of state over the lifetime of a sandbox, this system marshals tool calls into a long running process via JSON RPC to a server process. That server then dispatches tool calls to tool specific @method handlers.

Each tool should have its own subdirectory that contains the following files:

• json_rpc_methods.py

  This module contains all of the JSON RPC @method functions — one for each tool (e.g. the web browser tool is actually a set of distinct tools). It is responsible for unpacking the JSON RPC request and forwarding the call to a transport-agnostic, strongly typed, stateful controller.

• tool_types.py

  This module includes the pydantic models representing the types for tool call parameters and results.

• controller.py

  This is transport-agnostic, strongly typed code that manages the tool specific in-process state and performs requested commands.

Architecture Overview

The inspect_sandbox_tools package is part of a split architecture that separates tool support into two independent systems:

• Legacy system (inspect_tool_support): Temporarily handles web browser functionality. Uses JSON-RPC communication but deploys code via Docker images built from Dockerfiles until the engineering to get Playwright included in the PyInstaller bundled executable works robustly.
• This system (inspect_sandbox_tools): Handles all other tools (bash_session, text_editor, MCP). Uses JSON-RPC communication with runtime executable injection for deployment.

Build Process

Linux executables are built via PyInstaller --onedir and packaged as a gzipped tar of the bundle tree (the launcher plus its _internal directory). At injection the tar is extracted into the container (tar xzf, with a host-side fallback to an uncompressed tar for containers whose tar lacks gzip support) so the launcher runs against the on-disk tree — nothing self-extracts per exec. Because libc is not bundled, the build base image's libc sets the runtime floor, so two variants are built per arch: a glibc variant (built against a conda-forge CPython at the glibc 2.17 floor, covering Ubuntu 16.04+ and other glibc distros from CentOS 7 forward) and a musl variant (built on alpine3.16 / musl 1.2.3, for Alpine/musl sandboxes). Injection detects the sandbox's libc and arch (recon.detect_sandbox_os) and selects the matching artifact. All four (arch × libc) are uploaded to S3; only the glibc pair is bundled into the wheel — musl is fetched from S3 on demand. Build scripts live in src/inspect_ai/tool/_sandbox_tools_utils/ and output to src/inspect_ai/binaries/. See RELEASING.md for build, validation, and release commands.

Container Injection Mechanism

When a tool needs to run in a container, the system automatically injects the appropriate executable:

1. Tool requests a sandbox via sandbox_with_injected_tools()
2. System checks for a trustworthy existing installation: /var/tmp/.da7be258e003d428 must be a real directory owned by the tools user with mode 0700, in a parent that other users cannot use to replace it, and must hold inspect-sandbox-tools as a regular file. The check runs as root (and confirms it really ran as uid 0) and falls back to the default user only when the sandbox cannot exec as root; whichever user the tools are found (or installed) under becomes the tools user. A merely readable launcher is not enough.
3. If missing, the injection process:
   • Detects container architecture (amd64/arm64) and libc (glibc/musl)
   • Selects the matching pre-built artifact from local binaries, S3, or a local Docker build
   • Creates /var/tmp/.da7be258e003d428 with mode 0700 as the tools user through the verified framework-directory helper (inspect_ai/util/_sandbox/_framework_directory.py). A pre-existing entry that is a symlink, not a directory, or owned by another uid fails injection with an error naming the path and the reason; it is never adopted or repaired. A root-owned directory whose mode is not 0700 fails the same way. In a rootless sandbox (tools user = default user, so the agent shares its uid) a directory that user owns is tightened to 0700 and reused instead, which is the shape older releases left behind (including on the host for the local sandbox).
   • Writes the gzipped onedir tar into the container and extracts it with the verified directory as the working directory, then re-verifies the directory immediately before starting the server from it.
   • A root-owned 0700 tree prevents access by other, non-root users in the sandbox; it is not a boundary against a process running in the sandbox as root.

The system includes fallback mechanisms to download executables from S3 or build them locally if needed.

RPC Communication

Tools communicate through a two-layer RPC architecture:

Layer 1 - Host to Container (stateless):

1. Tool creates JSON-RPC request on host
2. SandboxJSONRPCTransport executes: sandbox.exec(["/var/tmp/.da7be258e003d428/inspect-sandbox-tools", "exec"], input=json_rpc_request)
3. JSON-RPC payload passed via stdin to the injected executable
4. Response returns via stdout

Layer 2 - Container Internal (stateful operations):

1. When stateful execution is needed, the injected executable acts as a client
2. It starts a server process if not already running
3. Sends JSON-RPC requests to the server via HTTP over a Unix socket in the server's private state directory (.server/sandbox-tools.sock beside the launcher; INSPECT_SANDBOX_TOOLS_DIR overrides the location, as the local sandbox does)
4. Server maintains state across requests and returns responses
5. The stateless executable forwards the response back through Layer 1

Releasing

See RELEASING.md for the end-to-end process for building, publishing, and distributing new sandbox tools versions.

Testing

When running pytest with inspect to test interactions with this package, you may wish to test your local version of the inspect_tool_support code instead of the latest published package. Passing the flag --local-inspect-tools to pytest when running tests from test_inspect_container_tools.py will build and install the package from source, for example:

pytest tests/tools/test_inspect_container_tools.py --runslow --local-inspect-tools