{
  "schemaVersion": 1,
  "title": "harness.fail: what AI agent harnesses enforce, scored against abstract requirements",
  "description": "The capabilities an AI agent harness would need to enforce a file-access policy, as derived from the agentaccess.txt specification and written as abstract requirements, with each harness scored against them.",
  "creator": {
    "name": "Peter Seprus",
    "url": "https://github.com/ppseprus"
  },
  "homepage": "https://harness.fail/#matrix",
  "license": "CC-BY-4.0",
  "updated": "2026-10-06",
  "groups": [
    {
      "id": "A",
      "name": "Declarative surface",
      "question": "can the user state the policy?",
      "requirements": [
        {
          "id": "A1",
          "name": "Path rules",
          "description": "User-authorable file-access rules exist at all: an ignore file, deny globs, private-file patterns.",
          "spec": [
            {
              "label": "§1",
              "url": "https://agentaccesstxt.org/SPEC.html#1-motivation"
            }
          ]
        },
        {
          "id": "A2",
          "name": "Tree-resident",
          "description": "The rules live in the directory they govern and travel with it — clone the tree, keep the policy. Any directory can carry its own policy file, applied by the path being accessed; a single project-root file, or one chosen by where the session starts, is partial.",
          "spec": [
            {
              "label": "§4",
              "url": "https://agentaccesstxt.org/SPEC.html#4-file-name-and-discovery"
            }
          ]
        },
        {
          "id": "A3",
          "name": "Per-path",
          "description": "Rules distinguish paths by pattern, not only whole-project switches.",
          "spec": [
            {
              "label": "§5",
              "url": "https://agentaccesstxt.org/SPEC.html#5-syntax"
            }
          ]
        },
        {
          "id": "A4",
          "name": "Per-agent",
          "description": "Rules can name *which* tool they bind — \"tool X may work here, tool Y may not.\"",
          "spec": [
            {
              "label": "§6",
              "url": "https://agentaccesstxt.org/SPEC.html#6-evaluation-semantics"
            }
          ]
        },
        {
          "id": "A5",
          "name": "Shared grammar",
          "description": "The rules format is shared across tools rather than a proprietary dialect.",
          "spec": [
            {
              "label": "§8",
              "url": "https://agentaccesstxt.org/SPEC.html#8-relationship-to-existing-conventions"
            }
          ]
        }
      ]
    },
    {
      "id": "B",
      "name": "Coverage",
      "question": "what do the rules actually gate?",
      "requirements": [
        {
          "id": "B1",
          "name": "Context gating",
          "description": "Restricted content is kept out of model context — reads, indexing, attachments — not merely deprioritized.",
          "spec": [
            {
              "label": "§3",
              "url": "https://agentaccesstxt.org/SPEC.html#3-terminology"
            },
            {
              "label": "§7",
              "url": "https://agentaccesstxt.org/SPEC.html#7-conformance"
            }
          ]
        },
        {
          "id": "B2",
          "name": "Shell coverage",
          "description": "The restriction holds when the model reaches for the shell — `cat`, subprocesses, terminal tools.",
          "spec": [
            {
              "label": "§7",
              "url": "https://agentaccesstxt.org/SPEC.html#7-conformance"
            }
          ]
        },
        {
          "id": "B3",
          "name": "Write gating",
          "description": "Rules cover create/modify/delete, not only reads — a declarative do-not-touch.",
          "spec": [
            {
              "label": "§7",
              "url": "https://agentaccesstxt.org/SPEC.html#7-conformance"
            }
          ]
        },
        {
          "id": "B4",
          "name": "Pre-flight",
          "description": "Rules are evaluated *before* content is touched — at workspace open for ambient agents, before each operation otherwise — and changes take effect without a restart.",
          "spec": [
            {
              "label": "§4",
              "url": "https://agentaccesstxt.org/SPEC.html#4-file-name-and-discovery"
            }
          ]
        }
      ]
    },
    {
      "id": "C",
      "name": "Fail direction",
      "question": "which way do errors fail?",
      "requirements": [
        {
          "id": "C1",
          "name": "Deny precedence",
          "description": "A restriction holds against tool-native grants: a session approval, allowlist entry, remembered permission or bypass mode does not lift it.",
          "spec": [
            {
              "label": "§7",
              "url": "https://agentaccesstxt.org/SPEC.html#7-conformance"
            }
          ]
        },
        {
          "id": "C2",
          "name": "Fail-closed policy errors",
          "short": "Fail-closed errors",
          "description": "A malformed or invalid rule restricts rather than being silently dropped.",
          "spec": [
            {
              "label": "§5",
              "url": "https://agentaccesstxt.org/SPEC.html#5-syntax"
            }
          ]
        },
        {
          "id": "C3",
          "name": "Self-protection",
          "description": "The agent cannot create, modify, or delete the policy that restricts it.",
          "spec": [
            {
              "label": "§7",
              "url": "https://agentaccesstxt.org/SPEC.html#7-conformance"
            }
          ]
        },
        {
          "id": "C4",
          "name": "Policy as data",
          "description": "The policy is evaluated in deterministic code and never enters model context as natural language — a policy the model can read is a prompt-injection surface.",
          "spec": [
            {
              "label": "§9",
              "url": "https://agentaccesstxt.org/SPEC.html#9-security-considerations"
            }
          ]
        }
      ]
    },
    {
      "id": "D",
      "name": "Enforcement quality",
      "question": "does the mechanism hold?",
      "requirements": [
        {
          "id": "D1",
          "name": "Harness enforcement",
          "description": "Restrictions are enforced in deterministic harness code, never entrusted to the model's judgment or a system-prompt request.",
          "spec": [
            {
              "label": "§9",
              "url": "https://agentaccesstxt.org/SPEC.html#9-security-considerations"
            }
          ]
        },
        {
          "id": "D2",
          "name": "Canonicalization",
          "description": "Symlinks resolved and paths canonicalized before the policy check — a recurring class of enforcement bug.",
          "spec": [
            {
              "label": "§5",
              "url": "https://agentaccesstxt.org/SPEC.html#5-syntax"
            },
            {
              "label": "§9",
              "url": "https://agentaccesstxt.org/SPEC.html#9-security-considerations"
            },
            {
              "label": "A.6",
              "url": "https://agentaccesstxt.org/SPEC.html#a6-enforcement-bugs-in-otherwise-honest-tools"
            }
          ]
        },
        {
          "id": "D3",
          "name": "Deliberate override",
          "description": "A restriction is lifted only by an explicit user action naming that restriction — never by a blanket bypass mode, and never by consent the model parsed out of its own context.",
          "spec": [
            {
              "label": "§7",
              "url": "https://agentaccesstxt.org/SPEC.html#7-conformance"
            }
          ]
        },
        {
          "id": "D4",
          "name": "Delegation",
          "description": "Restrictions carry to subagents and spawned processes; they attenuate through delegation, never reset.",
          "spec": [
            {
              "label": "§7",
              "url": "https://agentaccesstxt.org/SPEC.html#7-conformance"
            }
          ]
        }
      ]
    }
  ],
  "harnesses": [
    {
      "id": "aider",
      "name": "Aider",
      "marks": {
        "A1": "y",
        "A2": "p",
        "A3": "y",
        "A4": "n",
        "A5": "p",
        "B1": "p",
        "B2": "u",
        "B3": "u",
        "B4": "u",
        "C1": "u",
        "C2": "u",
        "C3": "u",
        "C4": "u",
        "D1": "u",
        "D2": "u",
        "D3": "u",
        "D4": "u"
      },
      "note": "`.aiderignore`, `/read-only`, `--add-gitignore-files` — long-standing, gitignore syntax ([FAQ](https://aider.chat/docs/faq.html), [options](https://aider.chat/docs/config/options.html), [commands](https://aider.chat/docs/usage/commands.html)) (A1/A3 ●, A5 ◐); the ignore file is one per repository, \"default: .aiderignore in git root\" (A2 ◐). Aider's workflow is explicit-add — files enter context when the user adds them — which gates context by construction but is a workflow property, not an enforcement claim (B1 ◐). Enforcement locus and shell coverage undocumented."
    },
    {
      "id": "claude-code",
      "name": "Claude Code",
      "marks": {
        "A1": "y",
        "A2": "p",
        "A3": "y",
        "A4": "n",
        "A5": "n",
        "B1": "p",
        "B2": "p",
        "B3": "y",
        "B4": "y",
        "C1": "y",
        "C2": "p",
        "C3": "p",
        "C4": "y",
        "D1": "y",
        "D2": "p",
        "D3": "p",
        "D4": "p"
      },
      "note": "`permissions.deny` with `Read()`/`Edit()` path rules ([permissions docs](https://code.claude.com/docs/en/permissions)); project settings can be committed with the repo (`.claude/settings.json`), but Claude Code reads that one file \"from the session's primary working directory\" ([settings](https://code.claude.com/docs/en/settings)), with no per-directory files (A2 ◐). Vendor states enforcement is deterministic: \"Permission rules are enforced by Claude Code, not by the model\" (D1 ●), with documented precedence \"deny, then ask, then allow\". A `Read` deny also blocks Edit and Write on the same path (B3 ●), evaluated before the tool call executes, and edits to `permissions` reach \"the running session without a restart\" (B4 ●) — but for context surfaces (Grep, Glob, `@`-mentions) the vendor's own wording is a \"best-effort attempt\", and NotebookEdit isn't covered (B1 ◐). Shell coverage is explicitly partial: recognized file commands only, rules \"don't apply … to arbitrary subprocesses\"; the opt-in [OS sandbox](https://code.claude.com/docs/en/sandboxing) closes that, with a default-enabled but prompt-gated unsandboxed-retry escape hatch (B2 ◐). Invalid managed sandbox values fail closed per the [changelog](https://raw.githubusercontent.com/anthropics/claude-code/main/CHANGELOG.md) (2.1.283), and an unparseable managed settings file stops every session; one the OS denies reading starts the session without its policies (2.1.285), and malformed `mcp__` parameter rules are skipped with a warning (C2 ◐). Scoped rules \"don't change which tools Claude sees. Claude Code checks them when Claude attempts a call, leaving the prefix intact\", and a bare tool-name deny removes the tool rather than describing it ([prompt caching](https://code.claude.com/docs/en/prompt-caching#denying-an-entire-tool)) (C4 ●). [CVE-2025-54794](https://github.com/anthropics/claude-code/security/advisories/GHSA-pmw4-pwvc-3hx2) was a prefix-matching-instead-of-canonicalization bug, fixed; the permissions docs now check both the requested path and the file a symlink resolves to, and symlink bypasses of `Read` deny rules through @-mentions and IDE selections were fixed in 2.1.289–2.1.290 (D2 ◐). [Permission modes](https://code.claude.com/docs/en/permission-modes) state that \"Deny rules block in every mode, including bypassPermissions\" — the blanket bypass lifts prompts, not deny rules (C1 ●) — and writes to protected paths, `.claude` settings among them, cannot be pre-approved by allow rules, yet in auto mode (the starting mode since 2.1.284) a model classifier decides them, and bypassPermissions allows them outright (C3 ◐). That protected-path restriction is the D3 gap: user deny rules lift only when the user edits them, but the built-in restriction falls to a blanket mode and, in auto mode, to a classifier model's judgment (D3 ◐). Subagent docs cover the delegation interplay partially — a subagent cannot self-declare bypassPermissions, and background subagents surface permission prompts to the main session — and state that built-in subagents inherit the parent conversation's permission rules, with no such statement for custom ones (D4 ◐)."
    },
    {
      "id": "cline",
      "name": "Cline",
      "marks": {
        "A1": "y",
        "A2": "p",
        "A3": "y",
        "A4": "n",
        "A5": "p",
        "B1": "n",
        "B2": "n",
        "B3": "u",
        "B4": "u",
        "C1": "u",
        "C2": "u",
        "C3": "u",
        "C4": "u",
        "D1": "p",
        "D2": "u",
        "D3": "u",
        "D4": "u"
      },
      "note": "`.clineignore`, gitignore syntax, one file \"in your project root\" (A1/A3 ●, A2 ◐, A5 ◐) — and the [docs](https://docs.cline.bot/customization/clineignore) state plainly that it \"is not a security or access-control boundary — ignored files can still be read via explicit @ mentions or shell commands\": it controls automatic loading, not access (B1 ○, B2 ○). The page carries a deprecation banner and offers a PreToolUse hook script as the enforced replacement, which \"actively blocks the tool call\" when a read, edit, or shell command targets an ignored file: opt-in, user-installed code that does not resolve symlinks and is disabled in `--yolo` mode (D1 ◐). Writes and the rest of groups C–D are undocumented (?)."
    },
    {
      "id": "cursor",
      "name": "Cursor",
      "marks": {
        "A1": "y",
        "A2": "p",
        "A3": "y",
        "A4": "n",
        "A5": "p",
        "B1": "p",
        "B2": "n",
        "B3": "u",
        "B4": "u",
        "C1": "u",
        "C2": "p",
        "C3": "u",
        "C4": "u",
        "D1": "p",
        "D2": "p",
        "D3": "u",
        "D4": "u"
      },
      "note": "`.cursorignore`, gitignore syntax, one file \"in your root directory\", with an opt-in setting that searches parent directories, not subdirectories ([ignore docs](https://cursor.com/docs/reference/ignore-file)) (A1/A3 ●, A2 ◐, A5 ◐). The docs disclaim completeness — \"complete protection isn't guaranteed due to LLM unpredictability\" — and state plainly that agent terminal and MCP tools \"cannot block access\" to ignored code (B1 ◐, B2 ○). Whether the rules gate writes, and when rule changes take effect, is undocumented (B3/B4 ?). [CVE-2025-64110](https://nvd.nist.gov/vuln/detail/cve-2025-64110) was an ignore-file fail-open (a new `.cursorignore` invalidated existing protections), fixed in 2.0 (C2 ◐). [CVE-2026-50548](https://nvd.nist.gov/vuln/detail/CVE-2026-50548) and [CVE-2026-50549](https://nvd.nist.gov/vuln/detail/CVE-2026-50549) (Cato Networks) let the agent write files outside the workspace through a sandbox write-path override and a failed-canonicalization fallback through an in-workspace symlink; both fixed in Cursor 3.0 per the NVD records (D2 ◐). \"Cursor blocks access to files listed in `.cursorignore`\", under the same no-guarantee caveat (D1 ◐)."
    },
    {
      "id": "deepseek-harness",
      "name": "DeepSeek Harness",
      "marks": {
        "A1": "n",
        "A2": "n",
        "A3": "n",
        "A4": "n",
        "A5": "n",
        "B1": "u",
        "B2": "p",
        "B3": "p",
        "B4": "u",
        "C1": "n",
        "C2": "u",
        "C3": "p",
        "C4": "u",
        "D1": "p",
        "D2": "p",
        "D3": "n",
        "D4": "p"
      },
      "note": "A named per-session abstraction — executors run \"under the session file policy\" ([v0.1.6-alpha.1](https://github.com/deepseek-ai/deepseek-harness/releases/tag/dsh-v0.1.6-alpha.1), carried into [v0.1.7-rc.1](https://github.com/deepseek-ai/deepseek-harness/releases/tag/dsh-v0.1.7-rc.1)) — documented as one of three sandbox modes (`read-only`, `workspace-write`, `danger-full-access`) over a workspace root, with no path patterns ([sandbox docs](https://github.com/deepseek-ai/deepseek-harness/blob/master/docs/subsystems/sandbox.md)) (A1/A2/A3 ○, B2/D4 ◐). An approved escalated retry \"is a new call with a wider policy\" (C1 ○), and `danger-full-access`, which \"bypasses confinement\", ships in the default preset table ([permission presets](https://github.com/deepseek-ai/deepseek-harness/blob/master/docs/subsystems/permission-presets.md)) (D3 ○). Enforcement fixes track the classic classes: POSIX symlink/parent-directory combinations, Windows drive-relative paths ([v0.1.6-alpha.1](https://github.com/deepseek-ai/deepseek-harness/releases/tag/dsh-v0.1.6-alpha.1)), cross-workspace deletion escapes ([v0.1.7-alpha.1](https://github.com/deepseek-ai/deepseek-harness/releases/tag/dsh-v0.1.7-alpha.1)) (B3 ◐, D2 ◐). [CVE-2026-82533](https://nvd.nist.gov/vuln/detail/CVE-2026-82533) (OX Research): the sandboxed agent could reach the harness's own unauthenticated control interface and elevate its session to `danger-full-access` — the enforcement layer operable by the thing it governs; fixed August 27, 2026 in 0.1.2-alpha.1, whose [release notes](https://github.com/deepseek-ai/deepseek-harness/releases/tag/dsh-v0.1.2-alpha.1) require a one-time token for network access to the web interface (C3 ◐: the bar exists now, its guarantee is undocumented). [SAFETY.md](https://github.com/deepseek-ai/deepseek-harness/blob/master/SAFETY.md) states plainly that sandboxing, approval prompts, and permission controls \"do not guarantee isolation or prevent damage\" (D1 ◐)."
    },
    {
      "id": "gemini-cli",
      "name": "Gemini CLI",
      "marks": {
        "A1": "y",
        "A2": "p",
        "A3": "y",
        "A4": "n",
        "A5": "p",
        "B1": "y",
        "B2": "p",
        "B3": "p",
        "B4": "p",
        "C1": "u",
        "C2": "n",
        "C3": "n",
        "C4": "u",
        "D1": "y",
        "D2": "p",
        "D3": "u",
        "D4": "u"
      },
      "note": "`.geminiignore` plus `.gitignore` reuse ([docs](https://github.com/google-gemini/gemini-cli/blob/main/docs/cli/gemini-ignore.md)) (A1/A3 ●, A5 ◐); `.geminiignore` is one file \"in the root of your project directory\" — nested `.gitignore` files are honored per directory ([source](https://github.com/google-gemini/gemini-cli/blob/main/packages/core/src/utils/gitIgnoreParser.ts)), but that is the version-control file, not the harness's policy file (A2 ◐). `read_file` enforces deterministically via `shouldIgnoreFile()` (B1/D1 ●) — and a feature request on the project's tracker names `cat` via `run_shell_command` as the workaround for reading ignored files ([#13775](https://github.com/google-gemini/gemini-cli/issues/13775)); only the opt-in tool sandbox, off by default, turns ignored paths into kernel-denied paths for shell commands ([source](https://github.com/google-gemini/gemini-cli/blob/main/packages/core/src/config/config.ts)) (B2 ◐). Changes require a restart to take effect, per the same docs (B4 ◐). Negation handling has shipped a bug that errs restrictive (`!` rules fail to un-ignore, [#5444](https://github.com/google-gemini/gemini-cli/issues/5444), closed by the stale bot without a fix), but the error paths fail open: the matcher library skips a pattern ending in an unescaped backslash without a warning ([node-ignore](https://github.com/kaelzhang/node-ignore/blob/7.0.0/index.js)), an unreadable `.geminiignore` loads as no rules ([source](https://github.com/google-gemini/gemini-cli/blob/main/packages/core/src/utils/ignoreFileParser.ts)), and the ignore check returns not-ignored on any exception ([source](https://github.com/google-gemini/gemini-cli/blob/main/packages/core/src/utils/gitIgnoreParser.ts)) (C2 ○). The write tools carry no ignore check at all — `write_file` and `edit` never consult the ignore service, verifiable in [source](https://github.com/google-gemini/gemini-cli/blob/main/packages/core/src/tools/write-file.ts); the opt-in sandbox denies only shell writes to ignored paths (B3 ◐). Nothing protects `.geminiignore` itself: \"If the workspace is writable, we allow editing .gitignore and .geminiignore by default\" ([sandbox source](https://github.com/google-gemini/gemini-cli/blob/main/packages/core/src/sandbox/linux/bwrapArgsBuilder.ts)) (C3 ○). `read_file` checks the ignore rules against both the requested path and its resolved real path ([source](https://github.com/google-gemini/gemini-cli/blob/main/packages/core/src/tools/read-file.ts)); other surfaces are undocumented (D2 ◐)."
    },
    {
      "id": "github-copilot-agent-surfaces",
      "name": "GitHub Copilot (agent surfaces)",
      "marks": {
        "A1": "y",
        "A2": "n",
        "A3": "y",
        "A4": "n",
        "A5": "n",
        "B1": "p",
        "B2": "p",
        "B3": "p",
        "B4": "p",
        "C1": "p",
        "C2": "u",
        "C3": "p",
        "C4": "u",
        "D1": "p",
        "D2": "p",
        "D3": "u",
        "D4": "u"
      },
      "note": "[Content exclusion](https://docs.github.com/en/copilot/concepts/security-governance-and-network-settings/content-exclusion) exists and is deterministic for completions, chat, and review — and the docs exclude both agent surfaces by name: \"GitHub Copilot CLI and Agent mode in Copilot Chat in IDEs do not support content exclusion\" ([configuring exclusions](https://docs.github.com/en/copilot/how-tos/configure-content-exclusion/exclude-content-from-copilot)). The path rules that do reach an agent surface are Copilot CLI's own: `Read(...)`, `Edit(...)`, and `Write(...)` glob rules in enterprise managed settings, session `--deny-tool` patterns such as `read(.env)` and `write(PATH)`, and `deniedPaths` for sandboxed commands ([configuration reference](https://docs.github.com/en/copilot/reference/copilot-cli-reference/cli-config-dir-reference), [command reference](https://docs.github.com/en/copilot/reference/copilot-cli-reference/cli-command-reference#tool-permission-patterns)) (A1/A3 ●); agent mode in IDEs documents no equivalent. None of them lives in the tree: the repository settings file `.github/copilot/settings.json` supports only a listed set of keys, permission rules not among them, and content exclusion is server-delivered (A2 ○). Coverage is CLI-only: `Read` rules gate file reads, `Edit`/`Write` rules also cover a recognized set of shell redirections and in-place `sed` edits, but when the target \"comes from a variable\" the path rules \"do not apply\" (B1/B2/B3 ◐); rules are checked per request, and managed settings re-apply hourly \"without restarting the session\" (B4 ◐). A `write(PATH)` match \"resolves symlinks and . / .. segments\", while content exclusion documents symlinks and remote filesystems as not covered, with up to 30-minute propagation delay (D2 ◐). The CLI's rules are enforced by the harness, but the restriction an organization declares through content exclusion is not enforced on either agent surface (D1 ◐). File-based managed settings are rejected when they are \"symlinks, not owned by root, or world-writable\"; the docs state no such guard for the user-level settings file (C3 ◐). [v1.0.88](https://github.com/github/copilot-cli/releases/tag/v1.0.88) closed a management-layer fail-open where ACP, AHP-host, and `--server` sessions \"previously ran with no managed MCP, permission, or plugin policy\". Copilot CLI's own deny rules hold against grants — \"Deny rules always take precedence over allow rules, even when `--allow-all` is set or a matching approval has been saved\" ([allowing tools](https://docs.github.com/en/copilot/how-tos/copilot-cli/use-copilot-cli/allowing-tools)) — while agent mode in IDEs documents no equivalent (C1 ◐)."
    },
    {
      "id": "goose",
      "name": "goose",
      "marks": {
        "A1": "n",
        "A2": "n",
        "A3": "n",
        "A4": "n",
        "A5": "n",
        "B1": "u",
        "B2": "u",
        "B3": "u",
        "B4": "u",
        "C1": "p",
        "C2": "u",
        "C3": "u",
        "C4": "u",
        "D1": "p",
        "D2": "u",
        "D3": "u",
        "D4": "u"
      },
      "note": "`.gooseignore` is gone from the current tree: the docs carry no page for it, the string survives only in the repo's own `.gitignore`, and [v1.44.0](https://github.com/aaif-goose/goose/releases/tag/v1.44.0)'s \"Remove stale gooseignore references\" was the cleanup — current permissions are per-tool (allow / ask / deny), not per-path, so no declarative path-rule surface exists today (A1/A2/A3 ○). v1.49.0 shipped \"Give permission denies precedence\" ([#11477](https://github.com/aaif-goose/goose/pull/11477)), so a saved `never_allow` now beats an overlapping `always_allow` — but Auto mode approves every tool without consulting it ([permission inspector](https://github.com/aaif-goose/goose/blob/main/crates/goose/src/permission/permission_inspector.rs)) (C1 ◐), and hooks carry a documented deny contract ([v1.44.0](https://github.com/aaif-goose/goose/releases/tag/v1.44.0)), though a policy hook that fails lets the call through unless set to `on_failure: block` ([hooks docs](https://github.com/aaif-goose/goose/blob/104ddde585112af6f074f912f3ceb405fb5b800e/documentation/docs/guides/context-engineering/hooks.md)) (D1 ◐). What the per-tool permissions gate operationally is undocumented (B1–B4 ?)."
    },
    {
      "id": "jetbrains-ai-assistant-junie",
      "name": "JetBrains AI Assistant / Junie",
      "marks": {
        "A1": "y",
        "A2": "p",
        "A3": "y",
        "A4": "p",
        "A5": "p",
        "B1": "p",
        "B2": "n",
        "B3": "p",
        "B4": "u",
        "C1": "u",
        "C2": "u",
        "C3": "u",
        "C4": "u",
        "D1": "p",
        "D2": "u",
        "D3": "n",
        "D4": "u"
      },
      "note": "`.aiignore`, honoring `.cursorignore`/`.codeiumignore`/`.aiexclude` \"as long as they are located in the root folder of the project\" (A5 ◐), plus `.noai` \"in the root directory of the project\" as a whole-project kill switch — a vendor-specific marker, per-agent in the binary, single-vendor sense (A4 ◐) ([docs](https://www.jetbrains.com/help/ai-assistant/disable-ai-assistant.html)); the documented locations are the project root, not per directory (A2 ◐). Enforcement is vendor-admitted best-effort: \"ignored files may still be processed due to unforeseen issues\" ([docs](https://www.jetbrains.com/help/ai-assistant/disable-ai-assistant.html)), and file names and paths stay visible ([support KB](https://youtrack.jetbrains.com/articles/SUPPORT-A-3359)) (B1/D1 ◐). Allowlisted commands skip `.aiignore` checks entirely (B2 ○), and Brave Mode bypasses the approval prompts wholesale (D3 ○). Edits to ignored files are approval-gated rather than hard-blocked — Junie \"will ask for explicit approval before viewing or editing the contents\" (B3 ◐)."
    },
    {
      "id": "openai-codex-cli",
      "name": "OpenAI Codex CLI",
      "marks": {
        "A1": "p",
        "A2": "p",
        "A3": "p",
        "A4": "n",
        "A5": "n",
        "B1": "p",
        "B2": "y",
        "B3": "y",
        "B4": "p",
        "C1": "p",
        "C2": "p",
        "C3": "p",
        "C4": "p",
        "D1": "y",
        "D2": "p",
        "D3": "p",
        "D4": "y"
      },
      "note": "The long-standing read-restriction gap has begun closing: beta [permission profiles](https://learn.chatgpt.com/docs/permissions) add per-path filesystem rules — glob `deny` entries that deny \"both reads and writes\" — configurable in project `.codex/config.toml` files, which Codex loads \"from the project root to your current working directory\", closest wins, trusted projects only ([advanced config](https://learn.chatgpt.com/docs/config-file/config-advanced)): the files travel with the repo, but which ones apply depends on where the session starts, not on the path being accessed (A1/A2 ◐, A3 ◐, B1 ◐: beta, config-based, sandbox-enforced; the `.codexignore` request [#1397](https://github.com/openai/codex/issues/1397) was closed into #2847, itself closed completed June 2026 with this feature as the answer). The profile semantics lean fail-closed: deny takes precedence over write and write over read, and \"missing or empty filesystem tables keep filesystem access restricted\" with a startup warning (C2 ◐); the `:workspace` profile keeps the workspace `.codex` directory read-only by default (C3 ◐); the deny list is enforced in sandbox code, but the harness also writes its paths and globs into model context as a developer message ([source](https://github.com/openai/codex/blob/4d15794336668d6098c0e36eb8e96cbbbfdb1d2d/codex-rs/prompts/src/permissions_instructions.rs#L382-L396)) (C4 ◐). The enforcement substrate is an always-on OS sandbox (macOS Seatbelt, Linux bubblewrap; [docs](https://learn.chatgpt.com/docs/sandboxing), [independently investigated](https://simonwillison.net/2025/Nov/9/codex-sandbox-investigation/)) confining writes and network at kernel level, with spawned commands inheriting the same boundaries (B2/B3 ●, D1 ●, D4 ●); whether an edited profile reaches a running session without a restart is undocumented (B4 ◐). On Linux, literal deny paths are kept alongside their resolved targets so denials hold across writable symlinks ([#48155](https://github.com/openai/codex/pull/48155)) (D2 ◐). Approved escalations run unsandboxed and a `danger-full-access` mode exists (D3 ◐); approved commands \"retain explicit filesystem denials\" ([rust-v0.159.0](https://github.com/openai/codex/releases/tag/rust-v0.159.0)), but the `:danger-full-access` profile \"removes local sandbox restrictions\" ([permissions](https://learn.chatgpt.com/docs/permissions)) (C1 ◐)."
    },
    {
      "id": "pi",
      "name": "Pi",
      "marks": {
        "A1": "n",
        "A2": "n",
        "A3": "n",
        "A4": "n",
        "A5": "n",
        "B1": "n",
        "B2": "n",
        "B3": "n",
        "B4": "n",
        "C1": "na",
        "C2": "na",
        "C3": "na",
        "C4": "na",
        "D1": "p",
        "D2": "na",
        "D3": "na",
        "D4": "n"
      },
      "note": "Ships no file-access rules: an issue on its tracker lists the only mechanisms as the coarse `--tools` flag, custom TypeScript hooks, and interactive confirmation ([#4459](https://github.com/earendil-works/pi/issues/4459), which proposed native allow/deny policy files — auto-closed during a refactor without review) (A1–B4 ○, C1–C4 —). The [security doc](https://github.com/earendil-works/pi/blob/main/packages/coding-agent/docs/security.md) places safety outside the harness — it \"comes from limiting the files, credentials, processes, and network services Pi can access\" — and spells out the consequence: the working folder \"does not prevent commands from accessing other paths available to the Pi process,\" project trust \"does not limit what tool calls can access,\" and child processes \"run with those same permissions\" (D4 ○). What Pi does have is a deterministic enforcement point: the `tool_call` hook can block operations in host-side code before execution, and the shipped [permission-gate example](https://github.com/earendil-works/pi/blob/main/packages/coding-agent/examples/extensions/permission-gate.ts) blocks by default when no UI is present — a policy layer could attach there, but enforcement itself is user-written code, not a shipped feature (D1 ◐)."
    },
    {
      "id": "windsurf-devin",
      "name": "Windsurf / Devin",
      "marks": {
        "A1": "y",
        "A2": "p",
        "A3": "y",
        "A4": "n",
        "A5": "p",
        "B1": "y",
        "B2": "p",
        "B3": "y",
        "B4": "u",
        "C1": "p",
        "C2": "p",
        "C3": "u",
        "C4": "u",
        "D1": "y",
        "D2": "p",
        "D3": "u",
        "D4": "p"
      },
      "note": "`.devinignore` (with legacy `.windsurfignore` / `.codeiumignore` \"still read and enforced alongside\" it), gitignore syntax, one file at \"the root of your repository\": paths matched \"are excluded from indexing and cannot be viewed, edited, or created by Devin\" ([docs](https://docs.devin.ai/desktop/context-awareness/devin-ignore)) (A1/A3 ●, A2 ◐, A5 ◐, B1/B3 ●). Since v3.9.19 removed Cascade ([changelog](https://docs.devin.ai/desktop/changelog)), [Devin Local](https://docs.devin.ai/desktop/devin-local) is the only agent in Devin Desktop, and it shares Devin CLI's [permission rules](https://docs.devin.ai/cli/reference/permissions.md), evaluated in a fixed order where \"A deny rule always wins\" (C1 ◐: only organization-level rules are guaranteed against Bypass mode). Smart mode's model judgment \"only applies where no rule already decides the call, so a `deny` rule blocks the action\"; the `.devinignore` docs do not name where it is enforced (D1 ●). An opt-in [OS sandbox](https://docs.devin.ai/cli/sandbox.md) hides `Read(...)`-denied paths from shell commands (B2 ◐) and errs restrictive: unsupported exclusion rules are \"ignored with a warning\", and a sandbox that cannot be resolved refuses to start (C2 ◐). The write tools \"refuse to write through a symlink\" (v3.6.27), with reads unstated (D2 ◐), and [background subagents](https://docs.devin.ai/cli/subagents.md) get only tools already approved in the session (D4 ◐)."
    },
    {
      "id": "zed",
      "name": "Zed",
      "marks": {
        "A1": "y",
        "A2": "y",
        "A3": "y",
        "A4": "n",
        "A5": "n",
        "B1": "y",
        "B2": "n",
        "B3": "y",
        "B4": "p",
        "C1": "y",
        "C2": "p",
        "C3": "p",
        "C4": "u",
        "D1": "y",
        "D2": "p",
        "D3": "u",
        "D4": "u"
      },
      "note": "`private_files` and `read_only_files` glob settings, project-resident ([configuration](https://zed.dev/docs/reference/all-settings), [tool permissions](https://zed.dev/docs/ai/tool-permissions)); project settings files can also sit in subdirectories ([configuring Zed](https://github.com/zed-industries/zed/blob/main/docs/src/configuring-zed.md)), and the agent's file tools look the settings up for the path being accessed ([source](https://github.com/zed-industries/zed/blob/main/crates/agent/src/tools/read_file_tool.rs)) (A1/A2/A3 ●, B1/B3 ●). v1.21.0 added `\"...\"` inheritance-extension semantics for `read_only_files`, and v1.22.0 spread them to more glob settings. The terminal tool is governed by command-pattern rules, not path rules, and under the new [OS sandbox](https://zed.dev/docs/ai/sandboxing) terminal commands \"can read most of the filesystem\" (B2 ○). v1.22.0 stopped one invalid pattern from discarding the valid exclusion/read-only/private rules beside it, which narrows a fail-open in their own dialect rather than closing it: a matcher that cannot be built still matches nothing ([#64525](https://github.com/zed-industries/zed/pull/64525)) (C2 ◐). `always_deny` rules are \"highest priority, cannot be overridden\" and still block when every tool action is auto-approved (C1 ●); agent edits inside `.zed/`, where project `private_files` live, always prompt ([source](https://github.com/zed-industries/zed/blob/main/crates/agent/src/tools/tool_permissions.rs)), but sandboxed terminal commands can still write project files (C3 ◐). Enforcement is deterministic editor code, confirmed by the project's own advisory for [CVE-2026-27967](https://github.com/zed-industries/zed/security/advisories/GHSA-786m-x2vc-5235) — agent file tools checked `private_files` on logical paths, so symlinks escaped; patched 0.225.9 (D1 ●, D2 ◐)."
    }
  ],
  "findings": "Three columns carry the headline. **A4 (per-agent) has no ●**: no harness lets a directory say which tools are welcome; the closest is JetBrains’ `.noai`, a single-vendor kill switch. **A5 (shared grammar) has no ●**: every rules format is a proprietary dialect, at best borrowing `.gitignore` syntax, and rules travel between tools only where one vendor chooses to read another’s file. **B2 (shell coverage) has a single ●**: only OpenAI Codex CLI’s always-on kernel sandbox closes it by default; five harnesses close it only partly, mostly with an opt-in sandbox, and five not at all. The {?} marks are a finding too: for Aider, goose and Cline most of these questions have no documented answer, and more than a quarter of all cells are {?}.",
  "history": [
    {
      "date": "2026-10-06",
      "label": "initial scoring",
      "text": "thirteen harnesses against seventeen requirements; every sourced cell checked against its live primary source."
    }
  ]
}