Skip to content

Record denied Bash commands for later inspection - #322

Open
zwanski-start wants to merge 1 commit into
AutoForgeAI:masterfrom
zwanski-start:swarm/issue-73-dennis
Open

zwanski-start wants to merge 1 commit into
AutoForgeAI:masterfrom
zwanski-start:swarm/issue-73-dennis

Conversation

@zwanski-start

Copy link
Copy Markdown

Closes #73

What changed

  • security.py: bash_security_hook now records every blocked Bash command (not on the allowlist, on the hard/org blocklist, failed extra validation for pkill/chmod/init.sh/playwright-cli, or couldn't be parsed) to {project}/.autoforge/denied_commands.jsonl. Each line has timestamp, command (the specific command that caused the block, e.g. pytest), full_command (truncated to 2000 chars) and reason (first line). The validation logic moved unchanged into _validate_bash_command, which also returns the offending command name. Recording errors are logged and ignored, so they never change the allow/block decision.
  • New helpers: record_denied_command, load_denied_commands, summarize_denied_commands (aggregates by command: count, last seen, most recent example), clear_denied_commands.
  • autoforge_paths.py: get_denied_commands_log_path(), and denied_commands.jsonl / .1 added to the generated .autoforge/.gitignore.
  • API: GET /api/projects/{name}/denied-commands returns the aggregated summary and the 100 most recent entries. DELETE on the same URL clears the log.
  • Docs: new "Reviewing Denied Commands" section in examples/README.md (includes a jq one-liner), plus CLAUDE.md updates.

Verification

  • python test_security.py: 182 passed, including 13 new checks in test_denied_commands_logging covering recording, aggregation, no-project-dir, malformed lines, rotation and clearing.
  • python test_security_integration.py: all passed.
  • ruff check . is clean. mypy . reports no new errors (24 before and after, none in the changed files).
  • pytest test_client.py test_ws_security.py test_devserver_security.py gives the same 21 failures on unmodified master, so they are not caused by this change.
  • Manual test: registered a temp project, ran the hook with pytest -x, openssl genrsa 2048, pytest and ls, then called the endpoints through FastAPI's TestClient. GET returned pytest ×2 and openssl ×1 (ls was not logged), DELETE cleared the log, and an unknown project returned 404.

Assumptions / notes

  • The log is a plain JSONL file rather than a DB table, so it can be read with cat/jq. Parallel agents append single short lines, which is safe enough on POSIX.
  • The file rotates once (.1) above about 1 MB, so it can't grow without limit.
  • full_command is stored as-is. It could contain secrets the agent typed (for example a token in a curl command). It stays in the project's gitignored .autoforge/ directory, the same trust level as the rest of the agent's local state.
  • This PR adds no UI. The REST endpoint is in place, so a viewer in the settings or debug panel can be a follow-up.

🤖 Generated with Claude Code

The security hook now appends every blocked Bash command to
.autoforge/denied_commands.jsonl (timestamp, offending command, full
command line, reason) so users can see which tools the agent wanted and
extend their allowlists. The log is gitignored and rotates at ~1 MB.

Adds GET/DELETE /api/projects/{name}/denied-commands to view the log
aggregated by command and to clear it, plus tests and docs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zwanski-start

Copy link
Copy Markdown
Author

🔍 QA report: ✅ Passed

Tester: Guido (cubefarm agent, in a fresh session) · Round: 1 · Author: Dennis

The PR records every Bash command the security hook blocks to .autoforge/denied_commands.jsonl and adds GET/DELETE /api/projects/{name}/denied-commands, which meets issue #73. The allow/block logic is unchanged: all security unit and integration tests pass, and recording errors never change the decision. Tested against a real uvicorn server with an isolated registry: aggregation, recent entries, clearing, 404/400 errors, rotation and skipping of non-JSON lines all work. Only minor non-blocking nits found: the GET returns 500 if a hand-edited log line is valid JSON with wrong field types, and the per-command name field is not truncated.

Check Details
✅ Issue repro: blocked commands such as pytest and openssl are recorded for later inspection Ran bash_security_hook with tool_name=Bash and project_dir for 'pytest -x', 'ls && openssl genrsa 2048', 'pytest', 'ls' and 'pip install requests'. The JSONL entries had the correct command, full_command, timestamp and first-line reason, and allowed 'ls' was not recorded.
✅ Security decisions unchanged by refactor Blocked: sudo (blocklist), pkill -9 bash, chmod 777, unlisted commands, and compound commands containing one. Allowed: ls, npm test. 182/182 security unit tests and 9/9 integration tests pass. client.py still passes project_dir to the hook via context.
✅ Recording failures never affect the allow/block decision With .autoforge/ read-only (chmod 500) the command was still blocked and only a warning was logged. With no context, the command is blocked and nothing is recorded.
✅ Edge cases: unicode, long commands, rotation, non-JSON lines Non-ASCII command logged as UTF-8. A 5,000-char command had full_command truncated to 2,000 (the 'command' name field is not truncated, minor). A log over 1 MB rotated to .1 and both files were read back. Non-JSON lines and JSON arrays were skipped by the API.
✅ GET /api/projects/{name}/denied-commands Real uvicorn server on 127.0.0.1:5820 with HOME isolated in scratch. Returned 200 with total, commands aggregated by count desc (pytest x2, openssl x1, pip x1) and recent entries newest first. Unknown project: 404. Invalid name: 400.
✅ DELETE /api/projects/{name}/denied-commands Returned 200 and removed the log files; a second DELETE with no file also returned 200; unknown project returned 404.
✅ Robustness to type-corrupted log lines Non-blocking nit: a valid-JSON line with wrong types ({"command":5,"timestamp":3}) makes the GET return 500 (pydantic/compare error). The hook never writes such lines; only reachable by hand-editing the file.
✅ Generated .autoforge/.gitignore includes denied_commands.jsonl and .1 Confirmed in a freshly scaffolded project created via POST /api/projects.
✅ No new failures vs master (pytest, mypy) Same 21 pytest failures (all test_devserver_security.py) on master and the PR, with identical failing-test names; the PR adds 1 passing test. mypy: 25 errors on both, none in the changed files.
✅ UI smoke (PR adds no UI) UI build OK; the served SPA loads with 0 console errors. e2e and responsive checks skipped because no UI files changed.
🧪 Commands run
Command Result
ruff check . All checks passed
mypy . 25 errors in 10 files, identical to origin/master; none in changed files
python test_security.py 182 passed, 0 failed
python test_security_integration.py 9 passed, 0 failed
python -m pytest -q (PR vs origin/master worktree) PR: 145 passed / 21 failed; master: 144 passed / 21 failed; same failing tests, all pre-existing
npm --prefix ui run build success
npm --prefix ui run lint 0 errors, 6 warnings
uvicorn server.main:app --port 5820 (HOME=scratch) + curl GET/DELETE denied-commands 200/404/400 as expected; 500 only for a type-corrupted hand-edited line
scratch probe.py driving bash_security_hook recording, truncation, rotation and read-only-dir fallback behave as described

No browser screenshots were taken in this round.

Posted by cubefarm · merges automatically once GitHub's checks pass

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature request: Gather a list of denied commands during execution for later inspection

1 participant