[zwanski-start]

Closes #157

What changed

• regression_runner.py (new): loads tests/regression/manifest.json ({"<feature_id>": {"command": "...", "timeout": 300}}) from the generated project and runs each assigned feature's command from the project root. The timeout defaults to 300s and is capped at 900s; on timeout the whole process tree is killed. Commands are checked against the same bash allowlist and blocklist as the agent.
• agent.py: a testing agent runs these scripts before it opens a Claude session. A feature whose script passes needs no LLM session. If every assigned feature passes, the process exits without using any tokens. Features with no script, or whose script failed, go to the agent, and the tail of each failing script's output is added to its prompt.
• security.py: moved the hook's validation logic into a synchronous validate_bash_command(command, project_dir). bash_security_hook now calls it, and its behaviour is unchanged.
• testing_prompt.template.md: new STEP 4 tells the testing agent to write a deterministic script for each verified feature, register it in the manifest, run it and commit it. If the script failed, the agent fixes the app or the script.

Verification

• python test_security.py: 169 passed
• ruff check on the changed files: clean
• Checked by hand that the example manifest commands (npx/node/npm) pass the allowlist and that sudo is rejected.

Left to do (why this is a draft)

• Unit tests for regression_runner.py (manifest parsing, pass/fail/timeout, security rejection) have not been written.
• mypy and the full pytest suite have not been run.
• No end-to-end run against a real generated project yet.
• This PR is opened from the fork zwanski-start/autoforge. The agent's GitHub account only has read access to AutoForgeAI/autoforge (push returned 403), so the branch couldn't be pushed to origin.

Assumptions

• A failing script does not automatically mark the feature as failing, because the script itself might be flaky. The LLM agent decides and fixes either the app or the script.
• Existing projects keep their already-scaffolded testing_prompt.md, so they will not start writing scripts on their own. The runner still uses any manifest it finds. Only new projects get the updated template.
• The default allowlist does not include python, so the prompt suggests Node/npm/Playwright commands.

🤖 Generated with Claude Code