Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
05b5dfe
Update opentelemetry-sdk requirement from >=1.44.0 to >=1.45.0
dependabot[bot] Sep 29, 2026
91a0eb2
Update opentelemetry-api requirement from >=1.44.0 to >=1.45.0
dependabot[bot] Sep 29, 2026
b196f71
Update cryptography requirement from >=50.0.1 to >=50.0.2
dependabot[bot] Oct 5, 2026
b4052c0
Update boxsdk requirement from <11,>=10.15.0 to >=10.17.0,<11
dependabot[bot] Oct 5, 2026
d0bc67a
Exit 1 from the legacy CLI flags when an action failed
JE-Chen Oct 8, 2026
9c4e77e
build(deps-dev): update prometheus-client requirement
dependabot[bot] Oct 8, 2026
c9a5ec2
Merge pull request #111 from Integration-Automation/dependabot/pip/de…
JE-Chen Oct 11, 2026
038b9d3
Merge pull request #106 from Integration-Automation/dependabot/pip/de…
JE-Chen Oct 11, 2026
91ecac6
Merge dev and preserve dependency minimum updates
JE-Chen Oct 11, 2026
75422ef
Merge pull request #104 from Integration-Automation/dependabot/pip/de…
JE-Chen Oct 11, 2026
12413e7
Merge remote-tracking branch 'origin/dev' into feat/universal-storage…
JE-Chen Oct 11, 2026
10b54fc
Merge pull request #108 from Integration-Automation/feat/cli-exit-status
JE-Chen Oct 11, 2026
4418322
Merge dev and retain legacy CLI failure reporting
JE-Chen Oct 11, 2026
67b8c29
Merge remote-tracking branch 'origin/docs/fileautomation-1.0-roadmap'…
JE-Chen Oct 11, 2026
dd46c7a
Merge dev and preserve both OpenTelemetry updates
JE-Chen Oct 11, 2026
4fd7dbc
Merge pull request #105 from Integration-Automation/dependabot/pip/de…
JE-Chen Oct 11, 2026
a0f025e
Merge pull request #103 from Integration-Automation/dependabot/pip/de…
JE-Chen Oct 11, 2026
1efab72
Merge remote-tracking branch 'origin/dev' into feat/universal-storage…
JE-Chen Oct 11, 2026
d8837ac
Make circuit breaker state tests independent of wall clock timing
JE-Chen Oct 11, 2026
4669677
Merge remote-tracking branch 'origin/main' into feat/universal-storag…
JE-Chen Oct 11, 2026
013961a
Record full pull request validation results
JE-Chen Oct 11, 2026
169e8fa
Merge pull request #109 from Integration-Automation/feat/universal-st…
JE-Chen Oct 11, 2026
002e08d
Fix circuit breaker fixture naming and unused arguments
JE-Chen Oct 11, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1373,6 +1373,8 @@ python -m automation_file --execute_str '[["FA_create_dir",{"dir_path":"x"}]]'
python -m automation_file --create_project ./my_project
```

The legacy flags exit with status 1 when an action of the list failed, after the rest of the list has run, and print `error: N action(s) failed` on stderr.

## JSON action format

Each entry is either a bare command name, a `[name, kwargs]` pair, or a
Expand Down
2 changes: 2 additions & 0 deletions README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -1328,6 +1328,8 @@ python -m automation_file --execute_str '[["FA_create_dir",{"dir_path":"x"}]]'
python -m automation_file --create_project ./my_project
```

旧式标志在清单中有动作失败时,会在其余动作执行完之后以状态 1 结束,并在 stderr 打印 `error: N action(s) failed`。

## JSON 动作格式

每一项动作可以是单纯的命令名称、`[name, kwargs]` 组合,或 `[name, args]`
Expand Down
2 changes: 2 additions & 0 deletions README.zh-TW.md
Original file line number Diff line number Diff line change
Expand Up @@ -1329,6 +1329,8 @@ python -m automation_file --execute_str '[["FA_create_dir",{"dir_path":"x"}]]'
python -m automation_file --create_project ./my_project
```

舊式旗標在清單中有動作失敗時,會在其餘動作執行完之後以狀態 1 結束,並在 stderr 印出 `error: N action(s) failed`。

## JSON 動作格式

每一項動作可以是單純的指令名稱、`[name, kwargs]` 組合,或 `[name, args]`
Expand Down
5 changes: 4 additions & 1 deletion architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -239,7 +239,10 @@ storage.observe → events.storage_bridge → StorageError (only for a failing b
contract, guarded by `tests/test_legacy_cli_contract.py`. PyBreeze declares `automation-file` and, now that the SDKs and the GUI are extras, needs `automation-file[all]` to keep what it had (`progress.md` #29); it lists the package as
a dependency.
- **TestPioneer** imports `download_file` and `unzip_all` from the facade in-process
(`test_pioneer/executor/file/file_processing.py`). Its `parallel_run` does not spawn this package.
(`test_pioneer/executor/file/file_processing.py`). Its `file-runner` runs an action file through
`execute_action` in-process, and `parallel_run` spawns `python -m automation_file --execute_file <script>`
and reads the exit status: 1 when an action failed (`ActionExecutor.failure_count`, compared by
`__main__.main`), guarded by `tests/test_legacy_cli_contract.py`.
- **Names inherited from AutoControl**: the TCP starter is still called `start_autocontrol_socket_server`
and the action-dict key is `auto_control`. MailThunder has renamed both (`start_mail_thunder_socket_server`, `mail_thunder` key) and keeps the old names as deprecated aliases. MailThunder's
socket-server default is 9942, so it runs next to this package's servers (TCP 9943, HTTP 9944,
Expand Down
8 changes: 7 additions & 1 deletion automation_file/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@

from automation_file.cli_operations import add_operation_commands
from automation_file.cli_storage import add_storage_commands
from automation_file.core.action_executor import execute_action, execute_files
from automation_file.core.action_executor import execute_action, execute_files, executor
from automation_file.core.json_store import read_action_json
from automation_file.exceptions import ArgparseException
from automation_file.local.file_ops import create_file
Expand Down Expand Up @@ -249,6 +249,7 @@ def main(argv: list[str] | None = None) -> int:
return args.handler(args)

ran = False
failed_before = executor.failure_count
for key, handler in _LEGACY_DISPATCH.items():
value = getattr(args, key, None)
if value is None:
Expand All @@ -257,6 +258,11 @@ def main(argv: list[str] | None = None) -> int:
ran = True
if not ran:
raise ArgparseException("no argument supplied; try --help")
# A failed action does not stop the batch, so it is reported once the batch is done.
failed = executor.failure_count - failed_before
if failed:
print(f"error: {failed} action(s) failed", file=sys.stderr)
return 1
return 0


Expand Down
18 changes: 18 additions & 0 deletions automation_file/core/action_executor.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@

from __future__ import annotations

import threading
import time
from collections.abc import Mapping
from concurrent.futures import ThreadPoolExecutor
Expand Down Expand Up @@ -65,6 +66,8 @@ class ActionExecutor(_CoreActionExecutor):

def __init__(self, registry: ActionRegistry | None = None) -> None:
super().__init__(_SETTINGS, registry or build_default_registry())
self._failures = 0
self._failures_lock = threading.Lock()
self.registry.register_many(
{
"FA_execute_action": self.execute_action,
Expand All @@ -87,6 +90,18 @@ def _parse_action(action: list) -> ParsedAction:
return _PARSER.parse(action)

# Public API --------------------------------------------------------
@property
def failure_count(self) -> int:
"""How many actions have failed in this executor since it was created.

A failed action is recorded as ``repr(error)`` and the batch goes on, so
the return value of ``execute_action`` does not say whether anything
failed. A caller that must know (the CLI, for its exit status) compares
this count before and after. Dry runs are not counted.
"""
with self._failures_lock:
return self._failures

def validate(self, action_list: list | Mapping[str, Any]) -> list[str]:
"""Validate shape and resolve every name; return the list of action names.

Expand Down Expand Up @@ -182,6 +197,9 @@ def _run_one(self, action: list, dry_run: bool, display: list | None = None) ->
return repr(error)
finally:
record_action(name, time.monotonic() - started, ok)
if not ok:
with self._failures_lock:
self._failures += 1

def _run_dry(self, action: list, display: list | None = None) -> Any:
display_action = action if display is None else display
Expand Down
4 changes: 2 additions & 2 deletions dev_requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@ google-api-python-client
google-auth-httplib2
google-auth-oauthlib
APScheduler
cryptography>=50.0.1
prometheus_client>=0.25.0
cryptography>=50.0.2
prometheus_client>=0.26.0
defusedxml>=0.7.1
twine
build
7 changes: 7 additions & 0 deletions docs/source/Eng/usage/cli.rst
Original file line number Diff line number Diff line change
Expand Up @@ -128,3 +128,10 @@ offset is read as local time.
``purge`` deletes the records older than a number of days and prints how many it
removed. A pipeline run's ID is its correlation ID, so one search shows everything
a run did.

Exit status of the legacy flags
-------------------------------

``-e``, ``-d`` and ``--execute_str`` exit with status 1 when an action of the list failed. A failed
action does not stop the list: the rest runs first, and ``error: N action(s) failed`` is printed on
stderr at the end. A script that cannot be read exits with 1 as before.
7 changes: 7 additions & 0 deletions docs/source/Zh-CN/usage/cli.rst
Original file line number Diff line number Diff line change
Expand Up @@ -118,3 +118,10 @@ CLI
``--offset``。``count`` 接受相同的筛选条件。``--since`` 与 ``--until`` 接受 ISO 8601 的
日期或时间,没有 UTC 偏移时视为本地时间。``purge`` 会删除早于指定天数的记录,并输出
删除的条数。流水线运行的 ID 就是它的关联 ID,因此一次搜索就能看到一次运行所做的一切。

旧式标志的结束状态
------------------

``-e``、``-d`` 与 ``--execute_str`` 在清单中有动作失败时以状态 1 结束。失败的动作不会中止清单:
其余动作会先执行完,最后在 stderr 打印 ``error: N action(s) failed``。
无法读取的脚本和以前一样以 1 结束。
7 changes: 7 additions & 0 deletions docs/source/Zh-TW/usage/cli.rst
Original file line number Diff line number Diff line change
Expand Up @@ -118,3 +118,10 @@ CLI
``--offset``。``count`` 接受相同的篩選條件。``--since`` 與 ``--until`` 接受 ISO 8601 的
日期或時間,沒有 UTC 偏移時視為本地時間。``purge`` 會刪除早於指定天數的紀錄,並輸出
刪除的筆數。管線執行的 ID 就是它的關聯 ID,因此一次搜尋就能看到一次執行所做的一切。

舊式旗標的結束狀態
------------------

``-e``、``-d`` 與 ``--execute_str`` 在清單中有動作失敗時以狀態 1 結束。失敗的動作不會中止清單:
其餘動作會先執行完,最後在 stderr 印出 ``error: N action(s) failed``。
無法讀取的腳本和以前一樣以 1 結束。
22 changes: 22 additions & 0 deletions docs/updates/2026-10.md
Original file line number Diff line number Diff line change
Expand Up @@ -734,3 +734,25 @@ Index and query commands: [README.md](README.md). New entries go at the end.
- **Merge order**: PyBreeze #142 must pass and merge before FileAutomation #110 publishes the stable extras split.
- **Files**: `progress.md` and the update-log index.
- **Evidence**: https://github.com/Integration-Automation/PyBreeze/pull/142

## U-20261008-39 · 2026-10-08 · The legacy CLI flags exit 1 when an action failed · #feature #cli #testpioneer

- **What**: `python -m automation_file` with `-e`, `-d` or `--execute_str` ended with status 0 whatever happened, because the executor records a failed action as `repr(error)` and runs the next one. `ActionExecutor` now counts the actions that failed (`failure_count`, thread-safe, dry runs excluded), `__main__.main` compares the count before and after the legacy dispatch, and the process exits 1 with `error: N action(s) failed` on stderr. The rest of the list still runs first.
- **Why**: TestPioneer's `parallel_run` starts `--execute_file` and reads the exit status (its `docs/artifacts.rst`, "Native Support"), and a CI job does the same. LoadDensity made the same change in its PR #118.
- **Effect on callers**: a script with a failing action ends with status 1 where it ended with 0. PyBreeze only displays the status. `execute_action` and its return value are unchanged.
- **Also**: `architecture.md` §6 said TestPioneer's `parallel_run` does not spawn this package; it does, for its `file-runner`.
- **Tests**: `tests/test_legacy_cli_contract.py` +4 (real child processes), `tests/test_cli_main.py` +3.
- **Docs**: `README.md`, `README.zh-TW.md`, `README.zh-CN.md`, `docs/source/{Eng,Zh-TW,Zh-CN}/usage/cli.rst`, `architecture.md` §6.
- **Files**: `automation_file/core/action_executor.py`, `automation_file/__main__.py`, the tests and docs above.

## U-20261011-01 · 2026-10-11 · Integrate open pull requests and stabilize circuit breaker tests · #ci #tests #dependencies

- **What**: integrate the dependency updates, the legacy CLI failure exit status (#108), and the roadmap (#107) into the storage implementation branch (#109). Resolve adjacent dependency edits without losing either minimum version, retain the CLI and storage documentation in all three languages, and give the CLI update log its own ID (`U-20261008-39`).
- **CI finding**: main run 38042168722 failed `test_opens_after_threshold_failures` after the 0.1-second recovery timeout elapsed before the assertion. Replace wall-clock timing with a controlled monotonic clock in the threshold and recovery tests; production behavior is unchanged.
- **Validation**: The full Windows/Python 3.11 suite passes (5776 passed, 257 skipped); the updated circuit breaker tests separately pass all eight cases. Also verified 52 CLI/executor tests, 43 dependency-related tests against the five updated package versions, and 32 metadata/release tests after synchronizing main versions. Ruff lint/format and mypy checks pass. New GitHub Actions runs were queued during review; the original PR checks passed.
- **Files**: `tests/test_circuit_breaker.py`, the merged CLI code and documentation, dependency requirement files, and the update-log index.


## U-20261011-02 · 2026-10-11 · Keep circuit breaker clock fixtures lint clean · #done #ci

Name the clock fixture explicitly and request its side effect through usefixtures where no clock value is used. All eight circuit breaker regression tests pass.
5 changes: 4 additions & 1 deletion docs/updates/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,9 @@ In the same commit: delete the item from `progress.md`, add a `#done` entry here

| ID | Date | Title | Tags | Batch |
|---|---|---|---|---|
| U-20261011-02 | 2026-10-11 | Keep circuit breaker clock fixtures lint clean | #done #ci | [2026-10](2026-10.md) |
| U-20261011-01 | 2026-10-11 | Integrate open pull requests and stabilize circuit breaker tests | #ci #tests #dependencies | [2026-10](2026-10.md) |
| U-20261008-39 | 2026-10-08 | The legacy CLI flags exit 1 when an action failed | #feature #cli #testpioneer | [2026-10](2026-10.md) |
| U-20261008-38 | 2026-10-08 | Integrate the PyBreeze all-extras dependency before stable release | #done #dependencies | [2026-10](2026-10.md) |
| U-20261008-37 | 2026-10-08 | Main PR validation passes the full CI and analysis gates | #done #ci #security | [2026-10](2026-10.md) |
| U-20261008-36 | 2026-10-08 | Keep the GUI on the tested Qt 6.11 series | #incident #gui #ci | [2026-10](2026-10.md) |
Expand Down Expand Up @@ -133,5 +136,5 @@ In the same commit: delete the item from `progress.md`, add a `#done` entry here

| File | Period | Entries |
|---|---|---:|
| [2026-10.md](2026-10.md) | 2026-10 | 44 |
| [2026-10.md](2026-10.md) | 2026-10 | 52 |
| [2026-09.md](2026-09.md) | 2026-09 | 21 |
1 change: 0 additions & 1 deletion progress.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,4 +31,3 @@ Items #10 to #26 are what is left of the 1.0 roadmap (`docs/FILEAUTOMATION-1.0-R
- **#34** [BLOCKED] PyPI Trusted Publishing (roadmap §13). `publish.yml` and `publish-dev` still upload with the `PYPI_API_TOKEN` secret. Switching needs the owner to add a trusted publisher for each project on PyPI (`automation_file`: workflow `publish.yml`; `automation_file_dev`: workflow `ci-dev.yml`; an environment name if one is wanted) before the workflows can drop the token for `id-token: write` and `pypa/gh-action-pypi-publish`. Changing the workflows first would break both channels.

### Packaging follow-ups

6 changes: 3 additions & 3 deletions requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ watchdog
defusedxml>=0.7.1
PyYAML>=6.0.3
pyarrow>=25.0.1
opentelemetry-api>=1.44.0
opentelemetry-sdk>=1.44.0
opentelemetry-api>=1.45.0
opentelemetry-sdk>=1.45.0
msal>=1.39.0
boxsdk>=10.15.0,<11
boxsdk>=10.17.0,<11
tomli; python_version<"3.11"
21 changes: 16 additions & 5 deletions tests/test_circuit_breaker.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,20 +2,31 @@

from __future__ import annotations

import time
from types import SimpleNamespace

import pytest

from automation_file.core.circuit_breaker import CircuitBreaker
from automation_file.exceptions import CircuitOpenException


@pytest.fixture(name="breaker_clock")
def _breaker_clock(monkeypatch: pytest.MonkeyPatch) -> list[float]:
clock = [100.0]
monkeypatch.setattr(
"automation_file.core.circuit_breaker.time",
SimpleNamespace(monotonic=lambda: clock[0]),
)
return clock


def test_closed_passes_through() -> None:
cb = CircuitBreaker(failure_threshold=3, recovery_timeout=0.1)
assert cb.call(lambda x: x + 1, 1) == 2
assert cb.state == "closed"


@pytest.mark.usefixtures("breaker_clock")
def test_opens_after_threshold_failures() -> None:
cb = CircuitBreaker(failure_threshold=2, recovery_timeout=0.1)

Expand All @@ -38,23 +49,23 @@ def _raise_value_error(msg: str = "v") -> None:
raise ValueError(msg)


def test_half_open_allows_probe_and_closes_on_success() -> None:
def test_half_open_allows_probe_and_closes_on_success(breaker_clock: list[float]) -> None:
cb = CircuitBreaker(failure_threshold=1, recovery_timeout=0.05)
with pytest.raises(ConnectionError):
cb.call(_raise_connection_error)
assert cb.state == "open"

time.sleep(0.08)
breaker_clock[0] += 0.08
assert cb.state == "half_open"
assert cb.call(lambda: "ok") == "ok"
assert cb.state == "closed"


def test_half_open_failure_reopens() -> None:
def test_half_open_failure_reopens(breaker_clock: list[float]) -> None:
cb = CircuitBreaker(failure_threshold=1, recovery_timeout=0.05)
with pytest.raises(ConnectionError):
cb.call(_raise_connection_error)
time.sleep(0.08)
breaker_clock[0] += 0.08
assert cb.state == "half_open"
with pytest.raises(ConnectionError):
cb.call(_raise_connection_error, "y")
Expand Down
35 changes: 35 additions & 0 deletions tests/test_cli_main.py
Original file line number Diff line number Diff line change
Expand Up @@ -95,3 +95,38 @@ def test_execute_str_accepts_double_encoded_payload(

assert rc == 0
assert received == [actions]


def test_legacy_flag_returns_1_when_an_action_failed(
tmp_path: Any, capsys: pytest.CaptureFixture[str]
) -> None:
actions = tmp_path / "actions.json"
actions.write_text(json.dumps([["FA_no_such_action"], ["FA_no_such_action"]]), encoding="utf-8")

assert cli_main.main(["--execute_file", str(actions)]) == 1
assert "error: 2 action(s) failed" in capsys.readouterr().err


def test_legacy_flag_returns_0_when_every_action_passed(tmp_path: Any) -> None:
target = tmp_path / "created.txt"
actions = tmp_path / "actions.json"
actions.write_text(
json.dumps([["FA_create_file", {"file_path": str(target), "content": "x"}]]),
encoding="utf-8",
)

assert cli_main.main(["--execute_file", str(actions)]) == 0
assert target.exists()


def test_failures_of_an_earlier_run_are_not_counted_again(tmp_path: Any) -> None:
bad = tmp_path / "bad.json"
bad.write_text(json.dumps([["FA_no_such_action"]]), encoding="utf-8")
good = tmp_path / "good.json"
good.write_text(
json.dumps([["FA_create_file", {"file_path": str(tmp_path / "x.txt"), "content": "x"}]]),
encoding="utf-8",
)

assert cli_main.main(["--execute_file", str(bad)]) == 1
assert cli_main.main(["--execute_file", str(good)]) == 0
33 changes: 33 additions & 0 deletions tests/test_legacy_cli_contract.py
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,39 @@ def test_create_project(tmp_path, flag):
assert any(project.rglob("*.json")), result.stdout + result.stderr


def _failing_actions(target: Path) -> list:
"""An action that fails (no command has that name), then one that writes *target*."""
return [["FA_no_such_action"], *_actions(target)]


@pytest.mark.parametrize("flag", ["-e", "--execute_file"])
def test_a_failed_action_exits_1_after_the_rest_ran(tmp_path, flag):
target = tmp_path / "created.txt"
action_file = tmp_path / "actions.json"
action_file.write_text(json.dumps(_failing_actions(target)), encoding="utf-8")
result = _run_cli(tmp_path, flag, str(action_file))
assert result.returncode == 1
assert "error: 1 action(s) failed" in result.stderr
assert target.exists(), result.stdout + result.stderr


def test_a_failed_action_in_a_directory_exits_1(tmp_path):
action_dir = tmp_path / "actions"
action_dir.mkdir()
(action_dir / "a_ok.json").write_text(
json.dumps(_actions(tmp_path / "a.txt")), encoding="utf-8"
)
(action_dir / "b_bad.json").write_text(
json.dumps(_failing_actions(tmp_path / "b.txt")), encoding="utf-8"
)
assert _run_cli(tmp_path, "--execute_dir", str(action_dir)).returncode == 1


def test_a_failed_action_in_execute_str_exits_1(tmp_path):
payload = _pybreeze_execute_str(_failing_actions(tmp_path / "created.txt"))
assert _run_cli(tmp_path, "--execute_str", payload).returncode == 1


def test_no_flag_exits_non_zero(tmp_path):
assert _run_cli(tmp_path).returncode != 0

Expand Down
Loading