fix: don't retry an API error that declares itself non-retryable

[CyMule]

What

The client stops retrying an API error whose response body says it can't succeed on a retry. Today every 5xx is retried until the retry budget runs out, even when the server has already declared that resending the same request will fail the same way.

With the platform's retry settings (20s initial, 90s cap, 300s budget):

Before: POST → 503 {"retryable": false} → wait 20s → 503 → … → 6 attempts, ~320s, then ServerError
After:  POST → 503 {"retryable": false} → ServerError (1 attempt, same body)

Error body	Before	After
{"detail": …, "retryable": false, …}	retried	returned at once
{"detail": …, "retryable": true, …}	retried	retried
{"detail": "Internal server error"} (no verdict)	retried	retried
non-JSON, "retryable": "false" (string), unparsable	retried	retried

The same check covers the split-PDF path. A chunk that gets a declared non-retryable 5xx is sent once instead of using up its own per-chunk retry budget.

Why

The Unstructured API publishes its own verdict next to detail on errors it raises deliberately: error_type, audience, retryable and error_reason. An OCR (Tesseract) timeout, for example, answers 503 with retryable: false. The same image re-sent to the same time budget times out again. The retry loop matches on status before any caller can read the body, so no caller can act on that verdict. In production, one Transform record received 12 HTTP 500s across two dispatches over 670s. Attempt-6 retry markers are 98.6% as common as attempt-1 markers, so these failures do not recover on retry.

The retry loop is the one place that both whole-document requests and split-PDF chunk requests pass through. A caller-side hook can't reach the split-PDF chunks, because the split hook sends them through its own HTTP client. The check only ever stops a retry. A declared retryable: true never makes a status retryable that wasn't already.

retries.py already carries hand edits (#342). gen.yaml has persistentEdits enabled.

Notes for consumers

• No API change. Only a JSON object body with the boolean "retryable": false changes behaviour. An unread streamed response is never consumed; it keeps status-based retries.
• Retry-After is still not honored. This PR doesn't change that.

Validation

• Unit suite: 371 passed, 1 xfailed. make lint is clean (pylint 10/10, mypy).
• New tests cover sync and async retries, exact and range status codes, the split-PDF chunk path in buffered and cached modes, and public partition / partition_async through a mocked transport.
• Two rounds of Codex review. The finding from round 1 (an uncaught RecursionError on a deeply nested body) is fixed.

[cubic-dev-ai]

1 issue found across 5 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="src/unstructured_client/utils/retries.py">

<violation number="1" location="src/unstructured_client/utils/retries.py:94">
P3: This exception list misses `httpx.StreamConsumed` and `httpx.StreamClosed`, so a matching streamed response that was consumed or closed before inspection becomes a permanent error and skips the existing status retry. Treat these unavailable-body exceptions like `ResponseNotRead` and return `False`.</violation>
</file>

<sub>Shadow auto-approve: would not auto-approve because issues were found.

View guided diff | Re-trigger cubic</sub>