Skip to content

Implement standard request compression for generated Python clients - #855

Open
gouveags wants to merge 5 commits into
smithy-lang:developfrom
gouveags:feat/request-compression
Open

gouveags wants to merge 5 commits into
smithy-lang:developfrom
gouveags:feat/request-compression

Conversation

@gouveags

@gouveags gouveags commented Oct 19, 2026 •

Copy link
Copy Markdown

Related issue: #854.

Implement Smithy's requestCompression trait so generated clients compress eligible request bodies automatically. The public controls follow the shared AWS SDK configuration: disable_request_compression and request_min_compression_size_bytes (default 10240; inclusive range 0 through 10485760). Setting the threshold to zero compresses eligible buffered requests regardless of size; operations without gzip compression metadata remain uncompressed.

smithy-http owns the compression policy, smithy-core exposes the typed trait, and smithy-aws-core resolves standard configuration and integrates the policy with AWS protocols. Generated AWS clients inherit the controls; non-AWS generated HTTP clients receive the standard properties through the generator.

Compression runs after serialization and before signing. Wire headers reflect the compressed body, buffered-request retries reuse the same compressed bytes, and seekable streaming sources support hashing and rewinding without buffering the entire compressed payload. Event streams and streams requiring a known length remain excluded. Existing streaming-input retry restrictions remain.

The implementation also preserves buffered async-iterator data correctly on rewind, closes partially consumed gzip generators, and makes the aiohttp empty-body probe read one byte and restore position, including error and cancellation paths. Regression tests cover these transport boundaries.

Request compression can reduce upload bytes for conversation-heavy agents; it does not reduce model tokens or establish an end-to-end latency improvement. Bedrock Runtime still needs service acceptance and upstream compression metadata. Mantle's provider-compatible APIs need separate API/client coordination. No unsupported service operations are manually marked as compression-capable. Gzip is supported under the current contract; zstd requires separate contract and service agreement.

Related contributions: new SDK adoption and botocore threshold-range compatibility fix. Botocore already has its own compression implementation and does not depend on this PR.

Verification

Hosted CI on the final head reports action_required with no jobs run. Maintainer action is needed before hosted CI can validate this revision.

Reviewed implementation: 1ae921bac24a6ecebc61b8b43cd36382d359b4ec.

  • The original HTTP compression regression failed before implementation. Existing regressions cover compact trait values, request signing, buffered retries, stream seeking and cancellation/cleanup. Removal of the custom configuration field was also checked before and after narrowing the API.
  • make test-py: 2740 passed, 10 skipped.
  • make test-protocols: 783 passed, 32 expected failures across seven generated projections. Request-compression protocol cases are enabled.
  • make check-py: Ruff lint/format and strict Pyright passed.
  • make build-py, make build-java and Java formatting checks passed.
  • make generate-protocol-tests: passed after committing the regenerated configuration snapshot; generation reproduces the committed output.
  • git diff --check passed.

Validation used Python 3.12.14, Java 17 and Pandoc 3.8.2. The final follow-up commit changes only release-note wording; runtime/generator evidence is unchanged from 800d40ecc45e2f134a38a9625ba24dd46725d3c6. Tests are hermetic; no live endpoint acceptance or latency validation is claimed. The full supported-Python matrix and hosted CI are evaluated separately. Platform E2E changes are not needed: runtime, real-signer/transport regressions and generated protocol tests exercise the changed contract.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@gouveags gouveags changed the title Implement request compression for generated Python clients Implement standard request compression for generated Python clients Oct 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant