Skip to content

feat(bigtable): raise the UDS channel inbound message cap to 256 MB - #18782

Open
mutianf wants to merge 1 commit into
accel10from
accel11
Open

mutianf wants to merge 1 commit into
accel10from
accel11

Conversation

@mutianf

@mutianf mutianf commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

gRPC caps inbound messages at 4 MiB by default — a limit sized for a channel that crosses a network. The accelerator's UDS channel doesn't leave the machine: the daemon has already paid the cost of pulling the response off the wire, and all the cap does on this hop is turn an otherwise-fine read into RESOURCE_EXHAUSTED. This sets grpc.max_receive_message_length to 256 MB so the UDS leg is never the narrower of the two.

No test yet. The Go daemon still has its own 4 MiB cap on the sending side, so nothing above 4 MiB actually crosses the socket today and a test would only be asserting on a config value. The test lands once the daemon side is raised.

Only the inbound direction is changed here; grpc.max_send_message_length is untouched.

@mutianf
mutianf requested a review from a team as a code owner October 20, 2026 18:00
@mutianf
mutianf added this pull request to stack #18691 October 20, 2026 18:07

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request increases the inbound message size limit on the Unix Domain Socket (UDS) channel to 256 MiB for both the asynchronous and synchronous accelerator clients in the Google Cloud Bigtable library. This is achieved by defining _MAX_RECEIVE_MESSAGE_LENGTH and passing it via _CHANNEL_OPTIONS to the insecure_channel constructor, preventing potential RESOURCE_EXHAUSTED errors on local reads. There are no review comments, so I have no feedback to provide.

Change-Id: I6adb2982640fe2d6736ea63ed165c9ffd6feacb7
# the UDS leg is never the narrower of the two.
_MAX_RECEIVE_MESSAGE_LENGTH = 256 * 1024 * 1024

_CHANNEL_OPTIONS = (("grpc.max_receive_message_length", _MAX_RECEIVE_MESSAGE_LENGTH),)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two concerns with the one-sided cap:

1. Send-side left at the gRPC 4 MiB default — silent regression vs. the native path. The native google-cloud-bigtable Python client sets both grpc.max_send_message_length and grpc.max_receive_message_length to -1 (unlimited). This PR raises only inbound, so an ApplyBulk / MutateRow with a payload >4 MiB (allowed by the server's 100 MiB per-cell limit) will fail at the accelerator channel's outbound serialization check with no analogous server-side error. On the native path the same call goes through. Suggest raising send too:

Suggested change
_CHANNEL_OPTIONS = (("grpc.max_receive_message_length", _MAX_RECEIVE_MESSAGE_LENGTH),)
_CHANNEL_OPTIONS = (
("grpc.max_receive_message_length", _MAX_RECEIVE_MESSAGE_LENGTH),
("grpc.max_send_message_length", _MAX_RECEIVE_MESSAGE_LENGTH),
)

2. Pick a side on the 256 MB number. Native uses -1 (unlimited). 256 MB from thin air invites "why not 128? why not 1 GiB?" and future drift when the native client bumps. Either match native (-1, both directions) and document "match native" in the comment, or if 256 MB is a deliberate memory-safety choice at the UDS hop, say that explicitly.

self._metadata = (("x-accelerator-token", auth_secret),)
self._channel = insecure_channel(_uds_target(uds_path))
self._channel = insecure_channel(
_uds_target(uds_path), options=_CHANNEL_OPTIONS

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: no test exercises _CHANNEL_OPTIONS. Even a one-liner asserting the option tuple is on self._channel's options would pin both the key spelling (grpc.max_receive_message_length is easy to typo) and the value. Low priority.

@sushanb sushanb left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving to unblock the stack, but please address the send-side cap gap above before merge — leaving outbound at the gRPC 4 MiB default is a silent regression vs. the native client, which sets both directions to -1. Simplest fix is one extra line in .

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants