Skip to content

fix(bigquery-jdbc): format nested ARRAY fields as valid JSON in Struct.toString() - #14720

Merged
keshavdandeva merged 12 commits into
mainfrom
jdbc/fix-struct-nested-array-json
Oct 22, 2026
Merged

keshavdandeva merged 12 commits into
mainfrom
jdbc/fix-struct-nested-array-json

Conversation

@keshavdandeva

Copy link
Copy Markdown
Contributor

b/570669133

Summary

Fixes BigQueryBaseStruct.toString() (and ResultSet.getString() on STRUCT / ARRAY<STRUCT> columns) so that nested ARRAY fields inside a STRUCT have their elements formatted as valid JSON values instead of unquoted Arrays.deepToString() strings.

Problem

BigQueryBaseStruct.toString() constructs a JSON object string ({...}) and quotes scalar STRING, DATE, TIME, DATETIME, TIMESTAMP, GEOGRAPHY, and Base64-encoded BYTES attributes. However, when a STRUCT contained a nested ARRAY attribute (BigQueryBaseArray), BigQueryBaseStruct.toString() fell through to value.toString(), which called BigQueryBaseArray.toString() (Arrays.deepToString). This produced unquoted array elements inside the JSON object (e.g., {"str_arr":[hello, world],"date_arr":[2026-01-01, 2026-01-02]}), resulting in syntactically invalid JSON.

Solution

  • Extracted the existing scalar JSON value formatting logic in BigQueryBaseStruct.toString() into a shared helper method appendJsonValue(...).
  • When a STRUCT attribute is a BigQueryBaseArray, iterate over its elements and format each element using appendJsonValue(...) inside [...].
  • Top-level BigQueryBaseArray.toString() behavior remains unchanged for backwards compatibility.

Example Output

For SELECT STRUCT(1 AS id, ['hello', 'world'] AS str_arr, [DATE '2026-01-01'] AS date_arr, [TIME '12:34:56.789012'] AS time_arr, [TIMESTAMP '2026-01-01 00:00:00 UTC'] AS ts_arr, [b'abc'] AS bytes_arr):

  • Before: {"id":1,"str_arr":[hello, world],"date_arr":[2026-01-01],"time_arr":[12:34:56.789012],"ts_arr":[2026-01-01 00:00:00.0],"bytes_arr":[YWJj]} (Invalid JSON)
  • After: {"id":1,"str_arr":["hello","world"],"date_arr":["2026-01-01"],"time_arr":["12:34:56.789012"],"ts_arr":["2026-01-01 00:00:00.0"],"bytes_arr":["YWJj"]} (Valid JSON)

Testing

  • Added struct.toString() assertions covering all BigQuery array types to BigQueryJsonStructTest#structOfArrays and BigQueryArrowStructTest#structOfArrays.
  • Added integration test ITBigQueryJDBCTest#testStructWithNestedArraysValidJson verifying both REST and Storage Read API (EnableHighThroughputAPI=1) paths.

@keshavdandeva
keshavdandeva added this pull request to stack #14721 October 21, 2026 19:25
@keshavdandeva
keshavdandeva requested review from a team as code owners October 21, 2026 19:25
@keshavdandeva
keshavdandeva force-pushed the jdbc/fix-struct-nested-array-json branch from c966c60 to 84db5af Compare October 21, 2026 19:25

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the toString method in BigQueryBaseStruct to properly serialize nested arrays (BigQueryBaseArray) into valid JSON format. It extracts the JSON formatting logic into a new helper method appendJsonValue and adds unit and integration tests to verify the JSON serialization of structs with nested arrays. There are no review comments, and I have no additional feedback to provide.

…d-array-json

# Please enter a commit message to explain why this merge is necessary,
# especially if it merges an updated upstream into a topic branch.
#
# Lines starting with '#' will be ignored, and an empty message aborts
# the commit.
…bc/fix-struct-nested-array-json

# Conflicts:
#	java-bigquery-jdbc/src/test/java/com/google/cloud/bigquery/jdbc/it/ITBigQueryJDBCTest.java
…ray-json

# Conflicts:
#	java-bigquery-jdbc/src/test/java/com/google/cloud/bigquery/jdbc/it/ITBigQueryJDBCTest.java
Base automatically changed from jdbc/fix-time-precision to main October 22, 2026 17:20
…d-array-json

# Conflicts:
#	java-bigquery-jdbc/src/main/java/com/google/cloud/bigquery/jdbc/BigQueryBaseStruct.java
@keshavdandeva
keshavdandeva merged commit bc0126a into main Oct 22, 2026
199 checks passed
@keshavdandeva
keshavdandeva deleted the jdbc/fix-struct-nested-array-json branch October 22, 2026 18:05
jinseopkim0 pushed a commit that referenced this pull request Oct 23, 2026
🤖 I have created a release *beep* *boop*
---


<details><summary>1.94.0</summary>

##
[1.94.0](v1.93.0...v1.94.0)
(2026-10-22)


### Features

* **auth:** request certificate-bound tokens for agent identities by
([01eaba1](01eaba1))
* **bigquery-jdbc:** Add clearcut telemetry client
([#14607](#14607))
([c702438](c702438))
* **bigquery-jdbc:** expose QueryStatistics for jobless queries
([#14596](#14596))
([15f472c](15f472c))
* **bigquery:** expose QueryStatistics from TableResult
([#14588](#14588))
([40b2b1a](40b2b1a))
* **bigtable:** add per-method session diversion configuration
([#14681](#14681))
([042dd3e](042dd3e))
* **bigtable:** support SessionCheckAndMutateRow on the session client
([#14565](#14565))
([86de9c1](86de9c1))
* **bigtable:** support SessionReadModifyWriteRow on the session client
([#14585](#14585))
([8e653b2](8e653b2))
* **firestore:** DML
([#14703](#14703))
([11bc071](11bc071))
* **gax:** implement operation-level client request spans (T3)
([#14549](#14549))
([275063b](275063b))
* **gax:** link attempt spans to parent context and clean up on
operation completion
([#14459](#14459))
([0c481bd](0c481bd))
* **gax:** standardize mapping terminal resumable upload exceptions to
ApiException
([#14624](#14624))
([d4961f0](d4961f0))
* **google/devicesandservices/health/v4beta:** add health v4beta
([#14600](#14600))
([95f863c](95f863c))
* **google/shopping/merchant/reviews/v1:** add shopping-merchant-reviews
([#14724](#14724))
([13310c7](13310c7))
* include writerId as part of trace id
([#13488](#13488))
([ffa8063](ffa8063))
* log actionable errors on operation failure in LoggingTracer
([#14460](#14460))
([27559f4](27559f4))
* populate client_stats in AppendRowsRequest
([#14614](#14614))
([a7a0098](a7a0098))
* **spanner:** add option to create GcpFallbackChannel per channel
instead of per pool
([#14501](#14501))
([74f7591](74f7591))


### Bug Fixes

* **api-common:** validate path template variables against dot segments
([#14641](#14641))
([4ad59e2](4ad59e2))
* **auth:** fix serialization gaps in DownscopedCredentials and
UrlIdentityPoolSubjectTokenSupplier
([#14341](#14341))
([a3e3324](a3e3324))
* **bigquery-jdbc:** ensure JDBC uses PQC-compliant algorithm by default
([#14560](#14560))
([b0ffe3d](b0ffe3d))
* **bigquery-jdbc:** fix Exception handling
([#14566](#14566))
([3e7a206](3e7a206))
* **bigquery-jdbc:** fix missing logger instances
([#14733](#14733))
([0e822dc](0e822dc))
* **bigquery-jdbc:** fix pqc working in proxy env
([#14599](#14599))
([ceb45b4](ceb45b4))
* **bigquery-jdbc:** format nested ARRAY fields as valid JSON in
Struct.toString()
([#14720](#14720))
([bc0126a](bc0126a))
* **bigquery-jdbc:** format TIMESTAMP as UTC in getString and add
multi-zone TIMESTAMP(12) tests
([#14666](#14666))
([e666612](e666612))
* **bigquery-jdbc:** honor `Calendar` timezone and prevent
`ClassCastException` in ResultSet temporal getters
([#14657](#14657))
([7dd2b37](7dd2b37))
* **bigquery-jdbc:** limit Storage Read API Arrow batch queue to 32
batches
([#14709](#14709))
([d339c41](d339c41))
* **bigquery-jdbc:** normalize `setTime` from Calendar zone to JVM zone
([#14660](#14660))
([54d2bc6](54d2bc6))
* **bigquery-jdbc:** poll the JSON reference queue and release processed
finalizers
([#14719](#14719))
([1b7d00d](1b7d00d))
* **bigquery-jdbc:** preserve `Timestamp` instant in `PreparedStatement`
and `CallableStatement` Calendar overloads
([#14640](#14640))
([3b02a44](3b02a44))
* **bigquery-jdbc:** preserve the instant on the TIMESTAMP write path
and bind LocalDateTime as STRING
([#14647](#14647))
([2106e26](2106e26))
* **bigquery-jdbc:** stop per-ResultSet logger registration leak
([#14718](#14718))
([e83c768](e83c768))
* **bigquery-jdbc:** stop producer and release resources on
ResultSet.close
([#14712](#14712))
([6b99452](6b99452))
* **bigquery:** fix invalid mappings in domain models
([#14646](#14646))
([17048ed](17048ed))
* **bigquery:** remove duplicate google-cloud-storage dependency
([#14586](#14586))
([ab6173e](ab6173e))
* **datastore:** allow any gRPC TransportChannelProvider in
DatastoreOptions
([#14478](#14478))
([d9a8db5](d9a8db5))
* **datastore:** update emulator zip checksum for 2.3.1
([#14592](#14592))
([fb73ed5](fb73ed5))
* **deps:** update dependency
com.google.api-client:google-api-client-bom to v2.9.2
([#14725](#14725))
([1ffa467](1ffa467))
* **gax-httpjson:** gracefully handle error details with unrecognized
types
([#14612](#14612))
([7295640](7295640))
* revise health metrics inflightBytes to become inflightBytesMax
([#14613](#14613))
([44a12ef](44a12ef))
* **showcase:** update parent context retry tests for operation span in
ITOtelTracing
([#14673](#14673))
([312b5ea](312b5ea))
* **spanner:** increase max inbound message size from 256MB to 300MB
([#14551](#14551))
([70c7439](70c7439))
* **spanner:** increase max inbound message size from 256MB to 300MB as
([70c7439](70c7439))
* **spanner:** prevent stream leaks and race conditions on result set
close
([#14665](#14665))
([013b49b](013b49b))
* **spanner:** support autoscaling config, edition, and positive
capacity in cloud instance actions
([#14574](#14574))
([004f0fd](004f0fd))


### Performance Improvements

* **bigquery:** defer blocking jobs.get RPC in QueryPageFetcher until
getNextPage()
([#14650](#14650))
([6183b63](6183b63))


### Dependencies

* Upgrade jackson-bom to v2.22.4
([#14701](#14701))
([09ce4c5](09ce4c5))


### Documentation

* **auth:** clarify quota project behavior on ImpersonatedCredentials
([#14597](#14597))
([4e5a0d0](4e5a0d0))
* **pubsub:** Create a Cloud Pub/Sub sample for publishing with hedging
([#14623](#14623))
([9de611f](9de611f))
* **pubsub:** regenerate README.md sample table
([#14643](#14643))
([5394618](5394618)),
refs
[#14543](#14543)
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

---------

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants