Skip to content

Geometry and geography defaults make schema parsing fail with an unactionable NotImplementedError #3975

Closed as not planned

Description

@moomindani

A geometry or geography column that declares initial-default or write-default cannot have its schema parsed:

Schema.model_validate({"type": "struct", "schema-id": 0, "fields": [
    {"id": 1, "name": "geom", "required": False, "type": "geometry",
     "initial-default": "POINT (1 2)"}]})
# NotImplementedError: Geometry JSON deserialization requires WKT to WKB conversion, ...

Serializing a field that already holds WKB defaults fails symmetrically, as a PydanticSerializationError wrapping the same error.

The cause is that pyiceberg/conversions.py leaves to_json and from_json unimplemented for GeometryType and GeographyType, and those are reached from metadata handling rather than only from value conversion: from_json from _deserialize_default_value (pyiceberg/types.py:377), to_json from NestedField.serialize_model (pyiceberg/types.py:458). (from_json does pass bytes through unchanged; it is the WKT string form the spec mandates in JSON that fails.)

mkdocs/docs/geospatial.md already records WKB/WKT conversion as a deliberate omission, so I am not asking to revisit that. What looks unintended is how the omission surfaces: a NotImplementedError raised from inside Schema.model_validate, naming neither the field nor the default as the reason. The table simply cannot be loaded, and the message gives the caller nothing to act on. An explicit error about geospatial defaults being unsupported would at least make it diagnosable.

Happy to send that patch if it is wanted.

Related: #1818 lists Geometry/Geography as an open item but links #1820, which was closed once #2859 landed the types and the binary and Parquet paths.

Activity

  1. changed the title [-]Revisit the WKB/WKT conversion decision: geometry and geography defaults cannot be read or written[/-] [+]Geometry and geography defaults make schema parsing fail with an unactionable NotImplementedError[/+] on Sep 16, 2026
  2. moomindani commented on Sep 16, 2026

    @moomindani
    ContributorAuthor

    Closing this. I checked the reference implementation after filing, and Java's SingleValueParser has no GEOMETRY or GEOGRAPHY case in either direction — it throws UnsupportedOperationException("Type: geometry is not supported") for both. So no writer produces a geospatial JSON default in the first place, which means the schema this describes cannot arise, and PyIceberg's behaviour already matches Java's. The error message is no less actionable than Java's either.

    Sorry for the noise. If geospatial single-value serialization is ever added in Java, this becomes real, and the note above is the place to start.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions