Skip to content

Add option to pass storage config to polars lazy read #3128

Closed

Description

@johannesq23

Feature Request / Improvement

I was not able to use the table.to_polars() method because there is no option to include a storage_options param which is mentioned in the polars docs.

Activity

  1. smaheshwar-pltr commented on Mar 10, 2026

    @smaheshwar-pltr
    Contributor

    +1 it feels like here

    def to_polars(self) -> pl.LazyFrame:
    """Lazily read from this Apache Iceberg table.
    Returns:
    pl.LazyFrame: Unmaterialized Polars LazyFrame created from the Iceberg table
    """
    import polars as pl
    return pl.scan_iceberg(self)

    storage options should default to self.io.properties

  2. smaheshwar-pltr commented on Mar 17, 2026

    @smaheshwar-pltr
    Contributor

    Ah actually it can't just be self.io.properties because some of those non-IO properties conflict with storage properties and you might end up with

    ValueError: unsupported: cannot combine token with default-key in storage_options
    
  3. kris-gaudel commented on Apr 22, 2026

    @kris-gaudel
    Contributor

    Can I be assigned this? @kevinjqliu

  4. github-actions commented on Oct 20, 2026

    @github-actions

    This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions