Skip to content

Improving getting started with pyiceberg using the new notebook addition #2971

Closed as not planned

Description

@tusharchou

Feature Request / Improvement

1 .pyiceberg
2. pyspark using spark connect

Activity

  1. tusharchou commented on Jan 27, 2026

    @tusharchou
    ContributorAuthor

    Context
    With the recent merger of PR #2528, the repository now supports make notebook and make notebook-infra commands. These provide an instant, containerized environment for users to test PyIceberg with and without a full Spark infrastructure.

    Problem
    The current "Getting Started" documentation does not yet reflect these new capabilities. New users might still be manually configuring environments or struggling to set up Spark, unaware that a single command can now provision a full lab with PyIceberg, Spark Connect, MinIO, and a REST catalog.

    Proposal
    We should update the documentation (specifically the README.md and the "Getting Started" site page) to highlight this new workflow.

    Suggested Changes:

    1. PyIceberg Quickstart: Add a section explaining how to use make notebook to quickly launch a Jupyter environment for exploring the PyIceberg API.
    2. Full Integration Demo: Add a section explaining make notebook-infra for users who want to test PyIceberg with a local Spark cluster (using Spark Connect) and MinIO.
    3. Reference Example Notebooks: Link to the new notebooks/pyiceberg_example.ipynb and notebooks/spark_integration_example.ipynb as the recommended starting points.

    Benefits

    • Significantly lowers the barrier to entry for new contributors and users.
    • Standardizes the local development environment.
    • Showcases the PyIceberg + Spark Connect integration out of the box.
  2. github-actions commented on Jul 27, 2026

    @github-actions

    This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.

  3. github-actions commented on Aug 11, 2026

    @github-actions

    This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions