Skip to content

fix(files): Bulk insert file blob indexes during assembly - #124987

Open
sentry[bot] wants to merge 2 commits into
masterfrom
seer/fix-n1-file-blob-indexes
Open

sentry[bot] wants to merge 2 commits into
masterfrom
seer/fix-n1-file-blob-indexes

Conversation

@sentry

@sentry sentry Bot commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

This PR addresses an N+1 query issue in AbstractFile.assemble_from_file_blob_ids.

Problem:
Previously, the assemble_from_file_blob_ids method iterated over file_blobs and called self._create_blob_index(blob=blob, offset=offset) for each blob individually. This resulted in N separate INSERT queries into the sentry_fileblobindex table for a file composed of N blobs, leading to significant performance overhead, especially for large files with many chunks.

This N+1 pattern affected tasks such as assemble_preprod_artifact_installable_app, assemble_artifacts, and assemble_dif.

Solution:

  1. An abstract method _bulk_create_blob_indexes was added to AbstractFile.
  2. Concrete implementations of _bulk_create_blob_indexes were provided in File and ControlFile, utilizing FileBlobIndex.objects.bulk_create and ControlFileBlobIndex.objects.bulk_create respectively.
  3. The assemble_from_file_blob_ids method was refactored:
    • It now first pre-computes all (blob, offset) pairs in a single pass.
    • Then, it calls _bulk_create_blob_indexes once, performing a single batch INSERT query for all FileBlobIndex records.
    • Finally, it iterates through the blobs again to stream their contents to a temporary file and compute the checksum.

This change reduces the database operations from N individual INSERTs to a single bulk_create call, significantly improving performance for file assembly operations.

Verification:

  • ruff check src/sentry/models/files/abstractfile.py src/sentry/models/files/file.py src/sentry/models/files/control_file.py passed successfully.
  • Automated tests for tests/sentry/tasks/test_assemble.py, tests/sentry/models/test_file.py, and tests/sentry/preprod/test_tasks.py should be run in CI to confirm full functionality, as the local sandbox environment does not support running the full test suite.

Legal Boilerplate

Look, I get it. The entity doing business as "Sentry" was incorporated in the State of Delaware in 2015 as Functional Software, Inc. and is gonna need some rights from me in order to utilize my contributions in this here PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Sentry can use, modify, copy, and redistribute my contributions, under Sentry's choice of terms.

Fixes SENTRY-5QZ3

@sentry <feedback>: Autofix iterates on these changes
@sentry stop iterating: Autofix stops iterating on this run

This PR was automatically generated by Sentry. You can adjust this setting at any time.

@sentry
sentry Bot requested a review from a team as a code owner September 18, 2026 22:45
@github-actions github-actions Bot added the Scope: Backend Automatically applied to PRs that change backend components label Sep 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Scope: Backend Automatically applied to PRs that change backend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants