Repository navigation
Week 7 - M1: Finalize packfiles + storage performance baseline #44
Copy link
Copy link
Open
Labels
module-1Object storageObject storageweek-7Week 7 - optimization and polishWeek 7 - optimization and polish
Milestone
Description
- Owner: @mahis1067 ·
minigit/objects.py,tests/test_objects.py - Docs:
docs/packfiles.md - Depends on: Week 5 - M1: Packfile compression support #31, Week 6 - M1: Object-store fixes for push/pull round-trips #38
- Integration: coordinate the shared performance fixture with Week 7 - M4: Network diagnostics + end-to-end performance baseline #47
- Branch from updated
main:feat/packfile-finalization - Week 7 starts the roadmap's Weeks 7-8 optimization + polish phase; finish the linked prerequisites first
- Keep the object API, object hashes, and module ownership unchanged
Steps
- Finish the packfile and transfer-hardening work; check the implementation against
docs/packfiles.mdand make the document describe the actual format and packing entry point. - Close remaining gaps in packed-only, loose-only, and mixed lookup using fresh store instances.
- Loose lookup first; corrupt loose objects must not silently fall back to a packed copy
- Missing ->
ObjectNotFoundError; corrupt ->ObjectCorruptError has_objectand duplicate writes must preserve those distinctions
- Finalize validation of pack/index versions, counts, offsets, record boundaries, types, lengths, compression, and hashes. Published packs must preserve the exact bytes of blobs, trees, and commits.
- Verify temporary-file cleanup and pack-before-index publication. Interrupted publication must leave earlier objects readable; incomplete/unpublished packs must not become visible to readers.
- With M4, measure packing time, object-read time, and stored bytes on the shared moderately large fixture.
- Record fixture size, object counts, environment, and commands
- Compare loose and packed storage separately; report retained loose bytes separately from pack/index bytes
- Include fresh-store reads so a warm in-memory cache does not hide the initial cost
- Profile repeated lookups and fix a measured storage bottleneck if one is found. Any index cache must discover newly published packs and preserve corruption checks.
- Add a small regression for each fix; reuse existing pack and transfer tests. Leave pruning, delta compression, and format redesign outside this issue.
- Document reproduction commands and remaining performance findings for Week 8 in the PR;
scripts/quality-check.sh-> linked PR -> teammate review.
Tests
- Fresh-instance reads of blobs, nested trees, and two-parent commits from multiple packs
- Packed-only and mixed stores; duplicate requests and overlapping objects across packs
- Empty/binary/Unicode payloads retain the original type, bytes, and hash
- Invalid index/record, truncated data, bad zlib, wrong hash, failed publication
- Existing store instance can read a newly published pack; old objects remain readable
- Shared performance fixture plus a small correctness case in the normal test suite; no wall-clock CI threshold
Done when
- Packfile implementation and format/usage documentation agree
- Packed and mixed reads preserve validation and publication guarantees
- Reproducible storage baseline recorded; any optimization has before/after evidence
- Regression and integration checks pass
- Quality check green; PR open; review requested
Metadata
Metadata
Assignees
Labels
module-1Object storageObject storageweek-7Week 7 - optimization and polishWeek 7 - optimization and polish
Activity