Repository navigation
refactor: per-case store fallbacks on a copy, merge_stores (#1019 step 2b) - #1052
Merged
Merged
Conversation
This was referenced Oct 22, 2026
FabianHofmann
force-pushed
the
store/2a-store-protocol
branch
from
October 23, 2026 06:25
b4ce5ca to
5f297da
Compare
FabianHofmann
force-pushed
the
store/2b-fallbacks
branch
4 times, most recently
from
October 23, 2026 06:35
a9491a9 to
affe4d4
Compare
…ensify_all (#1020, part 2) A fallback converts a copy with a named reason and never the receiver. Dense operations call DenseStore; merge/align/concat go through linopy.store. Named sparse expressions stay sparse.
…covers every op(case)
FabianHofmann
force-pushed
the
store/2b-fallbacks
branch
from
October 23, 2026 06:57
affe4d4 to
bd44d92
Compare
Merging this PR will improve performance by 49.11%
Performance Changes
Tip Curious why performance improved? Comment Comparing Footnotes
|
Reassign the term-dim const and the factor-dim coeffs and const to their reduced form so the un-reduced concat results do not stay alive during absorb_absence; restores the peak memory of the dense merge to master's (CodSpeed var_mul_var, expr_mul_var, expr_add_masked).
FabianHofmann
marked this pull request as ready for review
October 23, 2026 07:26
5 tasks done
3 of 5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
The following content was generated by AI.
Stack 3/3, #1050 and #1051 merged; now based on master. Part 2 of 2 for #1020.
Closes #1020.
Changes proposed in this Pull Request
Step 2b of the store refactoring plan in #1019: a fallback never converts the receiver, and the frontend calls the store.
LinearExpression._fallback(reason)converts a copy with a named reason inop(case)form, for examplereindex(method=...)orwhere(non-scalar other). The receiver keeps its store..datareturns a converted copy under the reasondata(read)._densify,_densify_all,_try_csr_merge,_alignedand_concatenatedare deleted.linopy.storegainsmerge_stores,align_storesandconcat_stores, which return a store or a reason. Any sparse operand pulls a merge sparse; a reason falls the whole merge back to dense on copies.DenseStore; the inline dense bodies in the expression classes are gone.DenseStoreandCSRStoregaincombine,absorbedandcopy.Model.copy, netcdf writing andvars,coeffsorconstassignment, via a new_attrsslot.mul(quadratic product).LinearExpression.whereandQuadraticExpression.wheretakecond,otheranddroponly, matching xarray. The extra keyword arguments are gone.sumwith an unknown dimension raisesKeyErrorwithout densifying first.op(case)reason and asserts each has a row; new rows for products with constants,wherewith new dimensions, the override merge,getitemand division over a new dimension, andassignment(coeffs).Deferred to step 3 (#1045):
sel,isel,__getitem__,shift,roll,where,drop_varsand densereindexstill use the dense xarray code, because the grid rejects 2-D auxiliary coordinates and the grid path changes a coordinate dtype. A named sparse expression still reads back dense from netcdf until step 4c (#1048).Deferred cleanups from review:
_off_gridduplicates theDenseStore.gridcheck; the dense constant-combine exists inDenseStore.combineand in the base class for quadratic expressions;_printout_cellcould callstore.cell; the dense branches offlat,to_polarsandlinear_termsrepeatDenseStore.termsbut differ on absent cells;Variable.wherestill takes extra keyword arguments.Checklist
AGENTS.md).doc.doc/release_notes.rstof the upcoming release is included.