Skip to content

fix(placement): rank CART splits by impurity decrease before spacing - #110

Open
ChrisW09 wants to merge 1 commit into
mainfrom
fix/70-cart-split-ranking
Open

ChrisW09 wants to merge 1 commit into
mainfrom
fix/70-cart-split-ranking

Conversation

@ChrisW09

@ChrisW09 ChrisW09 commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #70.

CARTLocationSelector could drop the most informative split when a weak split lay just below it, so the default Preprocessor() (target-aware PLE) and every other CART-placed representation could lose the bin edge, center or knot where the target changes, on continuous, untied data.

Root cause

CARTLocationSelector returned its split candidates in ascending location order (sorted(set(...))), and BaseLocationSelector._enforce_spacing keeps the first of two candidates that are closer than min_location_spacing. A weak split just below the root split therefore evicted it, and the importance-based trim then usually removed the weak neighbour as well.

Fix

  • CARTLocationSelector._split_importance maps every in-range split threshold of the fitted tree to its weighted impurity decrease. It replaces _extract_split_points and _select_top_locations.
  • CARTLocationSelector._ordered_candidates returns the candidates ranked by decreasing impurity decrease (ties by location, _rank), as the LightGBM selector already does with gains, so the spacing filter keeps the stronger of two close splits. The trimming context is now the importance map instead of the fitted tree.
  • CARTLocationSelector._trim_over_max uses the same ranking and returns the kept locations sorted. select still returns locations in ascending order.

Behaviour changes

  • CART-placed locations (PLE thresholds, target-aware feature-map centers, CART-placed spline knots) can move when two candidate splits are closer than min_location_spacing (1% of the range by default): the more informative one now survives. On the issue's simulation (200 single-step datasets), PLETransformer(output_dim=7) now misses the root split in 0 of 200 datasets, down from 25.
  • The golden regression baselines pass unchanged.

Documentation

  • Docstrings: the CARTLocationSelector class docstring now says the impurity ranking decides both the spacing and the trim. The private BaseLocationSelector._ordered_candidates and _enforce_spacing docstrings describe the importance-descending order both selectors produce and which of two close candidates is kept.
  • No docs page changes: docs/core_concepts/resolution_and_placement.md describes CART placement as "split points from a per-feature decision tree fit against y", which still holds. The class docstring already promised that "the most informative splits are kept", and the code now does that.

Tests

  • test_cart_spacing_keeps_the_dominant_split_over_a_weak_neighbour: on the issue's seed-34 dataset the root split of a depth-1 tree is among the 6 selected locations. Fails on main, passes here.
  • test_cart_keeps_the_root_split_of_a_single_step_target[0-9]: the root split survives on ten single-step datasets. Seeds 2 and 5 fail on main, all ten pass here; the other eight pass on both and pin the sweep.
  • test_cart_candidates_are_ordered_by_impurity_decrease: _ordered_candidates returns candidates in non-increasing impurity-decrease order. Fails on main, passes here.

Reproduction

Output of the script from #70 on this branch, followed by the simulation script from the issue (LeakageWarnings omitted):

true step: 4.696  root split of a depth-1 tree: 4.71
RBF centers   : [2.215 2.444 2.982 3.822 4.71  7.17 ]
PLE thresholds: [2.215 2.444 2.982 3.822 4.71  7.17 ]
0 of 200 datasets miss the root split

Checks

  • just test: 1450 passed, 59 skipped, 7 xfailed; coverage 93.68% (gate 90%).
  • Minimum dependencies (CI job: Python 3.10, numpy 1.24.4, pandas 2.0.3, scipy 1.10.1, scikit-learn 1.6.0): 1437 passed, 72 skipped, 7 xfailed; scripts/quickstart.py passes.
  • just docs: strict build succeeds.
  • just check: all pre-commit and pre-push hooks pass (ruff format, ruff lint, prettier, file hygiene, pyright with 0 errors).
  • Import direction between subpackages is unchanged.

Merge notes

This change was previously part of #75, which is closed in favour of one pull request per issue.

CARTLocationSelector returned its split candidates in ascending location
order, and the minimum-spacing filter keeps the first of two candidates that
are closer than min_location_spacing. A weak split just below the dominant
one therefore evicted it, and the importance-based trim then usually removed
the weak neighbour as well, leaving no location where the target changes.

Candidates are now ordered by weighted impurity decrease (as the LightGBM
selector already does with gains), so the spacing filter keeps the stronger
of two close splits, and the same ranking drives the over-max trim. The
returned locations are still sorted by location.

Fixes #70

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CART location selector can drop the dominant split because spacing is enforced in ascending location order

1 participant