Skip to content

fix(search): pass groups to the splitter and fit params to the estimator - #126

Open
ChrisW09 wants to merge 1 commit into
mainfrom
fix/91-search-groups-fit-params
Open

ChrisW09 wants to merge 1 commit into
mainfrom
fix/91-search-groups-fit-params

Conversation

@ChrisW09

@ChrisW09 ChrisW09 commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #91.

RepresentationSearchCV could not use group splitters such as GroupKFold. fit called the splitter without groups and had no groups argument, and there was no way to pass sample_weight or other fit parameters to the estimator. Grouped data (patients, sessions, sites) could not be searched with a split that keeps each group on one side.

Root cause

RepresentationSearchCV.fit(X, y) took no other arguments. It called cv.split(np.zeros(n_samples), y_arr) without groups and fitted the estimator on each fold and at the refit with fit(x, y) only.

Fix

RepresentationSearchCV.fit is now fit(X, y=None, *, groups=None, **fit_params), as in scikit-learn's searches:

  • groups is passed to cv.split.
  • fit_params are made indexable once with scikit-learn's _check_method_params. Each fold's estimator gets them restricted to the fold's training rows, and the final refit of best_estimator_ gets them whole.
  • The candidate Preprocessors and the scorer do not receive them, so target-aware placement and validation scores stay unweighted.

Behaviour changes

  • fit accepts groups and arbitrary keyword fit parameters. Calls without them behave as before.
  • Passing groups with a splitter that does not use them shows scikit-learn's usual The groups parameter is ignored by ... warning.

Documentation

  • RepresentationSearchCV.fit docstring: new Parameters and Returns sections, including groups and **fit_params and who receives them.
  • cv parameter docstring: a group splitter such as GroupKFold needs groups at fit.
  • docs/core_concepts/target_awareness.md ("Searching over representations"): a GroupKFold example with groups=patient_id, and a paragraph explaining that fit parameters such as sample_weight reach the estimator on every fold and at the refit, but not the Preprocessor or the scorer.

Tests

  • test_group_splitter_receives_groups[group_kfold], [leave_one_group_out], [group_shuffle_split]: with groups and a per-row fit parameter, each fold's estimator sees exactly the splitter's training rows, and no group is on both sides of a split. Fails on main (TypeError), passes here.
  • test_groups_leave_non_group_splits_unchanged: with a splitter that ignores groups, passing groups leaves cv_results_ unchanged. Fails on main (TypeError), passes here.
  • test_fit_params_are_split_per_fold_and_passed_to_the_refit: a per-row fit parameter is restricted to each fold's training rows and passed whole to the refit. Fails on main (TypeError), passes here.
  • test_sample_weight_reaches_the_estimator: best_estimator_ has the coefficients of a weighted Ridge fit on the best representation, and the fold scores change with the weights. Fails on main (TypeError), passes here.

Reproduction

Output of the script from #91 on this branch:

[] -> ValueError: The 'groups' parameter should not be None.
['groups'] -> ok
['sample_weight'] -> ValueError: The 'groups' parameter should not be None.

The first and third calls give GroupKFold no groups, so they now fail in the splitter, exactly as GridSearchCV does with the same arguments. The third no longer fails with a TypeError for sample_weight. Passing groups=groups, sample_weight=np.ones(300) together fits.

Checks

  • just test: 1444 passed, 59 skipped, 7 xfailed; coverage 93.65% (gate 90%).
  • Minimum dependencies (CI job: Python 3.10, numpy 1.24.4, pandas 2.0.3, scipy 1.10.1, scikit-learn 1.6.0): 1431 passed, 72 skipped, 7 xfailed; scripts/quickstart.py passes.
  • just docs: strict build succeeds.
  • just check: all pre-commit and pre-push hooks pass (ruff format, ruff lint, prettier, file hygiene, pyright with 0 errors).
  • Import direction between subpackages is unchanged.

Merge notes

This change was previously part of #97, which is closed in favour of one pull request per issue.

RepresentationSearchCV.fit took only (X, y) and called cv.split without groups,
so a group splitter (GroupKFold, LeaveOneGroupOut, GroupShuffleSplit,
StratifiedGroupKFold) always raised "The 'groups' parameter should not be None"
and fit(X, y, groups=g) raised a TypeError. Grouped data such as several rows
per patient could not be searched with a leakage-safe group split, and there
was no way to pass sample_weight or other fit parameters.

fit now takes a keyword-only groups, forwarded to cv.split, and **fit_params,
forwarded to the estimator's fit on every fold (per-sample values restricted to
the fold's training rows, as in scikit-learn's searches) and on the final
refit. The Preprocessor and the scorer do not receive them. Splits without
groups are unchanged.

Fixes #91

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

RepresentationSearchCV cannot use group-based CV splitters and accepts no groups or fit parameters

1 participant