Skip to content

fix(preprocessor): accept input_features after fitting on a NumPy array - #108

Open
ChrisW09 wants to merge 1 commit into
mainfrom
fix/65-input-features-after-ndarray-fit
Open

ChrisW09 wants to merge 1 commit into
mainfrom
fix/65-input-features-after-ndarray-fit

Conversation

@ChrisW09

@ChrisW09 ChrisW09 commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #65.

After a fit on a NumPy array, Preprocessor.get_feature_names_out(input_features) raised input_features is not equal to feature_names_in_, even for names of the right length. As a result, Pipeline.get_feature_names_out() failed whenever Preprocessor followed a step that outputs arrays, such as SimpleImputer with default output.

Root cause

Preprocessor.fit names array columns feature_0, feature_1, ... before fitting the inner ColumnTransformer, and get_feature_names_out forwarded input_features to that transformer unchanged. The ColumnTransformer holds the synthetic labels as its feature_names_in_, so it accepted only exactly those names. Preprocessor itself never set feature_names_in_, so it could not apply scikit-learn's rule that an estimator fitted without feature names accepts any input_features of the right length.

Fix

  • Preprocessor.fit records feature_names_in_ (an object array) when the input is not a NumPy array and all its column labels are strings, and deletes it on a refit on an array.
  • Preprocessor.get_feature_names_out checks that input_features has n_features_in_ entries and, when feature_names_in_ is defined, that it equals feature_names_in_. Without feature_names_in_, any names of the right length are accepted.
  • New pretab.compose.inspection.feature_names_out builds the output names. For input_features after an array fit, it maps each fitted label to the given name and rebuilds every step's names from its fitted transformer with the new labels. _without_feature_names hands that call a shallow copy of the step, and of nested Pipeline and FeatureUnion steps, without feature_names_in_. scikit-learn then applies only its length check, and the fitted steps are left untouched.
  • clean_feature_names takes a keyword-only rename mapping and reports a renamed column's step under the new name. An array fit with input_features=["age", "income"] therefore gives the same names as a fit on those columns (num_age, not num_feature_0). Without rename, its output is unchanged.

Behaviour changes

  • A Preprocessor fitted on a DataFrame or dict whose column labels are all strings now has a feature_names_in_ attribute. A refit on an array removes it.
  • After an array fit, get_feature_names_out(input_features) accepts any n_features_in_ names and returns names built from them.
  • input_features of the wrong length raises scikit-learn's ValueError: input_features should have length equal to number of features (n), got k instead of input_features is not equal to feature_names_in_.
  • to_spec() state and fingerprint_ of newly fitted preprocessors with string column labels change, because the fitted state now includes feature_names_in_. The golden baselines in tests/regression/ pass unchanged. A preprocessor saved by 1.0.0 still loads; it has no feature_names_in_ and therefore accepts any input_features of the right length.

Documentation

  • Preprocessor class docstring, Attributes: new feature_names_in_ entry, defined only when every column label is a string.
  • Preprocessor.get_feature_names_out docstring: input_features follows scikit-learn's convention; new Raises section for a wrong length or names that differ from feature_names_in_.
  • Docstrings of the new feature_names_out and _without_feature_names, and of the rename argument of clean_feature_names.
  • docs/core_concepts/outputs_and_inspection.md ("Feature names"): explains the convention and shows make_pipeline(SimpleImputer(), Preprocessor(numerical_method="minmax")).get_feature_names_out() returning ['num_age', 'num_income'].

Tests

  • test_array_fit_uses_the_given_input_features (4 cases: minmax, polynomial, bspline with missing_policy="separate_state", ple with add_missing_indicator=True, all with missing values): an array fit with input_features=["age", "income"] gives the same names as a DataFrame fit. Fails on main, passes here.
  • test_array_fit_rejects_input_features_of_the_wrong_length: scikit-learn's length message. Fails on main, passes here.
  • test_named_fit_records_and_validates_feature_names_in: a DataFrame fit records feature_names_in_, matching names are accepted, other names raise, and a refit on an array removes the attribute. Fails on main, passes here.
  • test_pipeline_feature_names_propagate_through_an_array_step: make_pipeline(SimpleImputer(), Preprocessor(...)).get_feature_names_out() returns ["num_age", "num_income"]. Fails on main, passes here.

Reproduction

Output of the script from #65 on this branch:

['age' 'income']
['num_age' 'num_income']
['num_age' 'num_income']

Checks

  • just test: 1445 passed, 59 skipped, 7 xfailed; coverage 93.58% (gate 90%).
  • Minimum dependencies (CI job: Python 3.10, numpy 1.24.4, pandas 2.0.3, scipy 1.10.1, scikit-learn 1.6.0): 1432 passed, 72 skipped, 7 xfailed; scripts/quickstart.py passes.
  • just docs: strict build succeeds.
  • just check: all pre-commit and pre-push hooks pass (ruff format, ruff lint, prettier, file hygiene, pyright with 0 errors).
  • Import direction between subpackages is unchanged.

Merge notes

This change was previously part of #75, which is closed in favour of one pull request per issue.

Preprocessor forwarded input_features to its ColumnTransformer, which
was fitted on the synthetic feature_0, feature_1, ... names of array
input and so required exactly those names. That broke
Pipeline.get_feature_names_out() whenever an upstream step, such as
SimpleImputer, outputs arrays.

Following scikit-learn's convention, Preprocessor now records
feature_names_in_ only when fitted on string-labelled columns and keeps
validating input_features against it. Without it, any n_features_in_
names are accepted: each step's output names are rebuilt from its
fitted transformer with the given labels (on shallow copies without
their recorded feature names), so they match the names a fit on named
columns would produce, e.g. num_age instead of num_feature_0.

Behaviour change: a Preprocessor fitted on a DataFrame or dict with
string column labels now has a feature_names_in_ attribute, which also
changes the fingerprint_ of newly fitted preprocessors.

Fixes #65

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Preprocessor.get_feature_names_out(input_features) fails after fitting on an ndarray, breaking Pipeline.get_feature_names_out()

1 participant