You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Standard target encoding (MeanEncoder) replaces each category with the mean target value observed for that
category. When MeanEncoder is fitted and transformed on the same training data, every row contributes its own
target label to its encoded value. For small sample sizes or rare categories with low sample count $k$, each
sample accounts for $1/k$ of its own value (100% when $k=1$). This causes target leakage, resulting in models
memorizing training labels and over-optimistic training performance that collapses on validation or test sets.
This PR introduces **`OOFMeanEncoder`** (Out-of-Fold Mean Encoder) to solve target leakage during training:
- **`fit_transform(X, y)`**: Splits the training dataset into cross-validation folds according to `cv`. For each
fold, target encoding mappings are learned strictly using the remaining folds and applied to the held-out fold.
Consequently, no row's target value is ever used in its own encoding.
- fit(X, y): Computes and stores the target mapping across the full training dataset in encoder_dict_
and y_prior_.
- transform(X): Encodes unseen/test data using the full dataset mapping learned during fit(), matching
standard MeanEncoder behavior at inference time.
---
### Key Features & API Design
- **`cv`**: Supports an integer (default `5`), any scikit-learn CV splitter instance (`KFold`, `StratifiedKFold`,
TimeSeriesSplit, etc.), or an iterable yielding (train, test) index splits.
- smoothing: Supports 0.0 (unmodified means), positive numbers (additive smoothing towards the prior),
or 'auto' (empirical Bayes weighting).
- unseen: Strategy for unseen categories ('encode' with fold/dataset prior, 'ignore' returning np. nan with a warning, or 'raise' raising a ValueError).
- shuffle & random_state: Controls fold randomization when cv is an integer.
- stratify: Enables StratifiedKFold when cv is an integer (for discrete/classification targets).
- Standard feature-engine features: Supports variables, missing_values, ignore_format (enabling
numerical column encoding), and return_empty.
- Preserves pandas structures: Full index, column name, and dtype preservation.
---
### Example Usage
```python
import pandas as pd
from feature_engine.encoding import OOFMeanEncoder
# Training data
X_train = pd.DataFrame({"city": ["London", "London", "Paris", "Paris", "Berlin"]})
y_train = pd.Series([1, 0, 1, 1, 0])
# Initialize encoder
encoder = OOFMeanEncoder(
variables=["city"],
cv=3,
smoothing="auto",
unseen="encode",
shuffle=True,
random_state=42,
)
# Out-of-fold encoding for training (no target leakage)
X_train_oof = encoder.fit_transform(X_train, y_train)
# Inference on unseen/test data using full-dataset statistics
X_test = pd.DataFrame({"city": ["Paris", "Tokyo"]})
X_test_enc = encoder.transform(X_test)
──────
Changes Included
feature_engine/encoding/oof_mean_encoding.py:
• Implemented OOFMeanEncoder inheriting from CategoricalMethodsMixin and CategoricalInitMixinNA.
• Comprehensive docstring with math formulas, parameter documentation, and examples.
feature_engine/encoding/init.py:
• Exposed OOFMeanEncoder in public API and all.
tests/test_encoding/test_check_estimator_encoders.py:
• Added OOFMeanEncoder to scikit-learn's check_estimator and feature-engine's check_feature_engine_estimator.
docs/api_doc/encoding/OOFMeanEncoder.rst & index.rst:
• Added API documentation page and updated the encoder comparison table.
──────
Verification & Testing
• pytest tests/test_encoding/test_oof_mean_encoder.py (40 passed)
• pytest tests/test_encoding/test_check_estimator_encoders.py (52 passed, including sklearn check_estimator)
• Full test suite pytest tests/test_encoding/ (387 passed)
Hi @jusspatel, thanks for picking this up earlier -- I see we both ended up implementing it.
I'm the author of issue #1050, where the API was sketched, and I've now opened PR #1102 with an
implementation of it: OOFMeanEncoder with 46 tests, rebased on current main, with flake8, mypy and
the sphinx -W build all clean locally.
Your note about fit_transform(X) != fit(X).transform(X) for out-of-fold encoders is correct and
worth keeping visible -- I hit the same thing. I registered my class in the project's own
check_feature_engine_estimator list rather than in sklearn's check_estimator, for that reason.
Happy to share my test suite if it helps consolidate the best of both, or for @solegalli to pick
whichever fits the project best. Either way, good work.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1050
Standard target encoding ($k$ , each$1/k$ of its own value (100% when $k=1$ ). This causes target leakage, resulting in models
MeanEncoder) replaces each category with the mean target value observed for thatcategory. When
MeanEncoderis fitted and transformed on the same training data, every row contributes its owntarget label to its encoded value. For small sample sizes or rare categories with low sample count
sample accounts for
memorizing training labels and over-optimistic training performance that collapses on validation or test sets.
fold, target encoding mappings are learned strictly using the remaining folds and applied to the held-out fold.
Consequently, no row's target value is ever used in its own encoding.
-
fit(X, y): Computes and stores the target mapping across the full training dataset inencoder_dict_and
y_prior_.-
transform(X): Encodes unseen/test data using the full dataset mapping learned duringfit(), matchingstandard
MeanEncoderbehavior at inference time.TimeSeriesSplit, etc.), or an iterable yielding(train, test)index splits.-
smoothing: Supports0.0(unmodified means), positive numbers (additive smoothing towards the prior),or
'auto'(empirical Bayes weighting).-
unseen: Strategy for unseen categories ('encode'with fold/dataset prior,'ignore'returningnp. nanwith a warning, or'raise'raising aValueError).-
shuffle&random_state: Controls fold randomization whencvis an integer.-
stratify: EnablesStratifiedKFoldwhencvis an integer (for discrete/classification targets).- Standard feature-engine features: Supports
variables,missing_values,ignore_format(enablingnumerical column encoding), and
return_empty.- Preserves pandas structures: Full index, column name, and dtype preservation.
──────
Changes Included
feature_engine/encoding/oof_mean_encoding.py:
• Implemented OOFMeanEncoder inheriting from CategoricalMethodsMixin and CategoricalInitMixinNA.
• Comprehensive docstring with math formulas, parameter documentation, and examples.
feature_engine/encoding/init.py:
• Exposed OOFMeanEncoder in public API and all.
tests/test_encoding/test_oof_mean_encoder.py:
• 40 dedicated unit tests covering:
• Initialization parameter validations.
• Verification that out-of-fold encoding prevents leakage.
• Custom CV splitters (KFold, StratifiedKFold).
• Stratification behavior and regression error handling.
• Additive and 'auto' smoothing.
• Unseen category strategies ('encode', 'ignore', 'raise').
• Handling missing values ('raise' vs 'ignore').
• Numerical encoding (ignore_format=True).
• Preserving non-standard DataFrame indices.
• Pipeline integration.
• inverse_transform behavior.
tests/test_encoding/test_check_estimator_encoders.py:
• Added OOFMeanEncoder to scikit-learn's check_estimator and feature-engine's check_feature_engine_estimator.
docs/api_doc/encoding/OOFMeanEncoder.rst & index.rst:
• Added API documentation page and updated the encoder comparison table.
──────
Verification & Testing
• pytest tests/test_encoding/test_oof_mean_encoder.py (40 passed)
• pytest tests/test_encoding/test_check_estimator_encoders.py (52 passed, including sklearn check_estimator)
• Full test suite pytest tests/test_encoding/ (387 passed)