Repository navigation
Fix crashes on degenerate ROC curves and invalid input (2.0.5) - #2
Merged
Merged
Conversation
- Replace scipy.spatial.ConvexHull with a monotone-chain upper hull from (0,0) to (1,1). Qhull raised QhullError when the ROC curve had fewer than three points or all points on one line (constant scores, a diagonal ROC with ties). The hull uses the integer counts of false and true positives, so the collinearity test is exact and the hull vertices are the same as before on valid input. - Raise ValueError instead of calling sys.exit. - Raise a clear ValueError when the labels contain only one class. - Drop the check that scores lie in [0,1]. EMP depends only on the order of the scores. Raise ValueError for NaN, infinite or non-numeric scores. - Raise ValueError for inputs that are not one-dimensional. - Suppress the divide-by-zero RuntimeWarning from a horizontal last hull segment. Its lambda (or gamma) is infinite and the next line drops it. - Replace the Python loops in the input checks with numpy operations.
…fault The README said that the functions print by default, but print_output defaults to False. Also state the label and score conventions, and that invalid input raises ValueError.
Cover the README and test.py examples, random and credit-like data against a brute-force computation of EMP from its definition, the edge cases that failed in 2.0.4, input types, and warnings.
…lishing A published GitHub release runs the tests, builds the sdist and wheel, checks that the tag matches the package version, and uploads to PyPI after a reviewer approves the pypi environment. No API token is stored. The one-time setup on PyPI and GitHub is in the comments at the top of the workflow.
…requests The workflow runs pytest on Python 3.10 to 3.15 on Linux, on Python 3.15 on Windows and macOS, and with numpy 1.x on Python 3.12. The publish workflow now calls it instead of running its own test job.
setup.py gives the license as an SPDX expression instead of the deprecated classifier, lists pandas once, drops setup_requires and calls setup() outside the with block.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This pull request fixes how EMP-PY 2.0.4 fails on degenerate or invalid input, and it prepares version 2.0.5. Results on valid input do not change. It also adds a workflow that runs the tests on pushes and pull requests, and a workflow that publishes to PyPI with Trusted Publishing.
Changes
EMP/metrics.py:scipy.spatial.ConvexHullwith a monotone-chain upper hull from (0,0) to (1,1). Qhull raisedQhullErrorwhen the ROC curve had fewer than three points or all its points on one line, for example with constant scores orempCreditScoring([0.3, 0.3, 0.7, 0.7], [0, 1, 0, 1]). The hull uses the integer counts of false and true positives, so the test for collinear points is exact.empChurnuses the same helper.ValueErrorinstead of callingsys.exit, which ended the Python process.ValueErrorwhen the labels contain only one class. Version 2.0.4 failed withValueError: Points cannot contain NaN.ValueError.ValueErrorfor input that is not one-dimensional.RuntimeWarningfrom a horizontal last hull segment. Its lambda (or gamma) is infinite, and the existing< 1filter drops it.empCreditScoringhowEMPC_fractionis chosen when two cutoffs tie at lambda = 1.Other files:
README.md: the functions return the output and do not print it by default. The README now states the label and score conventions.tests/: 54 pytest tests.tests/brute_force.pycomputes EMP from its definition on a grid, without the hull.setup.py,README.md,NEWS: version 2.0.5 and a changelog.setup.py: require Python 3.10 or later (was 3.8). Give the license as the SPDX expressionGPL-3.0-or-laterinstead of the deprecated classifier, listpandasonce, and dropsetup_requires..github/workflows/tests.yml: runs the tests on each push tomainand on each pull request. The matrix is Python 3.10 to 3.15 on Linux, Python 3.15 on Windows and macOS, and numpy 1.x on Python 3.12..github/workflows/publish.yml: a published GitHub release runs the tests (throughtests.yml), builds the package, checks that the tag matches the version, and uploads to PyPI after a reviewer approves thepypienvironment.Results on valid input
The PyPI 2.0.4 wheel and this branch ran the same 6,030 calls: 1,005 data sets with 6 parameter settings. Both used numpy 2.5.4, scipy 1.18.1, scikit-learn 1.9.1 and pandas 3.1.0 on Python 3.14.6.
QhullError) and work in this branchRuntimeWarning: divide by zero, 2.0.4 / this branchAgainst brute force on 4,850 calls, EMPC and EMP agree within 2.3e-7, which is the grid error.
Behavior changes
ValueErrorinstead ofSystemExit.ValueError. Version 2.0.4 flattened nested lists of any shape, so[[0.1, 0.2], [0.3, 0.4]]was read as four scores.KeyError.Known limitation, not changed
Sometimes two ROC points tie exactly for the maximum profit at lambda = 1. Then the fraction at the p1 point mass is not unique. Version 2.0.4 and this branch both choose by how lambda rounds near 1. In 15 small-sample cases with ROI = 0.5, the fraction differs from brute force by up to 0.06. EMPC is not affected. This pull request keeps the 2.0.4 behavior and documents it in the comments of
empCreditScoring. In all 165 test calls with an exact tie, the choice follows the documented rule.Tests
All 54 tests pass on this branch and on the built wheel in a clean environment. On 2.0.4, 19 fail, which are the edge cases and the new behavior. The Tests workflow runs them on this pull request.
Before the first release with the workflow
Banking-Analytics-Lab, repositoryEMP-Py, workflowpublish.ymland environmentpypi.pypiwith at least one required reviewer.mainwith the tagv2.0.5.Not in this pull request
metrics.pystill importspandasbut does not use it, sopandasstays a requirement.DESCRIPTION,man/andinst/CITATIONare files from the R package.