diff --git a/.github/AGENTS.md b/.github/AGENTS.md new file mode 100644 index 00000000..19d0a307 --- /dev/null +++ b/.github/AGENTS.md @@ -0,0 +1,19 @@ +# AGENTS.md — .github/ + +CI/CD workflows and repo automation. + +## Workflows (source of truth) +- `conda-package.yml` — Intel-channel conda build and test +- `conda-package-cf.yml` — conda-forge conda build and test +- `build_pip.yml` — editable pip build, including pre-release NumPy +- `build-with-clang.yml` — build with `icx`/`icpx` from the oneAPI apt repository +- `build-with-standard-clang.yml` — build with upstream clang +- `build-docs.yml` — Sphinx build; publishes to `gh-pages` from `master` +- `pre-commit.yml` — lint/format checks +- `coverity.yml` — Coverity static analysis (see `coverity/README.md`) +- `openssf-scorecard.yml` — OpenSSF Scorecard +- `zizmor.yml` — GitHub Actions security lint + +## Policy +- Treat workflow YAML as canonical for platform/Python matrices. +- Avoid doc claims about CI coverage unless present in workflow config. diff --git a/.github/ISSUE_TEMPLATE/bug_report.yml b/.github/ISSUE_TEMPLATE/bug_report.yml new file mode 100644 index 00000000..d9a64c6b --- /dev/null +++ b/.github/ISSUE_TEMPLATE/bug_report.yml @@ -0,0 +1,66 @@ +name: Bug report +description: Report incorrect results, a crash, or a build failure +labels: ["bug"] +body: + - type: markdown + attributes: + value: | + For security vulnerabilities, do not open an issue — follow + [SECURITY.md](https://github.com/IntelPython/mkl_random/blob/master/SECURITY.md). + + - type: textarea + id: description + attributes: + label: Description + description: What happened, and what did you expect instead? + validations: + required: true + + - type: textarea + id: reproducer + attributes: + label: Reproducer + description: | + A minimal, self-contained snippet. Include the seed, the `brng`, the + distribution and its arguments, and `size`. + render: python + validations: + required: true + + - type: dropdown + id: install-source + attributes: + label: How was `mkl_random` installed? + options: + - Intel conda channel (software.repos.intel.com) + - conda-forge + - pip, Intel index (software.repos.intel.com) + - pip / PyPI + - Built from source + validations: + required: true + + - type: textarea + id: versions + attributes: + label: Versions + description: | + Output of: + ``` + python -c "import mkl_random, numpy; print(mkl_random.__version__); print(numpy.__version__)" + ``` + Add your OS and Python version too. + render: shell + validations: + required: true + + - type: textarea + id: notes + attributes: + label: Anything else + description: | + Optional. Whether NumPy patching was active (`mkl_random.is_patched()`), + whether you used `mkl_random.interfaces.numpy_random`, or a non-default + `MKL_NUM_THREADS`. + validations: + required: false diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml new file mode 100644 index 00000000..378931b5 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/config.yml @@ -0,0 +1,8 @@ +blank_issues_enabled: true +contact_links: + - name: Security vulnerability + url: https://www.intel.com/content/www/us/en/security-center/vulnerability-handling-guidelines.html + about: Report security vulnerabilities through Intel's process, not a public issue. + - name: Question about usage + url: https://intelpython.github.io/mkl_random/ + about: Check the documentation and README first. diff --git a/.github/ISSUE_TEMPLATE/feature_request.yml b/.github/ISSUE_TEMPLATE/feature_request.yml new file mode 100644 index 00000000..393e3d87 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/feature_request.yml @@ -0,0 +1,38 @@ +name: Feature request +description: Propose a new distribution, generator, or capability +labels: ["enhancement"] +body: + - type: textarea + id: problem + attributes: + label: What problem does this solve? + description: The use case, not the implementation. + validations: + required: true + + - type: textarea + id: proposal + attributes: + label: Proposal + description: | + What you would like `mkl_random` to do. If it mirrors a legacy + `numpy.random` function, name it — its signature is the contract for + the NumPy interface. + validations: + required: true + + - type: input + id: upstream + attributes: + label: Upstream equivalent + description: Link to the NumPy or oneMKL docs for the equivalent, if there is one. + validations: + required: false + + - type: textarea + id: alternatives + attributes: + label: Alternatives considered + description: Optional. Workarounds you are using today. + validations: + required: false diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md new file mode 100644 index 00000000..f049f55e --- /dev/null +++ b/.github/pull_request_template.md @@ -0,0 +1,27 @@ +# Description + + + +## Verification + + + +- Tests: +- Lint: + +## Not verified + + + +## Checklist + +- [ ] Legacy `numpy.random` API compatibility preserved, or the break is intentional and called out above. +- [ ] Seeded output is unchanged, or the `CHANGELOG.md` entry says which streams changed. +- [ ] Behavior changes have tests in `mkl_random/tests/`; bug fixes have a regression test. +- [ ] `CHANGELOG.md` updated under `[dev]` with a `[gh-NNN]` link, or the change isn't user-visible. + + diff --git a/.gitignore b/.gitignore index bd23b381..3318944d 100644 --- a/.gitignore +++ b/.gitignore @@ -9,5 +9,14 @@ mkl_random/mklrand.cpp # Byte-compiled / optimized / DLL files __pycache__/ +# Virtual environments, test caches, local env files +.venv/ +venv/ +.pytest_cache/ +.env + # ASV benchmark artifacts .asv/ + +# Developer-local coding agent settings +.claude/settings.local.json diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 00000000..a9e70fe6 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,60 @@ +# AGENTS.md — mkl_random + +Entry point for agent context in this repo. + +## What this project is +`mkl_random` is a NumPy-based interface to Intel® oneMKL random number +generation. `MKLRandomState` exposes the distributions of legacy +`numpy.random.RandomState` with a choice of MKL basic generators (`brng`). +`mkl_random.interfaces.numpy_random` is a drop-in replacement for legacy +`numpy.random`, and runtime patching can install it in NumPy's place. + +## Key components +- **Package and public API:** `mkl_random/`, `mkl_random/__init__.py` +- **Cython extension:** `mkl_random/mklrand.pyx` +- **C++ kernels:** `mkl_random/src/` +- **NumPy interface:** `mkl_random/interfaces/` +- **Patching:** `_patch_numpy.py`, plus persistent and one-shot patching in + `patch.py`, `with_patch.py`, `_patch_startup.py`, and the `__main__.py` CLI +- **Tests:** `mkl_random/tests/` +- **Docs:** `docs/` (Sphinx) +- **Packaging:** `conda-recipe/`, `conda-recipe-cf/` +- **Benchmarks:** `benchmarks/` + +## Build/runtime basics +- Build system: `pyproject.toml` + `meson.build` +- Build deps: `mkl-devel`, `numpy`, `meson-python`, `cmake`, `ninja`, `cython`, + and a C++ compiler +- Runtime deps: `numpy`; the conda recipes add MKL +- Setup, checks, and style: `CONTRIBUTING.md` +- Single test: `pytest mkl_random/tests/::` +- Single-file lint: `pre-commit run --files ` + +## Development guardrails +- Keep signatures and accepted arguments compatible with legacy `numpy.random`. +- Seeded output is part of the contract: a change to what a fixed seed + produces needs a CHANGELOG entry naming the affected streams. +- Edit `mklrand.pyx` and `src/`, not the Cython-generated C++. +- Keep patching reversible, with `is_patched()` reporting the truth. +- Keep the extension free-threading compatible. +- Pair behavior changes with tests and keep diffs minimal. +- Avoid hardcoding mutable versions/matrices/channels in docs. + +## Where truth lives +- Build/config: `pyproject.toml`, `meson.build` +- Dependencies: `pyproject.toml`, `conda-recipe*/meta.yaml` +- CI/workflows: `.github/workflows/*.yml` +- Public API: `mkl_random/__init__.py`, `mkl_random/interfaces/numpy_random.py` +- Tests: `mkl_random/tests/` + +## Directory map +Use nearest local `AGENTS.md` when present: +- `.github/AGENTS.md` — CI workflows and automation policy +- `mkl_random/AGENTS.md` — package modules, the extension, and generators +- `mkl_random/src/AGENTS.md` — C++ kernels +- `mkl_random/interfaces/AGENTS.md` — `numpy.random` drop-in interface +- `mkl_random/tests/AGENTS.md` — test scope and conventions +- `docs/AGENTS.md` — Sphinx documentation +- `conda-recipe/AGENTS.md` — Intel-channel conda packaging +- `conda-recipe-cf/AGENTS.md` — conda-forge recipe +- `benchmarks/AGENTS.md` — ASV performance suite diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 00000000..d43ecf9d --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,46 @@ +# Contributing to `mkl_random` + +See [README.md](README.md) for usage, [AGENTS.md](AGENTS.md) for a map of the +source tree, and [SECURITY.md](SECURITY.md) to report a vulnerability. + +## Setup + +Building needs a C++ compiler (`g++`, `clang++`, or `icpx`). + +```sh +conda create -n mkl_random-dev -c conda-forge python pip mkl-devel numpy \ + meson-python ninja cmake "cython>=3.1.0" pytest +conda activate mkl_random-dev +pip install -e . --no-build-isolation --no-deps +``` + +To build the docs, install `sphinx`, `furo`, `sphinx-design`, and +`sphinxcontrib-programoutput`, then run +`sphinx-build -M html docs/source docs/build`. + +## Checks + +```sh +pytest mkl_random/tests +pre-commit run --all-files +``` + +The pre-commit hooks enforce formatting; otherwise, match the surrounding code. + +## Guidelines + +- Keep changes small and focused. +- Keep signatures compatible with legacy `numpy.random`. +- If a change alters what a fixed seed produces, say which streams changed in + the `CHANGELOG.md` entry. +- Add tests with behavior changes, and a regression test with bug fixes. Seed + every generator a test uses. +- Edit `mklrand.pyx` and `src/`, not generated C++. +- Keep patching reversible. + +## Pull requests + +Work on a branch and fill in the PR template. For user-visible changes, add a +`CHANGELOG.md` entry under `[dev]` with a `gh-NNN` link. + +Contributions are licensed under the terms in [LICENSE.txt](LICENSE.txt). diff --git a/README.md b/README.md index ce6d04ba..9ed65604 100644 --- a/README.md +++ b/README.md @@ -158,3 +158,10 @@ then build against the existing installation with: ```sh python -m pip install --no-build-isolation --no-deps . ``` + +--- +# Contributing + +See [CONTRIBUTING.md](CONTRIBUTING.md) for the development workflow: setting up a +build environment, running the tests and lint hooks, code style, and what to +include in a pull request. diff --git a/SECURITY.md b/SECURITY.md new file mode 100644 index 00000000..556938bd --- /dev/null +++ b/SECURITY.md @@ -0,0 +1,12 @@ +# Security Policy + +## Report a Vulnerability + +Please report security issues or vulnerabilities to the [Intel® Security Center]. + +For more information on how Intel® works to resolve security issues, see +[Vulnerability Handling Guidelines]. + +[Intel® Security Center]:https://www.intel.com/content/www/us/en/security-center/default.html + +[Vulnerability Handling Guidelines]:https://www.intel.com/content/www/us/en/security-center/vulnerability-handling-guidelines.html diff --git a/benchmarks/AGENTS.md b/benchmarks/AGENTS.md new file mode 100644 index 00000000..335563e0 --- /dev/null +++ b/benchmarks/AGENTS.md @@ -0,0 +1,22 @@ +# AGENTS.md — benchmarks/ + +ASV performance suite for `mkl_random`. + +## Scope +- `asv.conf.json` — ASV configuration, channels, and regression thresholds +- `benchmarks/` — benchmark modules plus shared helpers in `_utils.py` +- `README.md` — coverage table, threading default, measurement notes, and run + commands + +## Guardrails +- Treat `asv.conf.json` as canonical for ASV settings; treat `README.md` as + canonical for what each module covers and how measurements are taken. +- Comparability across machines depends on the thread default in + `benchmarks/__init__.py`, the fixed seeds, and the warmup calls in the timing + benchmarks' `setup`. Changing any of them invalidates comparison against + existing results — call it out explicitly. +- Follow the engine and `method=` restrictions in `README.md` when adding + benchmarks. +- Report performance numbers with reproducible context: hardware, thread count, + versions, and the command used. +- Results under `benchmarks/.asv/` are local artifacts. diff --git a/conda-recipe-cf/AGENTS.md b/conda-recipe-cf/AGENTS.md new file mode 100644 index 00000000..d1603dfb --- /dev/null +++ b/conda-recipe-cf/AGENTS.md @@ -0,0 +1,14 @@ +# AGENTS.md — conda-recipe-cf/ + +conda-forge variant of the conda recipe. + +## Differences from `conda-recipe/` +- Installs with `pip install` directly; no wheel build or retag +- Pins NumPy with `pin_compatible` at run time and runs `pip check` in the + package test +- Adds macOS compiler entries to `conda_build_config.yaml` + +## Guardrails +- Keep conda-forge recipe semantics separate from the Intel-channel recipe. +- Keep changes in step with `.github/workflows/conda-package-cf.yml`, which + builds and tests this recipe. diff --git a/conda-recipe/AGENTS.md b/conda-recipe/AGENTS.md new file mode 100644 index 00000000..97e7fa37 --- /dev/null +++ b/conda-recipe/AGENTS.md @@ -0,0 +1,15 @@ +# AGENTS.md — conda-recipe/ + +Intel-channel conda packaging. + +## Files +- `meta.yaml` — package metadata, dependencies, and the package test + (`pytest --pyargs mkl_random`) +- `build.sh` / `bld.bat` — build a wheel with `python -m build`, then install + it; `build.sh` also retags the wheel's platform +- `conda_build_config.yaml` — compiler and C library pins + +## Guardrails +- Treat recipe files as canonical for packaging intent and dependency pins. +- Keep recipe changes in step with `.github/workflows/conda-package.yml`, which + builds and tests this recipe. diff --git a/docs/AGENTS.md b/docs/AGENTS.md new file mode 100644 index 00000000..dfd1b96a --- /dev/null +++ b/docs/AGENTS.md @@ -0,0 +1,17 @@ +# AGENTS.md — docs/ + +Sphinx sources for the published documentation. + +## Scope +- `source/` — pages; `source/conf.py` — Sphinx configuration +- `source/reference/` — API reference, with one page per basic generator +- `source/maintenance/index.rst` — the contributor page; keep its commands in + step with `CONTRIBUTING.md` + +## Guardrails +- `release` in `source/conf.py` is set by hand; keep it equal to + `mkl_random/_version.py`. +- Build locally with `sphinx-build -M html docs/source docs/build` before + changing `conf.py` or adding extensions. +- `.github/workflows/build-docs.yml` builds the docs on pull requests (as an + artifact) and publishes them to `gh-pages` on pushes to `master`. diff --git a/docs/source/maintenance/index.rst b/docs/source/maintenance/index.rst index e54b6574..ba2b1347 100644 --- a/docs/source/maintenance/index.rst +++ b/docs/source/maintenance/index.rst @@ -1,7 +1,7 @@ Contributing ============ -:mod:`mkl_random` is an free and open source project. +:mod:`mkl_random` is a free and open source project. We welcome and appreciate your contributions. To contribute, fork the repo https://github.com/IntelPython/mkl_random.git, @@ -32,20 +32,27 @@ package and located through CMake, so no environment variable needs to be set. .. code-block:: bash :caption: Building mkl_random - python -m pip install . --no-build-isolation --no-deps + python -m pip install -e . --no-build-isolation --no-deps -To run test suite, install :mod:`pytest`, and run +The editable (``-e``) install builds the extension in place, so the test suite +below runs against it from the repository root. + +To run the test suite, install :mod:`pytest`, and run .. code-block:: bash :caption: Running mkl_random test suite python -m pytest mkl_random/tests -To build documentation, install dependencies and running +To build documentation, install :mod:`sphinx`, :mod:`furo`, :mod:`sphinx-design`, +and :mod:`sphinxcontrib-programoutput`, and run .. code-block:: bash :caption: Building mkl_random documentation - $ sphinx-build -M html docs/source docs/build + sphinx-build -M html docs/source docs/build Rendered documentation can be found in "docs/build/html". + +For code style, the pre-commit hooks, and what to include in a pull request, +see ``CONTRIBUTING.md`` in the repository. diff --git a/meson.build b/meson.build index 2ff5a1c1..b076ad55 100644 --- a/meson.build +++ b/meson.build @@ -111,5 +111,6 @@ py.install_sources( install_subdir( 'mkl_random/tests', - install_dir: py.get_install_dir() / 'mkl_random' + install_dir: py.get_install_dir() / 'mkl_random', + exclude_files: ['AGENTS.md'] ) diff --git a/mkl_random/AGENTS.md b/mkl_random/AGENTS.md new file mode 100644 index 00000000..4c4f81c7 --- /dev/null +++ b/mkl_random/AGENTS.md @@ -0,0 +1,28 @@ +# AGENTS.md — mkl_random/ + +Package sources: the Cython extension, public API, and patching entry points. + +## Key files +- `mklrand.pyx` — the `mklrand` extension: `MKLRandomState`, the + distributions, and the basic-generator registry +- `__init__.py` — public API surface +- `_patch_numpy.py` — `patch_numpy_random()`, `restore_numpy_random()`, + `is_patched()`, and the `mkl_random()` context manager +- `patch.py`, `_patch_startup.py`, `with_patch.py`, `__main__.py` — persistent + (`.pth`) and one-shot patching behind `python -m mkl_random` +- `_version.py` — the version; `meson.build` reads it + +## Guardrails +- Use `MKLRandomState` for the MKL-specific API and + `interfaces.numpy_random.RandomState` for the NumPy drop-in in new code and + docs; `mkl_random.RandomState` is deprecated. +- Edit `mklrand.pyx`; Cython generates the C++ into the build directory. +- Adding a basic generator touches every registration point: the enum in + `src/randomkit.h` and its mirror in `mklrand.pyx`, `brng_list` in + `src/randomkit.cpp`, `_brng_dict` and `_brng_dict_stream_max` in + `mklrand.pyx`, the `brng` docstrings, the list in `README.md`, and a page + under `docs/source/reference/`. The enum value indexes `brng_list`, so keep + the order in sync. +- Keep `freethreading_compatible=True` in `mklrand.pyx`. +- New modules must be listed in `py.install_sources` in `meson.build`, or they + are not installed. diff --git a/mkl_random/interfaces/AGENTS.md b/mkl_random/interfaces/AGENTS.md new file mode 100644 index 00000000..d1fa1b44 --- /dev/null +++ b/mkl_random/interfaces/AGENTS.md @@ -0,0 +1,13 @@ +# AGENTS.md — mkl_random/interfaces/ + +Drop-in replacement for the legacy part of `numpy.random`. + +## Scope +- `numpy_random.py` — public module; `_numpy_random.py` — implementation +- `README.md` — the list of classes and functions the interface covers + +## Guardrails +- Legacy `numpy.random` signatures and semantics are the contract here. +- Keep `README.md` in sync when the covered set changes. +- Patching installs this interface in NumPy's place, so changes here reach + patched NumPy users too; cover them with tests in `mkl_random/tests/`. diff --git a/mkl_random/src/AGENTS.md b/mkl_random/src/AGENTS.md new file mode 100644 index 00000000..129af31e --- /dev/null +++ b/mkl_random/src/AGENTS.md @@ -0,0 +1,18 @@ +# AGENTS.md — mkl_random/src/ + +Hand-written C++11 kernels compiled into the `mklrand` extension. + +## Key files +- `randomkit.cpp`, `randomkit.h` — stream state and basic-generator setup + (`brng_list`) +- `mkl_distributions.cpp`, `mkl_distributions.h` — distribution kernels on MKL + VS +- `mklrand_py_helper.h` — Python/NumPy helpers for the extension +- `numpy_multiiter_workaround.h` — needed only to build against NumPy < 2.0 + (numpy/numpy#26990) + +## Guardrails +- Coordinate kernel changes with their callers in `mklrand.pyx`. +- A kernel change that alters what a fixed seed produces is user-visible; call + it out in the CHANGELOG. +- Keep code C++11; `meson.build` sets the standard. diff --git a/mkl_random/tests/AGENTS.md b/mkl_random/tests/AGENTS.md new file mode 100644 index 00000000..8bb77620 --- /dev/null +++ b/mkl_random/tests/AGENTS.md @@ -0,0 +1,21 @@ +# AGENTS.md — mkl_random/tests/ + +Test suite for the generators, the NumPy interface, and patching. +`meson.build` installs it with the package. + +## Files +- `test_random.py` — distributions and generators +- `test_patch.py` — patch and restore state +- `test_cli.py` — persistent patch install, uninstall, and status +- `test_freethreading.py` — concurrent use; the race tests run only on a + free-threaded build +- `third_party/` — tests adapted from NumPy's `numpy/random/tests` + +## Expectations +- Behavior changes include test updates in the same PR; bug fixes include a + regression test. +- Seed every generator a test uses, and keep tests free of timing assertions. + +## Entry points +- `pytest mkl_random/tests` from a checkout with an editable install +- `pytest --pyargs mkl_random` against an installed package