diff --git a/.github/AGENTS.md b/.github/AGENTS.md new file mode 100644 index 0000000..a318f75 --- /dev/null +++ b/.github/AGENTS.md @@ -0,0 +1,29 @@ +# AGENTS.md — .github/ + +CI/CD workflows, automation, security scanning, and package distribution. + +## Workflows +- **conda-package.yml** — main build/test pipeline (Linux/Windows, Python 3.10-3.14) +- **build-with-clang.yml** — Linux Clang compiler compatibility validation +- **pre-commit.yml** — code quality checks (flake8, etc.) +- **openssf-scorecard.yml** — security posture scanning + +## CI/CD policy +- Keep build matrix (Python versions, platforms) in workflow files only +- Required checks: conda build + test on supported Python versions/platforms in CI +- Artifact naming: `$PACKAGE_NAME $OS Python $VERSION` +- Channels: `conda-forge`, `conda-forge/label/python_rc`, Intel channel + +## Security +- OpenSSF Scorecard runs automatically +- CODEOWNERS enforces review policy +- Dependabot monitors dependencies (`.github/dependabot.yml`) + +## Platform specifics +- **Linux:** RTLD_GLOBAL handling for MKL library loading +- **Windows:** DLL search path configuration for venv/runtime loading + +## Notes +- Workflow/job renames are breaking for downstream tooling +- Cache key includes `meta.yaml` hash for conda packages +- Python 3.14 uses `conda-forge/label/python_rc` for pre-release support diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md new file mode 100644 index 0000000..b87b389 --- /dev/null +++ b/.github/copilot-instructions.md @@ -0,0 +1,47 @@ +# GitHub Copilot Instructions — mkl-service + +## Identity +You are an expert Python/Cython/C developer working on `mkl-service` at Intel. +Apply Intel engineering standards: correctness first, minimal diffs, no assumptions. + +## Source of truth +This file is canonical for Copilot/agent behavior. +`AGENTS.md` files provide project context. + +## Precedence +copilot-instructions > nearest AGENTS > root AGENTS +Higher-precedence file overrides; lower must not restate overridden guidance. + +## Mandatory flow +1. Read root `AGENTS.md`. If absent, stop and report. +2. For each edited file, locate and follow the nearest `AGENTS.md`. +3. If no local file exists, inherit rules from root `AGENTS.md`. + +## Contribution expectations +- Keep diffs minimal; prefer atomic single-purpose commits. +- Preserve public API signatures in `mkl/__init__.py` unless change is explicitly requested. +- For user-visible behavior changes: update tests in `mkl/tests/test_mkl_service.py`. +- For bug fixes: add or extend regression tests in the same change. +- Do not generate code without corresponding test updates when behavior changes. +- Run `pre-commit run --all-files` when `.pre-commit-config.yaml` is present. + +## Authoring rules +- Use source-of-truth files for all mutable details. +- Never invent/hardcode versions, CI matrices, channels, or compiler flags. +- Prefer stable local entry points: + - `python -m pip install -e .` + - `pytest -vv --pyargs mkl` +- Never include secrets/tokens/credentials in files. + +## Source-of-truth files +- Build/config: `pyproject.toml`, `setup.py` +- Recipe/deps: `conda-recipe/meta.yaml`, `conda-recipe/conda_build_config.yaml` +- CI: `.github/workflows/*.{yml,yaml}` +- API contracts: `mkl/__init__.py`, `mkl/_mkl_service.pyx` +- Tests: `mkl/tests/test_mkl_service.py` + +## MKL-specific constraints +- Linux runtime init path may require `RTLD_GLOBAL` preloading (`mkl/_mklinitmodule.c`). +- Windows venv DLL handling is in `mkl/_init_helper.py`. +- Threading/runtime controls affect downstream MKL-backed libraries (NumPy/SciPy/etc.); maintain compatibility. +- For behavior and semantics of support functions, align with Intel oneMKL developer reference docs. diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..5d422ef --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,88 @@ +# AGENTS.md + +Entry point for agent context in this repo. + +## What this repository is +`mkl-service` provides a Python API for runtime control of Intel® oneMKL (Math Kernel Library). It exposes support functions for: +- Threading control (set/get number of threads, domain-specific threading) +- Version information (MKL version, build info) +- Memory management (peak memory usage, memory statistics) +- Conditional Numerical Reproducibility (CNR) +- Timing functions (get CPU/wall clock time) +- Miscellaneous utilities (MKL_VERBOSE control, etc.) + +Originally part of Intel® Distribution for Python*, now a standalone package available via conda-forge and Intel channels. + +## Key components +- **Python interface:** `mkl/__init__.py` — public API surface +- **Cython wrapper:** `mkl/_mkl_service.pyx` — wraps MKL support functions +- **C init module:** `mkl/_mklinitmodule.c` — Linux-side MKL runtime preloading / initialization +- **Helper:** `mkl/_init_helper.py` — Windows venv DLL loading helper +- **Build system:** setuptools + Cython + +## Build dependencies +**Required:** +- Intel® oneMKL +- Cython +- Python 3.10+ + +**Conda environment:** +```bash +conda install -c conda-forge mkl-devel cython +python setup.py install +``` + +## CI/CD +- **Platforms in CI workflows:** Linux, Windows +- **Python versions:** 3.10, 3.11, 3.12, 3.13, 3.14 +- **Workflows:** `.github/workflows/` + - `conda-package.yml` — main conda build/test pipeline + - `build-with-clang.yml` — Linux Clang compatibility + - `pre-commit.yml` — code quality checks + - `openssf-scorecard.yml` — security scanning + +## Distribution +- **Conda:** `conda-forge` and `https://software.repos.intel.com/python/conda` +- **PyPI:** `python -m pip install mkl-service` + +## Usage +```python +import mkl +mkl.set_num_threads(4) # Set global thread count +mkl.domain_set_num_threads(1, "fft") # FFT functions run sequentially +mkl.get_version_string() # MKL version info +``` + +## How to work in this repo +- **API stability:** Preserve existing function signatures (widely used in ecosystem) +- **Threading:** Changes to threading control must be thread-safe +- **CNR:** Conditional Numerical Reproducibility flags require careful documentation +- **Testing:** Add tests to `mkl/tests/test_mkl_service.py` +- **Docs:** MKL support functions documented in [Intel oneMKL Developer Reference](https://www.intel.com/content/www/us/en/docs/onemkl/developer-reference-c/2025-2/support-functions.html) + +## Code structure +- **Cython layer:** `_mkl_service.pyx` + `_mkl_service.pxd` (C declarations) +- **C init:** `_mklinitmodule.c` handles Linux preloading (`dlopen(..., RTLD_GLOBAL)`) for MKL runtime +- **Windows loading helper:** `_init_helper.py` handles DLL path setup in Windows venv +- **Python wrapper:** `__init__.py` imports `_py_mkl_service` (generated from `.pyx`) +- **Version:** `_version.py` (dynamic via setuptools) + +## Notes +- RTLD_GLOBAL preloading is required on Linux (handled by `RTLD_for_MKL` context manager) +- MKL must be available at runtime (conda: mkl, pip: relies on system MKL) +- Threading functions affect NumPy, SciPy, and other MKL-backed libraries + +## Directory map +Below directories have local `AGENTS.md` for deeper context: +- `.github/AGENTS.md` — CI/CD workflows and automation +- `mkl/AGENTS.md` — Python/Cython implementation +- `mkl/tests/AGENTS.md` — unit tests +- `conda-recipe/AGENTS.md` — conda packaging +- `examples/AGENTS.md` — usage examples + +--- + +For broader IntelPython ecosystem context, see: +- `mkl_umath` (MKL-backed NumPy ufuncs) +- `mkl_random` (MKL-based random number generation) +- `dpnp` (Data Parallel NumPy) diff --git a/conda-recipe/AGENTS.md b/conda-recipe/AGENTS.md new file mode 100644 index 0000000..bd1a6af --- /dev/null +++ b/conda-recipe/AGENTS.md @@ -0,0 +1,34 @@ +# AGENTS.md — conda-recipe/ + +Conda package build recipe for conda-forge and Intel channel distribution. + +## Files +- **meta.yaml** — package metadata, dependencies, build requirements +- **build.sh** — Linux/macOS build script +- **bld.bat** — Windows build script +- **conda_build_config.yaml** — compiler / stdlib pinning used by conda-build variants + +## Build configuration +- **Channels:** `conda-forge`, `conda-forge/label/python_rc`, Intel channel +- **Python versions in CI build matrix:** 3.10, 3.11, 3.12, 3.13, 3.14 +- **Compilers:** pinned via `conda_build_config.yaml` (GCC/GXX on Linux, VS2022 on Windows) +- **Dependencies:** `mkl` (runtime), `mkl-devel` (build), `cython` + +## Build outputs +- Conda package: `mkl-service--.conda` +- Platform-specific artifacts from CI currently: `linux-64/`, `win-64/` + +## CI usage +- Built in `.github/workflows/conda-package.yml` +- Artifacts uploaded per Python version and platform +- Test stage uses built artifacts from a temporary local channel + +## Platform specifics +- **Linux:** RTLD_GLOBAL handling in runtime init path +- **Windows:** MKL library path configuration +- **macOS:** recipe scripts exist, but current GitHub Actions matrix does not build/test macOS + +## Maintenance +- Keep recipe metadata in sync with workflow matrix and package metadata +- MKL version pinning: track Intel MKL releases through recipe deps +- Python 3.14 in CI uses `conda-forge/label/python_rc` while pre-release diff --git a/examples/AGENTS.md b/examples/AGENTS.md new file mode 100644 index 0000000..d26679c --- /dev/null +++ b/examples/AGENTS.md @@ -0,0 +1,21 @@ +# AGENTS.md — examples/ + +Usage examples for mkl-service runtime control API. + +## Files +- **example.py** — basic usage: MKL version query, instruction dispatch control, and timing + +## Examples cover +- Querying MKL version/build info (`get_version()`, `get_version_string()`) +- Controlling instruction dispatch with `enable_instructions()` +- Using `dsecnd()` for simple timing + +## Running examples +```bash +python examples/example.py +``` + +## Notes +- Examples assume MKL is installed (conda: `mkl` package) +- Example is intended as a lightweight installation/runtime sanity check +- Threading/domain APIs are available in `mkl-service` but not exercised in this example diff --git a/mkl/AGENTS.md b/mkl/AGENTS.md new file mode 100644 index 0000000..9710b83 --- /dev/null +++ b/mkl/AGENTS.md @@ -0,0 +1,58 @@ +# AGENTS.md — mkl/ + +Core Python/Cython implementation: MKL support function wrappers and runtime control. + +## Structure +- `__init__.py` — public API, RTLD_GLOBAL context manager, module initialization +- `_mkl_service.pyx` — Cython wrappers for MKL support functions +- `_mkl_service.pxd` — Cython declarations (C function signatures) +- `_mklinitmodule.c` — C extension for Linux-side MKL runtime preloading/init +- `_init_helper.py` — Windows loading helper (DLL path setup in venv) +- `_version.py` — version string (dynamic via setuptools) +- `tests/` — unit tests for API functionality + +## API categories +### Threading control +- `set_num_threads(n)` — global thread count +- `get_max_threads()` — max threads available +- `domain_set_num_threads(n, domain)` — per-domain threading (FFT, VML, etc.) +- `domain_get_max_threads(domain)` — domain-specific max threads + +### Version information +- `get_version()` — MKL version dict +- `get_version_string()` — formatted version string + +### Memory management +- `peak_mem_usage(memtype)` — peak memory usage stats +- `mem_stat()` — memory allocation statistics + +### CNR (Conditional Numerical Reproducibility) +- `set_num_threads_local(n)` — thread-local thread count +- CNR mode control functions + +### Timing +- `second()` — wall clock time +- `dsecnd()` — high-precision timing + +## Development guardrails +- **Thread safety:** All threading functions must be thread-safe +- **API stability:** Preserve function signatures (widely used in ecosystem) +- **MKL dependency:** Assumes MKL is available at runtime (conda: mkl package) +- **RTLD_GLOBAL preload path:** Linux preload is handled in `_mklinitmodule.c`; Windows DLL setup is in `_init_helper.py` + +## Cython details +- `_mkl_service.pyx` → generates `_py_mkl_service` extension module +- `.pxd` file declares external C functions from MKL headers +- Cython build requires MKL headers (`mkl-devel`) + +## C init module +- `_mklinitmodule.c` → `_mklinit` extension +- Ensures MKL runtime is initialized with correct flags before Cython extension +- Platform-specific behavior in current code: + - Linux: `dlopen(..., RTLD_GLOBAL)` preload path (when applicable) + - Windows: runtime DLL loading support is handled by `_init_helper.py` + +## Notes +- Domain strings: "fft", "vml", "pardiso", "blas", etc. (see MKL docs) +- Threading changes affect all MKL-using libraries (NumPy, SciPy, etc.) +- `MKL_VERBOSE` environment variable controls MKL diagnostic output diff --git a/mkl/tests/AGENTS.md b/mkl/tests/AGENTS.md new file mode 100644 index 0000000..021d695 --- /dev/null +++ b/mkl/tests/AGENTS.md @@ -0,0 +1,28 @@ +# AGENTS.md — mkl/tests/ + +Unit tests for MKL runtime control API. + +## Test files +- **test_mkl_service.py** — API functionality, threading control, version info + +## Test coverage +- Threading: `set_num_threads`, `get_max_threads`, domain-specific threading +- Version: `get_version`, `get_version_string` format validation +- Memory: `peak_mem_usage`, `mem_stat` (if supported by MKL build) +- CNR: Conditional Numerical Reproducibility flags +- Edge cases currently covered: thread-local settings and API round-trips + +## Running tests +```bash +pytest mkl/tests/ +``` + +## CI integration +- Tests run in `conda-package.yml` workflow +- Separate test jobs per Python version and CI platform +- CI coverage: Linux + Windows + +## Adding tests +- New API functions → add to `test_mkl_service.py` with validation +- Threading behavior → test thread count changes take effect +- Use `mkl.get_version()` to check MKL availability before tests