Repository navigation
Build and test conda packages and wheels - #2409
jharlow-intel wants to merge 2 commits into
Conversation
Adds what Intel Python CI (gh-ci) needs: a ci/ directory and .github/workflows/intel-ci.yml, which build the conda packages and the wheels for every declared platform and Python and test each with ci/test.py, on the OpenCL CPU device in Actions and on a GPU in Jenkins. conda-recipe/ is now the only recipe, the conda-forge one that was conda-recipe-cf/; the Intel-channel recipe and conda-package.yml are removed. The dev uploads to anaconda.org and their cleanup move into intel-ci.yml, for pushes to master and tags only. The wheels leave the SYCL runtime out and require intel-sycl-rt and intel-cmplr-lib-rt, as the PyPI wheels do; scikit-build is pinned below 1.0 (#2341).
9fc1cc4 to
e53ef4e
Compare
|
View rendered docs @ https://intelpython.github.io/dpctl/pulls/2409/index.html |
|
coverage: 74.963% (-0.03%) from 74.989% — ci/intel-python-ci into master |
| if os.environ.get("GH_CI_KIND") == "wheel": | ||
| print("deselecting test_syclinterface: a wheel keeps no symlinks") | ||
| skipped.append("test_syclinterface") | ||
| if not has_compilers(env): |
There was a problem hiding this comment.
If the cell's config lists compilers, a failed probe should fail the run rather than deselect test_c_headers/test_cython_api. As is, a broken toolchain still passes.
| gpu = os.environ.get("GH_CI_DEVICE") == "gpu" | ||
| with tempfile.TemporaryDirectory() as merged: | ||
| env = sycl_env(gpu, pathlib.Path(merged)) | ||
| subprocess.run([sys.executable, "-m", "dpctl", "-f"], env=env) |
There was a problem hiding this comment.
The return code is ignored. Checking it would report an import or runtime failure here instead of later as a missing device.
| run: | | ||
| cd "$PACKAGES" | ||
| conda_packages=(dpctl-*.conda) | ||
| version=${conda_packages[0]#dpctl-} |
There was a problem hiding this comment.
The wheels are uploaded with the conda package's version, which can differ from the wheel's own PEP 440 version. If no dpctl-*.conda matches, the unexpanded glob ends up in version. Taking the version from each wheel, or at least failing on an empty glob, would avoid both.
| # with the run rather than with the machine's user. | ||
| home = pathlib.Path.cwd() / "home" | ||
| home.mkdir(exist_ok=True) | ||
| env = dict(os.environ, HOME=str(home)) |
There was a problem hiding this comment.
On Windows ~ resolves from USERPROFILE, not HOME, so .asv-machine.json still goes to the agent's profile. Setting USERPROFILE as well would fix it.
| if DEFINED OVERRIDE_INTEL_IPO ( | ||
| set "CMAKE_ARGS=%CMAKE_ARGS% -DCMAKE_INTERPROCEDURAL_OPTIMIZATION:BOOL=FALSE" | ||
| REM useful for building in resources constrained VMs (public CI) | ||
| set "CMAKE_ARGS=%CMAKE_ARGS% -DCMAKE_INTERPROCEDURAL_OPTIMIZATION:BOOL=FALSE" |
There was a problem hiding this comment.
The Intel-channel recipe only did this under OVERRIDE_INTEL_IPO. It is now unconditional, so release Windows packages are built without IPO, and build.sh has no equivalent. Is that intended?
| :: Make CMake verbose | ||
| set "VERBOSE=1" | ||
|
|
||
| :: set CMAKE to use less threads to avoid OOM |
There was a problem hiding this comment.
The comment says fewer threads, but CPU_COUNT is the full count, so this doesn't limit anything. Same in build.sh.
| ] | ||
| environment = { CC = "/opt/intel/oneapi/compiler/latest/bin/icx", CXX = "/opt/intel/oneapi/compiler/latest/bin/icpx", CMAKE_GENERATOR = "Ninja", CMAKE_ARGS = "-DDPCTL_LEVEL_ZERO_INCLUDE_DIR=/opt/level-zero/include -DDPCTL_WITH_REDIST=ON" } | ||
| # The SYCL and icx runtime libraries come from wheels at run time, see [tool.gh-ci] | ||
| repair-wheel-command = "auditwheel repair --exclude libsycl.so.9 --exclude libintlc.so.5 --exclude libimf.so --exclude libsvml.so --exclude libirng.so -w {dest_dir} {wheel}" |
There was a problem hiding this comment.
libsycl.so.9 here (and sycl9.dll in the delvewheel list) is pinned, while wheel-requires allows any 2026.x runtime. If the soname changes, the repair step vendors a second libsycl. Worth a comment tying these to wheel-requires.
asv needs PyYAML, which has no wheel on PyPI for Python 3.15 yet; conda-forge builds it.
jharlow-intel
changed the title
Adds
ci/and.github/workflows/intel-ci.yml, which build the conda packages and wheels for Linux and Windows on every declared Python, and test each in a clean environment.conda-recipe/is now the only recipe: the conda-forge one that wasconda-recipe-cf/. The Intel-channel recipe andconda-package.ymlare removed;conda-package-cf.ymlstays.dppy/label/devuploads and the cleanup of old dev packages move intointel-ci.yml, for pushes tomasterand tags only.intel-sycl-rtandintel-cmplr-lib-rt, as the PyPI wheels do.scikit-buildis pinned below 1.0 (Scikit-build (classic) 1.0 internals #2341).ci/test.pyrunspytest --pyargs dpctlon the installed package and fails when it finds no SYCL device: the OpenCL CPU device in Actions, or a GPU withGH_CI_DEVICE=gpu.Validation, with a temporary push trigger on this branch: run 41648323136 built and tested the conda packages and wheels for 3.10 and 3.15 on Linux and Windows, and pre-commit, zizmor and
conda-package-cf.ymlpassed. Before the rename, a full-matrix run passed all 61 jobs, 3.14t and 3.15t included. Not run yet: a GPU, and the uploads, which onlymasterand tags do.