Release the GIL around oneMKL VM calls by antonwolfy · Pull Request #3156 · IntelPython/dpnp

antonwolfy · 2026-11-03T21:01:47Z

On a CPU device oneMKL VM is synchronous: it waits on the depends events with sycl::event::wait_and_throw, computes on the calling thread and returns an empty event (src/vml/_src/onemkl_vm/source/core/cpu/intel/usm/linear/v21.hpp and the other CPU variants). The VM extension calls it with the GIL held. On an in-order queue the depends events are ordered after the host tasks dpnp submits to keep arguments alive, and these host tasks acquire the GIL, so the VM call never returns.

The PR releases the GIL right before each of the 45 oneMKL VM calls, after the type and offset checks that can throw, as the BLAS, LAPACK and FFT extensions already do (#3027, #3040).

Reproducer, hangs on master and finishes in about a second with the PR:

import dpctl, dpnp

q = dpctl.SyclQueue("cpu", property="in_order")
xp = dpnp.linspace(10.0, 0.0, 5, sycl_queue=q)
for _ in range(100):
    xs = xp[dpnp.argsort(xp)]  # submits host tasks keeping arguments alive
    xs[-1:] - xs[:1]           # oneMKL VM sub
q.wait()

dpnp.interp with period hangs the same way, e.g. test_sycl_queue.py::test_interp[180.0-None-None-opencl:cpu:0] when the default queue is in-order. With IntelPython/dpctl#2299, which makes submitted_events of an in-order queue return the queue's last event (the keep-alive host task), the full test_sycl_queue.py hung in 5 of 5 runs with in-order queues; with this PR it passes in 6 of 6.

dpnp/tests/test_vm_gil.py runs the reproducer in a subprocess with a timeout, since a deadlocked call holds the GIL and can't be interrupted from the test process. It fails after the 120 s timeout on master and passes in under a second with the PR.

Tested on Panther Lake H (Arc B390 iGPU) with DPC++ 2026.2: test_umath.py, test_mathematical.py, test_special.py, test_sycl_queue.py, test_usm_type.py, test_vm_gil.py and third_party/cupy/math_tests pass on Level Zero GPU and OpenCL CPU. The time of a VM add on 1024 elements is unchanged: 16.5-17.1 us on master vs 16.9-17.0 us with the PR on GPU, 13.1-13.2 us vs 13.3-13.4 us on CPU.

Have you provided a meaningful PR description?
Have you added a test, reproducer or referred to an issue with a reproducer?
Have you tested your changes locally for CPU and GPU devices?
Have you made sure that new changes do not introduce compiler warnings?
Have you checked performance impact of proposed changes?
Have you added documentation for your changes, if necessary?
Have you added your changes to the changelog?

On a CPU device oneMKL VM waits on the dependent events and computes on the calling thread. On an in-order queue those events are ordered after the host tasks dpnp submits to keep arguments alive, which acquire the GIL, so a VM call made with the GIL held deadlocks.

Copilot

🟢 Approval recommended

No active review issues are recorded for the CPU in-order queue deadlock fix.

No open issues 🎉

What changed in this PR

This PR prevents CPU in-order queue deadlocks by releasing Python’s global interpreter lock (GIL) around oneMKL vector math calls. It extends the approach already used by dpnp’s BLAS, LAPACK, and FFT extensions.

Changes:

Releases the GIL around all 45 oneMKL VM calls, after type and offset validation.
Adds a subprocess regression test with a timeout to detect deadlocks safely.
Records the fix in the changelog.

File	Description
`dpnp/tests/test_vm_gil.py`	Adds timeout-protected CPU deadlock regression coverage.
`dpnp/libtensor/extensions/vm/trunc.cpp`	Releases the GIL around `trunc`.
`dpnp/libtensor/extensions/vm/tanh.cpp`	Releases the GIL around `tanh`.
`dpnp/libtensor/extensions/vm/tan.cpp`	Releases the GIL around `tan`.
`dpnp/libtensor/extensions/vm/sub.cpp`	Releases the GIL around `sub`.
`dpnp/libtensor/extensions/vm/sqrt.cpp`	Releases the GIL around `sqrt`.
`dpnp/libtensor/extensions/vm/sqr.cpp`	Releases the GIL around `sqr`.
`dpnp/libtensor/extensions/vm/sinh.cpp`	Releases the GIL around `sinh`.
`dpnp/libtensor/extensions/vm/sin.cpp`	Releases the GIL around `sin`.
`dpnp/libtensor/extensions/vm/rint.cpp`	Releases the GIL around `rint`.
`dpnp/libtensor/extensions/vm/pow.cpp`	Releases the GIL around `pow`.
`dpnp/libtensor/extensions/vm/nextafter.cpp`	Releases the GIL around `nextafter`.
`dpnp/libtensor/extensions/vm/mul.cpp`	Releases the GIL around `mul`.
`dpnp/libtensor/extensions/vm/modf.cpp`	Releases the GIL around `modf`.
`dpnp/libtensor/extensions/vm/log2.cpp`	Releases the GIL around `log2`.
`dpnp/libtensor/extensions/vm/log1p.cpp`	Releases the GIL around `log1p`.
`dpnp/libtensor/extensions/vm/log10.cpp`	Releases the GIL around `log10`.
`dpnp/libtensor/extensions/vm/ln.cpp`	Releases the GIL around `ln`.
`dpnp/libtensor/extensions/vm/inv.cpp`	Releases the GIL around `inv`.
`dpnp/libtensor/extensions/vm/i0.cpp`	Releases the GIL around `i0`.
`dpnp/libtensor/extensions/vm/hypot.cpp`	Releases the GIL around `hypot`.
`dpnp/libtensor/extensions/vm/fmod.cpp`	Releases the GIL around `fmod`.
`dpnp/libtensor/extensions/vm/fmin.cpp`	Releases the GIL around `fmin`.
`dpnp/libtensor/extensions/vm/fmax.cpp`	Releases the GIL around `fmax`.
`dpnp/libtensor/extensions/vm/floor.cpp`	Releases the GIL around `floor`.
`dpnp/libtensor/extensions/vm/expm1.cpp`	Releases the GIL around `expm1`.
`dpnp/libtensor/extensions/vm/exp2.cpp`	Releases the GIL around `exp2`.
`dpnp/libtensor/extensions/vm/exp.cpp`	Releases the GIL around `exp`.
`dpnp/libtensor/extensions/vm/erf_funcs.cpp`	Releases the GIL in macro-generated error-function calls.
`dpnp/libtensor/extensions/vm/div.cpp`	Releases the GIL around `div`.
`dpnp/libtensor/extensions/vm/cosh.cpp`	Releases the GIL around `cosh`.
`dpnp/libtensor/extensions/vm/cos.cpp`	Releases the GIL around `cos`.
`dpnp/libtensor/extensions/vm/copysign.cpp`	Releases the GIL around `copysign`.
`dpnp/libtensor/extensions/vm/conj.cpp`	Releases the GIL around `conj`.
`dpnp/libtensor/extensions/vm/ceil.cpp`	Releases the GIL around `ceil`.
`dpnp/libtensor/extensions/vm/cbrt.cpp`	Releases the GIL around `cbrt`.
`dpnp/libtensor/extensions/vm/atanh.cpp`	Releases the GIL around `atanh`.
`dpnp/libtensor/extensions/vm/atan2.cpp`	Releases the GIL around `atan2`.
`dpnp/libtensor/extensions/vm/atan.cpp`	Releases the GIL around `atan`.
`dpnp/libtensor/extensions/vm/asinh.cpp`	Releases the GIL around `asinh`.
`dpnp/libtensor/extensions/vm/asin.cpp`	Releases the GIL around `asin`.
`dpnp/libtensor/extensions/vm/arg.cpp`	Releases the GIL around `arg`.
`dpnp/libtensor/extensions/vm/add.cpp`	Releases the GIL around `add`.
`dpnp/libtensor/extensions/vm/acosh.cpp`	Releases the GIL around `acosh`.
`dpnp/libtensor/extensions/vm/acos.cpp`	Releases the GIL around `acos`.
`dpnp/libtensor/extensions/vm/abs.cpp`	Releases the GIL around `abs`.
`CHANGELOG.md`	Documents the CPU in-order queue deadlock fix.

_{Review effort: Balanced}

ndgrigorian · 2026-11-03T21:24:27Z

+    // On a CPU device oneMKL VM waits on `depends` and computes on the
+    // calling thread, so release the GIL for host tasks that need it


Nit: Could all of these comments be pared down into a README or a piece of documentation (i.e., a docstring on some common code) which informs users more effectively and in a less verbose way?

Done: the comment now lives once, on a vm_gil_release alias in vm/common.hpp, and each call site is just vm_gil_release lock{};.

ndgrigorian

Aside from that nit LGTM

coveralls · 2026-11-03T22:44:08Z

coverage: 84.396% (+0.02%) from 84.376% — antonwolfy:vm-release-gil into IntelPython:master

Copilot

🟢 Approval recommended

There are no active review issues for the GIL-release change targeting CPU in-order queue deadlocks.

No open issues 🎉

_{Review effort: Balanced}

ndgrigorian

LGTM, this bug appeared in the dpnp.random rework PR so getting it in would be nice

On a CPU device oneMKL VM is synchronous: it waits on the `depends` events with `sycl::event::wait_and_throw`, computes on the calling thread and returns an empty event (`src/vml/_src/onemkl_vm/source/core/cpu/intel/usm/linear/v21.hpp` and the other CPU variants). The VM extension calls it with the GIL held. On an in-order queue the `depends` events are ordered after the host tasks dpnp submits to keep arguments alive, and these host tasks acquire the GIL, so the VM call never returns. The PR releases the GIL right before each of the 45 oneMKL VM calls, after the type and offset checks that can throw, as the BLAS, LAPACK and FFT extensions already do (#3027, #3040). Reproducer, hangs on master and finishes in about a second with the PR: ```python import dpctl, dpnp q = dpctl.SyclQueue("cpu", property="in_order") xp = dpnp.linspace(10.0, 0.0, 5, sycl_queue=q) for _ in range(100): xs = xp[dpnp.argsort(xp)] # submits host tasks keeping arguments alive xs[-1:] - xs[:1] # oneMKL VM sub q.wait() ``` `dpnp.interp` with `period` hangs the same way, e.g. `test_sycl_queue.py::test_interp[180.0-None-None-opencl:cpu:0]` when the default queue is in-order. With IntelPython/dpctl#2299, which makes `submitted_events` of an in-order queue return the queue's last event (the keep-alive host task), the full `test_sycl_queue.py` hung in 5 of 5 runs with in-order queues; with this PR it passes in 6 of 6. `dpnp/tests/test_vm_gil.py` runs the reproducer in a subprocess with a timeout, since a deadlocked call holds the GIL and can't be interrupted from the test process. It fails after the 120 s timeout on master and passes in under a second with the PR. Tested on Panther Lake H (Arc B390 iGPU) with DPC++ 2026.2: `test_umath.py`, `test_mathematical.py`, `test_special.py`, `test_sycl_queue.py`, `test_usm_type.py`, `test_vm_gil.py` and `third_party/cupy/math_tests` pass on Level Zero GPU and OpenCL CPU. The time of a VM `add` on 1024 elements is unchanged: 16.5-17.1 us on master vs 16.9-17.0 us with the PR on GPU, 13.1-13.2 us vs 13.3-13.4 us on CPU. 90622cd

antonwolfy requested review from ndgrigorian and vlad-perevezentsev as code owners November 3, 2026 21:01

Add IntelPythongh-3156 to the changelog

e863f96

antonwolfy mentioned this pull request Nov 3, 2026

Use a no-op order manager for in-order queues IntelPython/dpctl#2299

Open

7 of 8 tasks

antonwolfy self-assigned this Nov 3, 2026

antonwolfy added this to the 0.22.0 release milestone Nov 3, 2026

antonwolfy requested a balanced review from Copilot November 3, 2026 21:05

Copilot started reviewing on behalf of antonwolfy November 3, 2026 21:05

View session

Copilot AI reviewed Nov 3, 2026

View reviewed changes

ndgrigorian reviewed Nov 3, 2026

View reviewed changes

ndgrigorian approved these changes Nov 3, 2026

View reviewed changes

ndgrigorian mentioned this pull request Nov 4, 2026

Reimplement dpnp.random using oneMKL device RNG API #3147

Closed

6 of 7 tasks

antonwolfy and others added 2 commits November 4, 2026 11:38

Move the VM GIL release comment into a shared helper alias

12ee06a

Merge branch 'master' into vm-release-gil

8e7c94f

antonwolfy requested a balanced review from Copilot November 4, 2026 10:40

Copilot started reviewing on behalf of antonwolfy November 4, 2026 10:41

View session

Copilot AI reviewed Nov 4, 2026

View reviewed changes

Retrigger CI

8cf470e

ndgrigorian approved these changes Nov 4, 2026

View reviewed changes

antonwolfy merged commit 90622cd into IntelPython:master Nov 4, 2026
110 of 111 checks passed

antonwolfy deleted the vm-release-gil branch November 4, 2026 17:02

ndgrigorian mentioned this pull request Nov 4, 2026

Add multivariate distributions and permutations to dpnp.random.Generator #3160

Open

4 of 7 tasks

Navigation Menu

Repository navigation

Release the GIL around oneMKL VM calls - #3156

Release the GIL around oneMKL VM calls#3156
antonwolfy merged 5 commits into
IntelPython:masterfrom
antonwolfy:vm-release-gil

antonwolfy commented Nov 3, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

ndgrigorian Nov 3, 2026 •

edited

Loading

Uh oh!

antonwolfy Nov 4, 2026

Uh oh!

ndgrigorian left a comment

Uh oh!

coveralls commented Nov 3, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

ndgrigorian left a comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

		// On a CPU device oneMKL VM waits on `depends` and computes on the
		// calling thread, so release the GIL for host tasks that need it

Conversation

antonwolfy commented Nov 3, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟢 Approval recommended

Uh oh!

ndgrigorian Nov 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

antonwolfy Nov 4, 2026

Choose a reason for hiding this comment

Uh oh!

ndgrigorian left a comment

Choose a reason for hiding this comment

Uh oh!

coveralls commented Nov 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟢 Approval recommended

Uh oh!

ndgrigorian left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ndgrigorian Nov 3, 2026 •

edited

Loading

coveralls commented Nov 3, 2026 •

edited

Loading