Skip to content

Declare the extension free-threading compatible - #278

Merged
benoit74 merged 1 commit into
openzim:mainfrom
pawank925:issue-240-free-threaded
Oct 22, 2026
Merged

benoit74 merged 1 commit into
openzim:mainfrom
pawank925:issue-240-free-threaded

Conversation

@pawank925

@pawank925 pawank925 commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Cython 3.1+ can build extension modules that work on free-threaded CPython, but a module only runs without re-enabling the GIL if it opts in with the module-level freethreading_compatible directive. Without it, importing the module forces the GIL back on (with a runtime warning), so free-threaded builds gained nothing.

The extension does not rely on the GIL: the C++ libzim calls are wrapped in with nogil, and the only Python interaction is through the with gil callbacks, which already acquire the interpreter thread-state as needed. Declare the module free-threading compatible and build it with Py_GIL_DISABLED on free-threaded interpreters, so it is compiled against the free-threaded API.

The workarounds that forced the GIL back on are now unnecessary and removed:

  • the cibuildwheel override running tests with -X gil=1 on cp313t/cp314t,
  • the subprocess helper in the creator tests that passed -X gil=1,
  • the README note stating free-threaded CPython is unsupported.

README and CHANGELOG updated accordingly.

The Tests and QA workflows are also updated to run on pull_request (and on push to main only), matching the convention used in the other openZIM repositories, so the free-threaded tests actually run on this PR (refs #277).

Notes:

  • I could only build and run the full test suite (119 passed) on a GIL-enabled Python 3.14 locally; the free-threaded (3.14t) job in CI is what actually exercises the no-GIL path.
  • Thread-safety of the underlying C++ libzim is unchanged and remains the caller's responsibility (reading is mostly thread-safe, searching/creating is not).

Closes #240
Refs #277

@pawank925
pawank925 force-pushed the issue-240-free-threaded branch from 456dcff to 7d38b29 Compare October 21, 2026 06:28
@pawank925

pawank925 commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor Author

@benoit74 once look into this please thanks!

@benoit74

Copy link
Copy Markdown
Contributor

We will need to fix CI with #277 so that tests run. Obviously pretty important on such a PR 🤣

Cython 3.1+ can build extensions compatible with free-threaded CPython,
but a module must opt in with the freethreading_compatible directive;
otherwise importing it re-enables the GIL. The module was tested with the
GIL disabled and does not rely on it, so declare it compatible, build with
Py_GIL_DISABLED on free-threaded interpreters, and drop the test-time
workarounds that forced the GIL back on.

Also make the Tests and QA workflows run on pull requests, as expected by
the convention, so this PR's free-threaded tests actually run.

Closes openzim#240
Refs openzim#277
@pawank925
pawank925 force-pushed the issue-240-free-threaded branch from 7d38b29 to b67fa64 Compare October 22, 2026 06:20
@pawank925

pawank925 commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor Author

@benoit74 Thanks for pointing this out. The Tests and QA workflows were only triggered by push, so no test ran on this PR. They now follow the convention used across the openZIM repositories, triggering on pull_request and on push to main only, which lets the 3.14t job actually exercise the free-threaded path.

The CI trigger change is included in this PR and references #277.

@codecov

codecov Bot commented Oct 22, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.11%. Comparing base (e55f4ef) to head (b67fa64).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #278   +/-   ##
=======================================
  Coverage   94.11%   94.11%           
=======================================
  Files           1        1           
  Lines         544      544           
=======================================
  Hits          512      512           
  Misses         32       32           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@benoit74 benoit74 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you very much!

@benoit74
benoit74 merged commit a486a6e into openzim:main Oct 22, 2026
62 checks passed
@rgaudin

rgaudin commented Oct 22, 2026

Copy link
Copy Markdown
Member

In the future we should avoid feature and unrelated CI changes in the same commit… 🙃

@pawank925
pawank925 deleted the issue-240-free-threaded branch October 22, 2026 14:31
pawank925 pushed a commit to pawank925/python-libzim that referenced this pull request Oct 28, 2026
Declare the extension free-threading compatible
pawank925 added a commit to pawank925/python-libzim that referenced this pull request Oct 28, 2026
Declare the extension free-threading compatible
pawank925 added a commit to pawank925/python-libzim that referenced this pull request Oct 28, 2026
Declare the extension free-threading compatible
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

QA and Tests workflows must run on pull requests as expected by the convention Enhance no GIL / free threaded support

3 participants