Skip to content

Modernise packaging and tooling to match the MetEOR template - #38

Merged
shunt16 merged 10 commits into
mainfrom
update-package-tooling
Oct 27, 2026
Merged

shunt16 merged 10 commits into
mainfrom
update-package-tooling

Conversation

@shunt16

@shunt16 shunt16 commented Oct 27, 2026

Copy link
Copy Markdown
Collaborator

Brings obsarray's package-level setup in line with the newer MetEOR tools (eomatch, eoio, …). CI on main currently fails at "Install dependencies" on every job; this replaces it with a working setup, and fixes the issues the new checks and current numpy/xarray turned up.

Tooling

  • pyproject.toml only: removed setup.py, requirements.txt, tox.ini and environment.yml. Dev/docs dependencies are now the [dev] and [docs] extras, and pytest config lives in pyproject.toml.
  • Versioning: replaced versioneer with setuptools_scm (version from v* tags), with __version__ read via importlib.metadata.
  • Lint/format: replaced black with pinned ruff (check and format) and mypy in pre-commit, with an explicit ruff rule selection. The mypy hook installs numpy/xarray and is scoped to obsarray/, so it reports the same results as CI.
  • GitHub Actions:
    • tests.yml runs pytest on Python 3.11–3.13 and uploads test/coverage reports and coverage to Codecov. A separate lint job runs ruff format --check, ruff check and mypy.
    • publish.yml publishes to PyPI on a v* tag push (or a confirmed manual run), via trusted publishing.
  • Read the Docs: installs .[docs] and fetches full history so setuptools_scm sees the tags.
  • Python ≥ 3.11 (3.8–3.10 are end-of-life).
  • README: new badges, development and release instructions, and the licence corrected to LGPLv3.

Fixes needed to get CI green

  • NumPy 2 flag overflow: setting a flag to False raised OverflowError, which also broke the docs build. This is fixed by cherry-picking @mattgoalen's two fix commits from overflow-errors.
  • test_rename_dims: newer xarray's assert_identical also compares indexes, so the test now drops the retained time index before comparing. This was a test issue, not a library bug.
  • ruff and mypy findings: fixed in separate commits. They're mostly annotation-only; the small behaviour changes are listed in the "Fix mypy findings" commit message.
  • Formatting: a mechanical ruff format commit. It keeps black's 88-column line length rather than the template's 120, to avoid reflowing about 650 lines and conflicting with open branches.
  • Sphinx config: language = None changed to "en".

Testing

Run locally on Python 3.13 (numpy 2.5, xarray 2026.9):

  • All 173 tests pass.
  • ruff check, ruff format --check and mypy obsarray are clean, and pre-commit run -a passes.
  • The sdist and wheel build, with the version taken from tags.
  • The Sphinx HTML docs build. The remaining warnings are pre-existing and cosmetic.

Before the first release

  • Register a PyPI trusted publisher with owner comet-toolkit, repository obsarray, workflow publish.yml, environment pypi.
  • Add a CODECOV_TOKEN repository secret if you want coverage uploads.

Not changed

  • Docs stay on Read the Docs: no GitHub Pages docs.yml, because obsarray.readthedocs.io is the published URL.
  • No .gitlab-ci.yml, since obsarray is GitHub-only.
  • The license classifier is kept. setuptools warns that it's deprecated in favour of SPDX, but choosing between LGPL-3.0-only and LGPL-3.0-or-later is a licensing decision.
  • The old CI turned the coverage report into a PDF for the quality documentation. The new workflow uploads the HTML report as an artifact instead.

shunt16 and others added 10 commits October 27, 2026 17:19
- Move all package metadata into pyproject.toml and drop setup.py,
  requirements.txt, tox.ini and environment.yml (dev/docs deps are now
  the [dev] and [docs] extras; pytest config is in pyproject)
- Replace versioneer with setuptools_scm; __version__ now comes from
  importlib.metadata
- Replace black with ruff (lint + format) and mypy in pre-commit, with
  pinned versions and an explicit ruff rule selection
- Replace the old GitHub Actions workflows with tests.yml (pytest on
  Python 3.11-3.13 plus a ruff/mypy lint job) and publish.yml (tag-
  triggered PyPI release via trusted publishing)
- Update .readthedocs.yaml to install the docs extra and unshallow the
  clone so setuptools_scm can see tags
- Require Python >= 3.11 (3.8-3.10 are end-of-life)
- Refresh README badges, development and release instructions, and
  correct the licence name to LGPLv3
Mechanical only: ruff format at black's line length (removes blank lines
at the start of blocks and rejoins a few split expressions) and removes
unused imports flagged by ruff --fix. No behaviour change.
(cherry picked from commit 7b8f9bf)
(cherry picked from commit 61fc784)
- E721: compare exact types with `is` rather than `==` (same semantics)
- E712: mark the intentional elementwise numpy `== True/False`
  comparisons with noqa rather than changing their behaviour
- F841: use the already-computed sliced_dims rather than recomputing it,
  and drop unused variables from the create_ds test fixture
- F811: remove a verbatim duplicate of test___setitem___new_mask
- Reformat the cherry-picked overflow fix to the project line length
- test_rename_dims: newer xarray's assert_identical also compares
  indexes, and rename_dims keeps the index on the renamed "time"
  coordinate, so drop it before comparing with the expected dataset
Annotation fixes so the new mypy gate passes:

- Use numpy.typing.DTypeLike for dtype arguments (numpy.typecodes is a
  string constant, not a type)
- Make implicit-Optional defaults explicit and widen fill_value to
  accept numpy scalars, which get_default_fill_value returns
- Accept Hashable variable names where xarray hands them to us, and
  return xr.Dataset rather than an unbound TypeVar from rename methods
- Annotate err-corr slices as tuples, matching all callers and
  subclasses, and accept slices/ints in Flag indexing (e.g. ``[:]``)
- Build _expand_sli's list under its own name instead of retyping out_sli

Small behaviour changes that fall out of this:

- A "flag" template variable without attributes now raises a clear
  ValueError rather than AttributeError on None.pop
- An uncertainty tuple definition with attrs=None is treated as having
  no attrs rather than raising TypeError
- append_names/empty_err_corr_matrix format non-str names with str()
  rather than failing on string concatenation
Install numpy and xarray into the hook environment so type narrowing
behaves as it does in CI, restrict the hook to the package (as CI does),
and use the ruff-check hook id in place of the legacy ruff alias.
language = None is no longer valid in current Sphinx.
checkout v7, setup-python v7, upload-artifact v7, download-artifact v8
and codecov-action v7, clearing the Node.js 20 deprecation warnings.
None of their breaking changes affect these workflows (artifacts are
downloaded by name, no pull_request_target, no pip-install input).
@shunt16
shunt16 merged commit c94bc0f into main Oct 27, 2026
4 checks passed
@shunt16
shunt16 deleted the update-package-tooling branch October 27, 2026 17:44
@shunt16 shunt16 mentioned this pull request Oct 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants