Repository navigation
Run the darwin reference tests in CI - #32
Merged
Merged
Conversation
test_align compared pyopa with the darwin reference results, but was skipped as it takes about 20 minutes. Run a fixed subsample by default (every 20th result with an alignment, which also checks estimate_pam, and every 200th of the others; about 12 s), and all results if PYOPA_TEST_FULL is set. Failure messages give the row of the reference results. A new workflow runs the full comparison on pushes to master, weekly and on demand.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
test_darwin.AlignTest.test_aligncompares the alignment scores, aligned strings andestimate_pamresults with the 35,156 reference results from darwin, but it was skipped because it takes about 20 minutes.align_stringsandestimate_pam) and every 200th of the others. That's 335 results in about 12 s, so it runs in every CI job and in the wheel tests.PYOPA_TEST_FULL=1runs all results. Locally, the full run passes in 20 min on current master.full-tests.yml, runs the full comparison on pushes to master, weekly (Monday 03:00 UTC) and on demand (workflow_dispatch), on Ubuntu with Python 3.14.Most of the time goes to the scalar reference alignment (about 65 %), which is the reference check itself, so the speedup comes from the subsample.