Repository navigation
Feat: HTCondor core - #33
Open
Alexander-Auer wants to merge 77 commits into
Open
Alexander-Auer wants to merge 77 commits into
Alexander-Auer wants to merge 77 commits into
Conversation
… have them link to each other
…ede into feat/core-contract
…a consumer should choose to use it or not
* feat: Models the core contract needs * Feat: Protocols and Exceptions added for the core contract * Feat: Private protocols for the BatchBackend * feat: a fake backend to run unit tests against * fix: some code cleanup in the helper functions * feat: setup for the unit test suite * chore: style fix * chore: styling, unit test fixture * chore: single letter var `h` refering to hanlde changed to `handle` * fix: unused parameter in `test_injected_failure_is_reported_per_job()` removed * feat: Models the core contract needs * Feat: Protocols and Exceptions added for the core contract * Feat: Private protocols for the BatchBackend * feat: a fake backend to run unit tests against * fix: some code cleanup in the helper functions * feat: setup for the unit test suite * chore: style fix * chore: styling, unit test fixture * chore: single letter var `h` refering to hanlde changed to `handle` * fix: unused parameter in `test_injected_failure_is_reported_per_job()` removed * fix: made the OutputSpec optional as it is not suppoerted by all and a consumer should choose to use it or not --------- Co-authored-by: alexander <alexander.auer@cern.ch>
The merge of stack/registry kept a helper and a section header from the old test_config.py; the helper referred to resolve() and BackendConfig.model_validate(), which no longer exist.
Brings in the per-kind configuration models of stack/registry, and moves the credential code onto them: - credential_requirements(config, scopes) in _backends/credentials.py replaces the methods on the removed internal settings classes, with the same rules: a CE asks for its endpoint with the configured token and/or proxy, SSH for user@host:port unless an SSH agent holds the key, and a local batch system for nothing. - registry.get_backend(config, credentials) checks the provider and passes it to the parts that authenticate: single-class backends and transports. - The credential requirement tests move from test_config.py to test_credentials.py and build their configurations through validate_config().
Two tests had moved without changing, which only made the diff harder to read.
# Conflicts: # src/intercede/_backends/fake.py # src/intercede/registry.py # tests/unit/conftest.py # tests/unit/test_registry.py
…, squash-merged as DIRACGrid#29)
A value given by mistake, such as a misspelled field or plain data in a CredentialSet, may be the secret itself. Validation errors now hide their input, as the configuration models already do.
The message comes from the consumer's code and may hold a token or a secret URL. The AuthenticationError names only its type; the exception itself is still available as __cause__.
The fields were described in comments, which the generated reference does not show. They now use attribute docstrings, as the configuration models do.
Name every argument and the return value, and drop the blank lines before closing quotes.
This was referenced Nov 3, 2026
4 tasks
4 tasks
…only allow one credential type instead of combinations
…n, and removed auth value from config as it was not used
Moves test_fake_backend.py to tests/unit/backends/fake/, ready for the HTCondor tests that follow. The tests themselves do not change. Part of DIRACGrid#23.
Renders a submit description from a plan: one process per copy, per-copy values indexed by ProcId, quoting, and refusing what a description cannot hold. It sets no policy on the job except keeping finished jobs until they are purged, so output can be collected again. Shared with the HTCondor scheduler to come (DIRACGrid#21), so it knows nothing about any backend. Expected values were checked against HTCondor 25. Part of DIRACGrid#23.
Builds the tools' command lines and reads what they print: condor_submit -terse, condor_rm, and the job ads of condor_q and condor_history, mapped to the common JobStatus. A finished job's result comes from its exit, a removed job keeps the result it had, and any hold that interCEde did not cause counts as aborted. Part of DIRACGrid#23.
interCEde no longer supports proxies, so a description has nothing to say about one.
Alexander-Auer
force-pushed
the
stack/htcondor-1-core
branch
from
November 6, 2026 13:11
be3a762 to
e05f6dd
Compare
Alexander-Auer
force-pushed
the
stack/htcondor-1-core
branch
from
November 16, 2026 08:24
e05f6dd to
c7d9385
Compare
Alexander-Auer
marked this pull request as ready for review
November 16, 2026 09:10
4 tasks
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
feat: the shared HTCondor core
Part of #23; #21 will reuse it. PR 1 of 6 for the HTCondor-CE backend.
No dependencies: this applies directly to
mainand can merge before or after #30.Adds
_backends/_htcondor, which holds what interCEde knows about HTCondor itself. Nothing calls it yet: the backend arrives in #35.What's added
submit: renders a submit description from a plan, with one process per copy and per-copy values indexed byProcId.commands: builds the tools' command lines, splits job ids into chunks of 100, and reads the output ofcondor_submit -terseandcondor_rm.status: reads job ads fromcondor_qandcondor_historyand maps them toJobStatus.test_fake_backend.pymoves totests/unit/backends/fake/, unchanged, so each backend has its own test folder.Decisions to check
leave_in_queue = (JobStatus == 4)), so output can be collected again. This is the only policy set on a job: no holds, retries or periodic removal.Aborted.LastJobStatusdecides, so a finished job that was purged staysDoneorFailed.ValueError.Testing
80 new unit tests, with no HTCondor needed. Expected values were checked against HTCondor 25 (
condor_submit -dry-runand a live schedd).