Repository navigation
fix(deployer): reject missing accelerator mappings - #43
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces thread-safe device material detection and caching across the deployer modules. It adds a locking mechanism to prevent race conditions during device detection, updates the Docker container creation logic to handle device mapping more robustly, and includes comprehensive tests to verify concurrent device detection and error handling. I have no feedback to provide as there were no review comments.
There was a problem hiding this comment.
🟡 Changes recommended
Empty device discovery allows privileged accelerator requests to pass with fallback-only UNKNOWN_* masks, permitting container creation without enforcing the requested device selection.
1 open issue
What changed in this PR
This PR addresses missing accelerator mappings in GPUStack Runtime’s deployment layer, targeting the incorrect Ascend device selection reported in gpustack/gpustack#6755.
Changes:
- Publish completed device mappings under a lock and retry discovery after empty results.
- Reject empty Docker device mappings and adapt Kubernetes’ policy-specific cache.
- Add tests for missing Ascend mappings, discovery recovery, and concurrent lookups.
| File | Description |
|---|---|
tests/gpustack_runtime/deployer/test_visible_devices_ordering.py |
Adds rejection, retry, and concurrent mapping tests. |
gpustack_runtime/deployer/kubernetes.py |
Adapts policy-specific caching to the new material lookup flow. |
gpustack_runtime/deployer/docker.py |
Consolidates device requests and rejects empty mappings. |
gpustack_runtime/deployer/__types__.py |
Synchronizes cache initialization and leaves empty discovery results uncached. |
Review effort: Balanced
- Publish completed device mappings atomically so concurrent workload creation sees consistent results. - Retry empty device discovery results and preserve policy-specific Kubernetes device mapping caches. - Reject Docker accelerator requests without mappings and cover missing devices and concurrent initialization. Signed-off-by: yxf <xf.ye@gpustack.ai>
7665adb to
c2ddfb8
Compare

Publish device mappings only after detection completes and retry empty results on the next lookup. Reject Docker workloads when their requested accelerator mapping is missing so a privileged container cannot start with an empty device mask.
Ref gpustack/gpustack#6755
Validation
make lintpassed.2,3for prefill and0,1for decode.Hardware validation covered Docker container configuration. Full vLLM PD inference is pending.