Skip to content

fix(kubernetes): 非特权 workload 显式禁止权限提升 - #44

Open
zbb88888 wants to merge 2 commits into
gpustack:mainfrom
zbb88888:security/kubernetes-no-new-privs
Open

zbb88888 wants to merge 2 commits into
gpustack:mainfrom
zbb88888:security/kubernetes-no-new-privs

Conversation

@zbb88888

Copy link
Copy Markdown

基线

基于 gpustack/runtime main@42d51503,推送前 fetch/rebase。接续 qbox/gstack#479 和 qbox/gstack-kubespray#244。

问题

Kubernetes 生成端省略 allowPrivilegeEscalation,非特权容器仍可借助 setuid/file-capability 可执行文件提权,不能将省略视作 no_new_privs。

修改

在资源注入完成后、追加容器前,根据最终 securityContext 显式设置 allowPrivilegeEscalation。普通/init 容器默认 false;保留 privileged 或 CAP_SYS_ADMIN 的 Kubernetes 兼容语义(true)。不改变 Docker/Podman 或 Worker DaemonSet,不管理外部注入的 ephemeral containers。

验证

通过 make prepare 和 wheel 安装验证构建,5 项新测试 + 13 项现有特权解析测试通过。make lint 因本地缺 uv 未执行;本地 ruff 报既有 UP042/FURB110 和既有 event-sort 格式差异,未修改无关代码。未部署,未验证 GPU/RDMA/其他厂商硬件兼容性。

后续

runtime 合入发布后更新 gstack 固定依赖,验证新模型重建与推理,再要求 admission 中所有普通/init/ephemeral 容器显式 false。现有 privileged/SYS_ADMIN workload 不因此变成安全隔离容器。

Signed-off-by: zhangbingbing <zbb88888@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 11, 2026 07:53

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a helper function _set_privilege_escalation to dynamically configure the allow_privilege_escalation security context field for Kubernetes containers, along with associated unit tests. The review feedback suggests improving the capability detection logic to be case-insensitive and to support alternative formats such as CAP_SYS_ADMIN, ALL, and CAP_ALL. Additionally, it recommends expanding the test suite to cover these capability variations.

Comment on lines +695 to +703
def _set_privilege_escalation(container: kubernetes.client.V1Container) -> None:
security = container.security_context or kubernetes.client.V1SecurityContext()
capabilities = security.capabilities
# Kubernetes cannot disable escalation for privileged containers or CAP_SYS_ADMIN.
security.allow_privilege_escalation = bool(
security.privileged
or (capabilities and "SYS_ADMIN" in (capabilities.add or [])),
)
container.security_context = security

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

在 Kubernetes 中,用房声明的 Capability 可能会带有 CAP_ 前缀(例如 CAP_SYS_ADMIN),或者使用 ALL / CAP_ALL 来启用所有权限。此外,这些字焦串可能是大小写不敏感的。

如果用户指定了 CAP_SYS_ADMIN、ALL 或 CAP_ALL,当前的检查 "SYS_ADMIN" in (capabilities.add or []) 将无法识别,从而导致 allow_privilege_escalation 被错误地设置为 False,这可能会导致容器因权限不足而无法正常运行。

建议对 Capability 进行不区分大小写的处理,并同时支持 CAP_SYS_ADMIN、ALL 和 CAP_ALL。

Suggested change
def _set_privilege_escalation(container: kubernetes.client.V1Container) -> None:
security = container.security_context or kubernetes.client.V1SecurityContext()
capabilities = security.capabilities
# Kubernetes cannot disable escalation for privileged containers or CAP_SYS_ADMIN.
security.allow_privilege_escalation = bool(
security.privileged
or (capabilities and "SYS_ADMIN" in (capabilities.add or [])),
)
container.security_context = security
def _set_privilege_escalation(container: kubernetes.client.V1Container) -> None:
security = container.security_context or kubernetes.client.V1SecurityContext()
capabilities = security.capabilities
# Kubernetes cannot disable escalation for privileged containers or CAP_SYS_ADMIN.
add_caps = {cap.upper() for cap in (capabilities.add or [])} if capabilities else set()
security.allow_privilege_escalation = bool(
security.privileged
or {"SYS_ADMIN", "CAP_SYS_ADMIN", "ALL", "CAP_ALL"}.intersection(add_caps),
)
container.security_context = security

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已修复。新增 capability 规范化:大小写不敏感,去除可选 CAP_ 前缀,并识别 SYS_ADMIN、CAP_SYS_ADMIN、ALL、CAP_ALL。其他 capability(如 IPC_LOCK)仍可显式关闭权限提升。新增 6 个覆盖前缀/大小写/ALL 的测试,连同既有特权测试共 23 项通过。未把不确定的 capability 字符串扩大为允许提权。提交已推送到 PR 分支。

Comment on lines +15 to +20
(
kubernetes.client.V1SecurityContext(
capabilities=kubernetes.client.V1Capabilities(add=["SYS_ADMIN"]),
),
True,
),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

建议增加对带有 CAP_ 前缀、大小写混合以及 ALL 等 Capability 的测试用例,以确保新逻辑的正确性。

        (
            kubernetes.client.V1SecurityContext(
                capabilities=kubernetes.client.V1Capabilities(add=["SYS_ADMIN"]),
            ),
            True,
        ),
        (
            kubernetes.client.V1SecurityContext(
                capabilities=kubernetes.client.V1Capabilities(add=["cap_sys_admin"]),
            ),
            True,
        ),
        (
            kubernetes.client.V1SecurityContext(
                capabilities=kubernetes.client.V1Capabilities(add=["ALL"]),
            ),
            True,
        ),

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已补充 SYS_ADMIN、CAP_SYS_ADMIN、sys_admin、cap_sys_admin、ALL、CAP_ALL 六个回归样例;23 项 runtime deployer 测试通过。

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Cluster validation is still needed to confirm no_new_privs enforcement across generated workloads and compatibility with supported GPU/RDMA images that may rely on setuid or file capabilities.

No open issues 🎉

What changed in this PR

This PR hardens Kubernetes workload creation by explicitly disabling privilege escalation for non-privileged containers, while preserving Kubernetes behavior for privileged containers and those granted SYS_ADMIN.

Changes:

  • Sets allowPrivilegeEscalation from the final security context after resource injection, for both regular and init containers.
  • Adds five parameterized cases covering default, privileged, and capability-based behavior.
  • Leaves Docker, Podman, and externally injected ephemeral containers unchanged.
File Description
tests/​gpustack_runtime/​deployer/​test_privilege_escalation.py Adds security-context tests for privilege-escalation settings.
gpustack_runtime/​deployer/​kubernetes.py Applies explicit privilege-escalation settings before appending each container to the pod.

Review effort: Balanced

Signed-off-by: zbb88888 <jmdxjsjgcxy@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants