Repository navigation
feat(release): sync staging to main for Omnia 2.3 RC3 - #5812
abhishek-sa1 wants to merge 260 commits into
Conversation
…hestrator_config - Add validate_additional_cloud_init (default false) to test_config.yml - Check the flag in _load_additional_cloud_init_config before touching OIM - Remove orchestrator_config.yml from aci_enabled dataset; the dataset now only supplies additional_cloud_init.yml via sync overlay - Update dataset and fvt README skip documentation Signed-off-by: sayuri <sayuri.kamble@dell.com>
- Change additional_cloud_init_smd_groups from V095 to V600 - Change additional_cloud_init_metadata_groups from V096 to V601 - Update both README sections to match Signed-off-by: sayuri <sayuri.kamble@dell.com>
…use-only-rhel10.2 Use Image Thrillhouse as the only image build engine
* fix(utils) - security fix - Remove password-based SSH authentication from install_os deploy workflow (#5509) Changes: 1. Remove sshpass-based password authentication from iso_delivery/tasks/deploy_os.yml - Removed "Verify SSH access with root password" task - Removed "Warn if passwordless SSH key access failed" task - Removed "Warn if root password SSH access failed" task - Removed "Fail if both SSH key and password authentication fail" task - Simplified to single SSH key-based verification with retries 2. Fix ssh_private_key_path derivation in iso_delivery/vars/main.yml - Fixed regex_replace pattern from '\\.pub$' to '\.pub$' (YAML folded scalar escaping issue) - Added trim filter to handle trailing newlines - Added true flag to default() to handle empty strings correctly - This bug was causing ssh_private_key_path to remain as .pub file (public key instead of private key) 3. Update SSH key validation in validate_install_os_config/tasks/main.yml - Added deploy mode to SSH key validation condition - SSH key is now mandatory for all modes (build_iso, generate_ks, deploy, end-to-end) 4. Update error messages in iso_delivery/vars/main.yml - Removed ssh_password_fail_msg (no longer needed) - Renamed passwordless_ssh_fail_msg to ssh_key_auth_fail_msg - Updated os_install_success_msg to reference "SSH key authentication: verified" 5. Update documentation - iso_delivery/README.md: Added SSH Verification section explaining key-only auth - input-contract.md: Updated ssh_public_key_path description to include deploy flow - input-contract.md: Updated os_root_password description to clarify manual access only 6. Add test coverage - test_playbook.py: Added test_negative_deploy_mode_missing_ssh_key() - Tests that deploy mode fails when SSH key is missing Security improvements: - Eliminated password exposure in process lists and shell history (sshpass removal) - Enforced SSH key-based authentication as the only verification method Signed-off-by: venu <236371043+Venu-p1@users.noreply.github.com> * feat(build_stream): separate PXE restart processing Move BuildStream retry preparation and result persistence into dedicated pre- and post-PXE playbooks so the orchestrator PXE flow remains independent. Signed-off-by: SOWJANYAJAGADISH123 <sowjanya.jagadish@dell.com> * fix(build_stream): keep restart flow in one playbook Keep the dependent pre- and post-PXE processing within restart.yml so they cannot be invoked as unsupported standalone workflows. Signed-off-by: SOWJANYAJAGADISH123 <sowjanya.jagadish@dell.com> --------- Signed-off-by: venu <236371043+Venu-p1@users.noreply.github.com> Signed-off-by: SOWJANYAJAGADISH123 <sowjanya.jagadish@dell.com> Co-authored-by: Abhishek S A <abhishek.sa3@dell.com> Co-authored-by: Venu-p1 <236371043+Venu-p1@users.noreply.github.com>
1. Rename idempotency tests to state stability (finding 1) 2. Remove upgrade-mode test that cannot verify upgrade behavior (finding 2) 3. Remove exclusivity guards from common-only/per-FG-only E2E tests so a single dataset with both sections exercises both (finding 3) 4. Add rpm -q bash artifact to dataset for package integration coverage 5. Fail on uncovered FGs in combined/mixed-directive tests (finding 4) 6. Fail when runcmd has no verifiable artifact instead of falling back to cloud-init status (finding 5) 7. Fail on unreachable nodes in RHEL compatibility check (finding 6) 8. Align test_config docs with code: missing file fails, not skips (finding 7) Signed-off-by: sayuri <sayuri.kamble@dell.com>
Signed-off-by: sayuri <sayuri.kamble@dell.com>
Replace module-level pytestmark with per-function decorators to match the codebase convention used across all other pxeboot test files. Signed-off-by: sayuri <sayuri.kamble@dell.com>
Signed-off-by: sayuri <sayuri.kamble@dell.com>
…r, and Alpine (#5577) - Containerfile: add apk upgrade for OS-level CVEs (curl, zlib, glibc, python-3.13, pip); upgrade uv 0.11.15->0.12.2 for rustls/quinn-proto/ quick-xml cargo CVEs; upgrade pip and clean vendored setuptools 70.3.0 metadata (CVE-2025-47273, CVE-2026-59890) - pyproject.toml: bump PyJWT>=2.15.0, urllib3>=2.8.0, cryptography>=50.0.0, starlette>=1.3.1, ansible-core>=2.21.2, pydantic-settings>=2.14.2, pydantic>=2.11.0; add anyio>=4.14.2, mako>=1.4.2, setuptools>=78.1.1, msgpack>=1.2.1 - uv.lock: regenerate with all updated dependencies - hosted_gitlab/cleanup_gitlab vars: upgrade GitLab Runner v19.2.0->v19.5.0, Alpine 3.23.3->3.24.2 CVEs addressed: CVE-2026-102268, CVE-2026-102271, CVE-2026-102273, CVE-2026-102272, CVE-2026-102267, CVE-2026-102266, CVE-2026-101917, CVE-2026-101918, CVE-2026-97689, CVE-2026-97687, CVE-2026-63374, CVE-2026-64847, CVE-2026-69248, CVE-2026-69249, CVE-2026-69247, CVE-2026-54283, CVE-2026-11332, CVE-2026-57585, CVE-2026-102991, CVE-2024-6345, CVE-2026-59890, CVE-2025-47273, CVE-2026-19931, CVE-2026-18924, CVE-2026-82209, CVE-2026-85091, CVE-2026-19499, CVE-2026-82049, GHSA-2mjx-qc3c-rqvc, GHSA-c8m5-c952-6cm4, GHSA-4xgf-cpjx-pc3j, RUSTSEC-2026-0195, RUSTSEC-2026-0194 Signed-off-by: SOWJANYAJAGADISH123 <sowjanya.jagadish@dell.com>
Update CUDA path detection to use dynamic platform detection instead of hardcoded paths. The test now sources omnia_platform.sh to derive OMNIA_PLATFORM_ROOT dynamically based on OS version and architecture, making it work across different RHEL versions and architectures. Signed-off-by: sayuri <sayuri.kamble@dell.com>
…ests test(orchestrator): add additional cloud-init extended, idempotency, and e2e FVT tests
…mponents to v1.18.0 (#5502) * Update PowerScale CSI and telemetry for Dell CSM 1.18 compatibility Update PowerScale CSI and telemetry for Dell CSM 1.18 compatibility - Upgrade CSI PowerScale driver: v2.17.0 → v2.18.0 - Upgrade all CSI sidecars to latest versions: - attacher: v4.11.0 → v4.13.0 - provisioner: v6.2.0 → v6.3.0 - snapshotter: v8.5.0 → v8.6.0 - resizer: v2.1.0 → v2.2.1 - registrar: v2.16.0 → v2.18.0 - health-monitor: v0.17.0 → v0.18.0 - Upgrade all CSM sidecars: - replicator: v1.15.0 → v1.16.0 - podmon: v1.16.0 → v1.17.0 - authorization-sidecar: v2.5.0 → v2.6.0 - metadata-retriever: v1.14.0 → v1.15.0 - Upgrade Karavi metrics images: - PowerScale: v1.12.0 → v1.13.0 - PowerFlex: v1.15.0 → v1.16.0 - PowerStore: v1.15.0 → v1.16.0 - PowerMax: v1.10.0 → v1.11.0 - Upgrade git repos: - karavi-observability: v1.15.0 → v1.16.0 - helm-charts: 1.10.0 → 1.11.0 - Update CSM Observability Helm chart reference: v1.17.1 → v1.18.0 - Add cert-manager-readiness-check cleanup resources (4 namespaced + 2 cluster-scoped) - Update catalog JSON files with all new CSI/CSM versions - Update build_stream, repo_manager, and test files with new CSI version constant All versions verified against official Dell helm-charts release-v1.18.0 Signed-off-by: Mithilesh Reddy <mithilesh.reddy@dell.com> * Update powerscale latest version images Signed-off-by: Mithilesh Reddy <mithilesh.reddy@dell.com> * Include bitname image in catalogs Signed-off-by: Mithilesh Reddy <mithilesh.reddy@dell.com> * Fix powerscale telemtry for latest CSM version upgrade * Fix deploy powerscale telemetry OOM issue * Fix victoria logs TLS timeout issue for powerscale * Fix some telemetry bugs * Powerscale telemetry and catalog version fixes * Fix external-snapshotter package placement in catalogs * Fix ansible lint issues * Update repo_status.yml sample file * Remove broken cert-manager post-install hook causing webhook connection errors --------- Signed-off-by: Mithilesh Reddy <mithilesh.reddy@dell.com>
* feat: Update telemetry and catalog package versions - VictoriaMetrics operator: 0.59.3 → 0.68.1 - VictoriaMetrics components (vmagent, vmstorage, vmselect, vminsert): v1.149.0 → v1.153.0 - VictoriaLogs components (vlstorage, vlselect, vlinsert, vlagent): v1.50.0 → v1.53.0 - Victoria operator: v0.68.3 → v0.75.0 - Strimzi Kafka operator: 1.1.0 → 1.2.0 - Strimzi Kafka bridge: 1.0.0 → 1.2.0 - Strimzi Kafka broker: 1.1.0-kafka-4.3.0 → 1.2.0-kafka-4.3.1 - Vector: 0.54.0-debian → 0.59.0-debian - ActiveMQ: 6.3.0 → 6.3.2 - curl: 8.17.0 → 8.22.0 - Updated vlagent init container to use busybox (required for v1.53.0 distroless base image) - Updated orchestrator inventory group names to include OS version (rhel_10_0) - Updated documentation examples to reflect new versions - Updated catalog files across RHEL 10.0 and 10.2 variants Signed-off-by: Kratika_Patidar <Kratika.Patidar@dell.com> * fix(telemetry): UFM FVT distinguish never-enabled from deleted-secret Add manifest path detection to check if UFM was previously enabled when the credentials secret is absent. Previously, the test unconditionally failed when the secret was missing, even on clusters where UFM was never enabled. Now it checks the secret manifest on the k8s cluster mount: - Manifest present + secret missing = error (should be preserved) - Manifest absent + secret missing = pass (never enabled, nothing to preserve) Changes: - Add CFG_KEY_K8S_CLUSTER_MOUNT, UFM_DEPLOYMENTS_SUBDIR, UFM_SECRET_MANIFEST_FILE - Add _resolve_ufm_secret_manifest_path() helper - Rewrite verify_ufm_resources_absent() to return secret_present, previously_enabled, manifest_path - Update test_ufm_disabled_state to use new return fields - Update assert message to clarify conditional secret preservation Signed-off-by: Kratika_Patidar <Kratika.Patidar@dell.com> --------- Signed-off-by: Kratika_Patidar <Kratika.Patidar@dell.com> Signed-off-by: Kratika Patidar <Kratika.Patidar@dell.com>
* fix(utils) - security fix - Remove password-based SSH authentication from install_os deploy workflow (#5509) Changes: 1. Remove sshpass-based password authentication from iso_delivery/tasks/deploy_os.yml - Removed "Verify SSH access with root password" task - Removed "Warn if passwordless SSH key access failed" task - Removed "Warn if root password SSH access failed" task - Removed "Fail if both SSH key and password authentication fail" task - Simplified to single SSH key-based verification with retries 2. Fix ssh_private_key_path derivation in iso_delivery/vars/main.yml - Fixed regex_replace pattern from '\\.pub$' to '\.pub$' (YAML folded scalar escaping issue) - Added trim filter to handle trailing newlines - Added true flag to default() to handle empty strings correctly - This bug was causing ssh_private_key_path to remain as .pub file (public key instead of private key) 3. Update SSH key validation in validate_install_os_config/tasks/main.yml - Added deploy mode to SSH key validation condition - SSH key is now mandatory for all modes (build_iso, generate_ks, deploy, end-to-end) 4. Update error messages in iso_delivery/vars/main.yml - Removed ssh_password_fail_msg (no longer needed) - Renamed passwordless_ssh_fail_msg to ssh_key_auth_fail_msg - Updated os_install_success_msg to reference "SSH key authentication: verified" 5. Update documentation - iso_delivery/README.md: Added SSH Verification section explaining key-only auth - input-contract.md: Updated ssh_public_key_path description to include deploy flow - input-contract.md: Updated os_root_password description to clarify manual access only 6. Add test coverage - test_playbook.py: Added test_negative_deploy_mode_missing_ssh_key() - Tests that deploy mode fails when SSH key is missing Security improvements: - Eliminated password exposure in process lists and shell history (sshpass removal) - Enforced SSH key-based authentication as the only verification method Signed-off-by: venu <236371043+Venu-p1@users.noreply.github.com> * feat(orchestrator): update PXE mapping file examples with OS-versioned functional groups and also update default catalog as hybrid catalog (#5566) * Update pxe_mapping_file.csv Signed-off-by: Katakam Rakesh Naga Sai <125246792+Katakam-Rakesh@users.noreply.github.com> * Create pxe_mapping_file_slurm_service_k8s_hybrid.csv Signed-off-by: Katakam Rakesh Naga Sai <125246792+Katakam-Rakesh@users.noreply.github.com> * Update pxe_mapping_file.csv Signed-off-by: Katakam Rakesh Naga Sai <125246792+Katakam-Rakesh@users.noreply.github.com> * Update pxe_mapping_file.csv Signed-off-by: Katakam Rakesh Naga Sai <125246792+Katakam-Rakesh@users.noreply.github.com> * Update default catalog to hybrid RHEL 10.2/10.0 multi-version layout - Restructure catalog_rhel.json to hybrid catalog: RHEL 10.2 x86_64 (Slurm control, K8s control-plane, K8s node) + RHEL 10.0 x86_64 (Slurm login) + RHEL 10.0 aarch64 (Slurm node, login-compiler), without VAST - Split baseos_group into versioned baseos_group_10.0 and baseos_group_10.2 to support mixed OS versions - Add new os_rhel_10_0_x86_64 functional layer in catalog - Update pxe_mapping_file.csv: align functional group names as per the hybrid catalog and add new os_rhel_10_0_aarch64 entry - Update docs (omnia-env.md, samples/README.md, main/README.md) to reflect hybrid catalog structure Signed-off-by: Katakam-Rakesh <katakam.rakesh@dell.com> * Update pxe_mapping_file_slurm_service_k8s_hybrid.csv Signed-off-by: Katakam Rakesh Naga Sai <125246792+Katakam-Rakesh@users.noreply.github.com> * Update login_node from rhel_10_0 to rhel_10_2 in hybrid catalogs - Rename login_node_rhel_10_0_x86_64 to login_node_rhel_10_2_x86_64 in all 8 hybrid catalogs and the default catalog_rhel.json - Update baseos_group_10.0 to baseos_group_10.2 in the login node functional layer - Align pxe_mapping_file.csv and hybrid PXE example with the updated functional group name - Update docs (omnia-env.md, samples/README.md) and catalog description to reflect login node on RHEL 10.2 Signed-off-by: Katakam-Rakesh <katakam.rakesh@dell.com> * Update hybrid catalog descriptions to reflect login node on RHEL 10.2 - Update description field in all 8 hybrid catalogs to list login node under RHEL 10.2 instead of RHEL 10.0 Signed-off-by: Katakam-Rakesh <katakam.rakesh@dell.com> * Update login_node from rhel_10_0 to rhel_10_2 in aarch64 hybrid catalogs - Rename login_node_rhel_10_0_aarch64 to login_node_rhel_10_2_aarch64 in slurm_hybrid_10_2_10_0_aarch64 and its no_vast variant - Update baseos_group_10.0 to baseos_group_10.2 in the login node functional layer - Update catalog descriptions accordingly Signed-off-by: Katakam-Rakesh <katakam.rakesh@dell.com> --------- Signed-off-by: Katakam Rakesh Naga Sai <125246792+Katakam-Rakesh@users.noreply.github.com> Signed-off-by: Katakam-Rakesh <katakam.rakesh@dell.com> * fix(discovery): Improve compliance, add OME integration tests, and fix functional group fallback (#5565) * fix(discovery): Improve compliance, add OME integration tests, and fix functional group fallback This PR enhances the Discovery module with improved compliance, OME integration testing, and functional group handling for Omnia 2.3. ## Changes ### Source Code (src/discovery) - Add network_spec.yml JSON schema validation - Add .lock files to cleanup paths (align with orchestrator) - Add no_log: true on credential tasks - Add return type annotations to Python functions - Fix cross-domain reference in generate_discovery_report.yml - Add architecture documentation ### Test Automation (test/discovery) - Add OME integration tests (V010-V014): - Verify OME static groups match PXE mapping - Verify devices without static group assignment - Verify ADMIN_MAC matches OME NIC - Verify IB_NIC_NAME matches OME IB NIC - Verify PXE mapping CSV format for Orchestrator - Add PXE mapping validation tests (V001-V009) - Add OME domain credentials support to setup_env.sh - Fix NFT test order to prevent credential removal before execute - Add nosec comments for Bandit B108 in test code - Accept valid default groups in OME static group check ### Functional Group Handling - Enforce OS version suffix pattern: <prefix>_rhel_<major>_<minor>_<arch> - Fallback to default functional group for invalid OME groups - Support any custom prefix with required rhel version suffix ### Compliance Improvements - Improve compliance score from 93.68 to 96.84 - Add Bandit B108 nosec comments for test temp directories Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> * fix(discovery): Improve compliance, add OME integration tests, and fix functional group fallback This PR enhances the Discovery module with improved compliance, OME integration testing, and functional group handling for Omnia 2.3. - Add network_spec.yml JSON schema validation - Add .lock files to cleanup paths (align with orchestrator) - Add no_log: true on credential tasks - Add return type annotations to Python functions - Fix cross-domain reference in generate_discovery_report.yml - Add architecture documentation - Add OME integration tests (V010-V014): - Verify OME static groups match PXE mapping - Verify devices without static group assignment - Verify ADMIN_MAC matches OME NIC - Verify IB_NIC_NAME matches OME IB NIC - Verify PXE mapping CSV format for Orchestrator - Add PXE mapping validation tests (V001-V009) - Add OME domain credentials support to setup_env.sh - Fix NFT test order to prevent credential removal before execute - Add nosec comments for Bandit B108 in test code - Accept valid default groups in OME static group check - Enforce OS version suffix pattern: <prefix>_rhel_<major>_<minor>_<arch> - Fallback to default functional group for invalid OME groups - Support any custom prefix with required rhel version suffix - Improve compliance score from 93.68 to 96.84 - Add Bandit B108 nosec comments for test temp directories Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> --------- Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> Co-authored-by: Abhishek S A <abhishek.sa3@dell.com> * fix(image_build_manager): allow watcher container transition (#5570) Refresh the managed SELinux compatibility module when its source changes and install it before any Podman access so watcher-triggered builds retain system hardening while entering the container runtime domain. Signed-off-by: SOWJANYAJAGADISH123 <sowjanya.jagadish@dell.com> Co-authored-by: Rajeshkumar S <rajeshkumar.s2@dell.com> * fix(build_stream): only mark ImageGroup FAILED on validate failure (#5573) Deploy and restart stage failures are job-level failures (e.g. configuration issues, queue submission errors) and do not indicate anything wrong with the images themselves. Previously, ImageGroup status was incorrectly transitioned to FAILED on deploy or restart playbook failures. Changes: - Remove _on_deploy_failure() call and method from ResultPoller - Remove _on_restart_failure() call and method from ResultPoller - Keep _on_restart_completed() for node_results/failed_nodes persistence - Update STATUS_FLOW on_failure to None for deploy and restart - Only validate failure transitions ImageGroup to FAILED - Update existing unit tests to match new behavior Job state, stage state, resume/retry flows, and artifact persistence are all unaffected by this change. Signed-off-by: SOWJANYAJAGADISH123 <sowjanya.jagadish@dell.com> Co-authored-by: Rajeshkumar S <rajeshkumar.s2@dell.com> * fix(orchestrator): renew OpenCHAMI tokens before provision (#5571) When running provision to add a node after the cluster has been running for more than 24 hours, the playbook fails with OpenCHAMI authentication errors. This is because metadata-service and boot-service exchange a bootstrap token at startup for a refresh token family that expires after 24 hours. Once expired, the services can no longer authenticate with SMD. This workaround restarts metadata-service, boot-service, and HAProxy in the provision preamble before OpenCHAMI authentication. The restart forces the services to re-exchange their bootstrap tokens and obtain a fresh refresh token family with a new 24-hour window. The restart is also added at the end of deploy_openchami for consistency, but this may be redundant since containers already start fresh during deployment. If the team agrees, this can be removed in a follow-up. This is a bounded Omnia recovery workaround until the upstream OpenCHAMI packaging provides a durable credential lifecycle (e.g., systemd timer for periodic token refresh). Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com> Co-authored-by: Sujit Jadhav <sujit.jadhav@dell.com> * fix(orchestrator): skip framework CUDA trees in HPC tools binary scan and drop buildstream marker The source-only binary scan in _hpc_benchmarks_helpers walked every executable under the staged hpc_tools paths, so CUDA trees staged under /hpc_tools/platforms/<os>/<ver>/<arch>/cuda were reported as tool binaries. The find command now prunes the framework-owned subdirectories (_STAGING_EXCLUDED_SUBDIRS) at any depth, and the scan path is shell-quoted. Remove the buildstream marker from the OpenMPI and HPC benchmarks source-only delivery tests so they are not selected in BuildStream runs. Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com> * fix(repo_manager): catalog add/delete merge sources for multi-arch catalogs and align AI skills with code (#5579) * Fix catalog add/delete for multi-arch catalogs and align AI skills with code Remove the buildstream marker from the HPC benchmarks source-only delivery test (V206). repo_manager catalog tooling: - add merges package sources per architecture and OS version instead of replacing the entry, so updates no longer drop other platforms; an identical re-add is a no-op - repeated package keys in one input file merge instead of last-wins - delete logs the functional layers affected when an emptied group is removed - legacy 1.0 catalogs get a "run transform first" message - unit tests for the merge behaviour BuildStream AI skills: - bulk-edit: full RHEL version switch (layer names, sources, base_os, metadata) and allowed_root on writes; worked example re-verified - catalog-editing: merge semantics, every-layer additions on hybrid catalogs, delete side effect, runtime catalogs out of scope - catalog-generation: repo_manager rules schema validation misses (layer naming, single OS type, source coverage); hybrid login note - master reference: tables regenerated from current catalogs and repo config (RHEL 10.2 sources, image tags, moved template paths) - catalog-diff: exit code 2 without --schema, HTML needs jinja2, example numbers updated - remove stale cross-skill claims; align manifests and READMEs - reference-integrity test now checks every src/ path, relative refs, manifest dependencies and description drift Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com> * docs(build_stream): replace 'offline' with specific terminology in AI skills Replace ambiguous 'offline' references with more specific terminology across BuildStream AI skills to clarify when online sources are unavailable and the master reference file is used as fallback. Changes: - 'offline gaps' → 'gaps when online sources are unavailable' - 'offline fallback' → 'fallback' or 'master reference file data' - 'online/offline' → 'online source lookup and fallback' - 'operates entirely offline' → 'operates without live dependency resolution' - 'offline mode' → 'when using the master reference file' This addresses reviewer feedback that 'offline' was being introduced in multiple skills and could cause confusion. --------- Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com> Co-authored-by: Abhishek S A <abhishek.sa3@dell.com> * fix(test): improve CUDA path detection with platform-aware checks Update CUDA path detection to use dynamic platform detection with proper validation: - Remove unnecessary backslash escapes that prevented variable expansion - Use . instead of source for consistency with other commands - Check for .done_cuda marker to verify complete installation - Check for executable nvcc instead of just file existence - Update lock check to use platform-specific .nfs_lock_cuda directory - Remove hardcoded /hpc_tools/cuda path from test function output - Dynamically report the detected CUDA path in test results This ensures tests work across different RHEL versions and architectures while properly validating that CUDA installation is complete. Signed-off-by: sayuri <sayuri.kamble@dell.com> * docs(ai_skills): fix catalog skills documentation issues and remove internal ER references (#5586) * fix(orchestrator): remove buildstream and sanity markers from failed test cases Remove @pytest.mark.buildstream and @pytest.mark.sanity markers from three test cases that are currently failing, so they are not selected in BuildStream or sanity test runs: - test_slurm_reconfigure (V029) - Slurm reconfiguration test - test_dcgm_cuda_login_compiler (V507) - CUDA toolkit on login_compiler - test_apptainer_concurrent_jobs (V080) - Concurrent container jobs These tests will be re-enabled with the markers once the underlying issues are resolved. Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com> * fix(repo-manager): priority value for epel (#5588) * epel priotity setting externally Signed-off-by: Nagachandan P <Nagachandan.p@dell.com> * Update repo_manager_config.yml fixing lint issues Signed-off-by: Nagachandan P <Nagachandan.p@dell.com> --------- Signed-off-by: Nagachandan P <Nagachandan.p@dell.com> * Remove sanity markers from failing PowerVault and HPC benchmark tests (#5591) * Update test_negative.py Signed-off-by: balajikumaran.cs <balajikumaran.cs@dell.com> * Update test_contracts.py Signed-off-by: balajikumaran.cs <balajikumaran.cs@dell.com> * Fix pylint errors in test_negative.py and remove sanity markers Fixed verify_pxeboot function calls to use correct signature with TestLogger and test case from TC. Removed @pytest.mark.sanity markers from test_neg_powervault_gpt_missing_label, test_neg_powervault_duplicate_fstab, and test_hpc_benchmarks_source_only_delivery as requested in PR #5591. Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com> --------- Signed-off-by: balajikumaran.cs <balajikumaran.cs@dell.com> Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com> * Resolve merge conflicts in catalog_rhel.json - From main: use baseos_group_10.2 for RHEL 10.2 - From rhel 10.2: use imagethrillhouse and remove image_builder groups - Keep version 10.2 for packages where appropriate - Resolve version conflicts to include both 10.2 and 10.0 where needed - Align with hybrid catalog structure Signed-off-by: SOWJANYAJAGADISH123 <sowjanya.jagadish@dell.com> --------- Signed-off-by: venu <236371043+Venu-p1@users.noreply.github.com> Signed-off-by: Katakam Rakesh Naga Sai <125246792+Katakam-Rakesh@users.noreply.github.com> Signed-off-by: Katakam-Rakesh <katakam.rakesh@dell.com> Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> Signed-off-by: SOWJANYAJAGADISH123 <sowjanya.jagadish@dell.com> Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com> Signed-off-by: sayuri <sayuri.kamble@dell.com> Signed-off-by: Nagachandan P <Nagachandan.p@dell.com> Signed-off-by: balajikumaran.cs <balajikumaran.cs@dell.com> Signed-off-by: saifudheene <saifudheen.e@dell.com> Signed-off-by: Sujit Jadhav <sujit.jadhav@dell.com> Signed-off-by: Rohith Ravut <rohith.ravut@dellteam.com> Signed-off-by: Abhishek S A <abhishek.sa3@dell.com> Signed-off-by: Mithilesh Reddy <mithilesh.reddy@dell.com> Signed-off-by: Rajeshkumar S <rajeshkumar.s2@dell.com> Signed-off-by: Narthan_S <narthan.s@dell.com> Signed-off-by: snarthan <narthan.s@dell.com> Signed-off-by: esaifudheen <esaifudheen@dell.com> Signed-off-by: Nagachandan-P <Nagachandan.p@dell.com> Signed-off-by: Venu-p1 <236371043+Venu-p1@users.noreply.github.com> Signed-off-by: VirajPatil-1500 <VirajPatil-1500@users.noreply.github.com> Signed-off-by: VirajPatil-1500 <Viraj.Patil@dellteam.com> Signed-off-by: Vrinda_Marwah <Vrinda.Marwah@dell.com> Signed-off-by: Sujal27 <sujal.sooryavamshi@dell.com> Signed-off-by: priti_parate <priti_parate@dell.com> Signed-off-by: priti-parate <140157516+priti-parate@users.noreply.github.com> Signed-off-by: Kratika_Patidar <Kratika.Patidar@dell.com> Signed-off-by: mithileshreddy04 <mithilesh.reddy@dell.com> Signed-off-by: Sushmitha-dell <Sushmitha.u@dell.com> Signed-off-by: SujalS27 <Sujal.Sooryavamshi@dell.com> Signed-off-by: Mithilesh Reddy <mithilesh.reddy@dell.com>" Signed-off-by: Kratika Patidar <Kratika.Patidar@dell.com> Signed-off-by: sushmitha-dell <sushmitha-dell@users.noreply.github.com> Signed-off-by: U, Sushmitha <Sushmitha_U@Dell.com> Signed-off-by: Sushmitha U <Sushmitha.U@dell.com> Signed-off-by: Kratika-P <Kratika.Patidar@dell.com> Co-authored-by: Abhishek S A <abhishek.sa3@dell.com> Co-authored-by: Venu-p1 <236371043+Venu-p1@users.noreply.github.com> Co-authored-by: Katakam Rakesh Naga Sai <125246792+Katakam-Rakesh@users.noreply.github.com> Co-authored-by: SujalS27 <Sujal.Sooryavamshi@dell.com> Co-authored-by: Rajeshkumar S <rajeshkumar.s2@dell.com> Co-authored-by: balajikumaran.cs <balajikumaran.cs@dell.com> Co-authored-by: Sujit Jadhav <sujit.jadhav@dell.com> Co-authored-by: sayuri <sayuri.kamble@dell.com> Co-authored-by: Nagachandan P <Nagachandan.p@dell.com> Co-authored-by: saifudheene <Saifudheen.E@dell.com> Co-authored-by: Rohith Ravut <rohith.ravut@dellteam.com> Co-authored-by: Mithilesh Reddy <mithilesh.reddy@dell.com> Co-authored-by: Sayuri Kamble <68416000+SAYUK09@users.noreply.github.com> Co-authored-by: snarthan <narthan.s@dell.com> Co-authored-by: esaifudheen <esaifudheen@dell.com> Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: VirajPatil-1500 <Viraj.Patil@dellteam.com> Co-authored-by: VirajPatil-1500 <VirajPatil-1500@users.noreply.github.com> Co-authored-by: Katakam-Rakesh <katakam.rakesh@dell.com> Co-authored-by: Vrinda Marwah <vrinda.marwah@dell.com> Co-authored-by: priti-parate <140157516+priti-parate@users.noreply.github.com> Co-authored-by: Kratika Patidar <Kratika.Patidar@dell.com> Co-authored-by: Sushmitha U <Sushmitha.U@dell.com> Co-authored-by: sushmitha-dell <sushmitha-dell@users.noreply.github.com> Co-authored-by: U, Sushmitha <Sushmitha_U@Dell.com> Co-authored-by: Super User <root@sowjanya.omnia.test>
Update GitLab Community Edition package from 18.8.0-ce.0 to 19.5.0-ce.0 to address security vulnerabilities. The package name format requires the -ce.0 suffix for Community Edition packages. Signed-off-by: SOWJANYAJAGADISH123 <sowjanya.jagadish@dell.com>
* fix(discovery): Improve compliance, add OME integration tests, and fix functional group fallback This PR enhances the Discovery module with improved compliance, OME integration testing, and functional group handling for Omnia 2.3. ## Changes ### Source Code (src/discovery) - Add network_spec.yml JSON schema validation - Add .lock files to cleanup paths (align with orchestrator) - Add no_log: true on credential tasks - Add return type annotations to Python functions - Fix cross-domain reference in generate_discovery_report.yml - Add architecture documentation ### Test Automation (test/discovery) - Add OME integration tests (V010-V014): - Verify OME static groups match PXE mapping - Verify devices without static group assignment - Verify ADMIN_MAC matches OME NIC - Verify IB_NIC_NAME matches OME IB NIC - Verify PXE mapping CSV format for Orchestrator - Add PXE mapping validation tests (V001-V009) - Add OME domain credentials support to setup_env.sh - Fix NFT test order to prevent credential removal before execute - Add nosec comments for Bandit B108 in test code - Accept valid default groups in OME static group check ### Functional Group Handling - Enforce OS version suffix pattern: <prefix>_rhel_<major>_<minor>_<arch> - Fallback to default functional group for invalid OME groups - Support any custom prefix with required rhel version suffix ### Compliance Improvements - Improve compliance score from 93.68 to 96.84 - Add Bandit B108 nosec comments for test temp directories Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> * fix(discovery): Improve compliance, add OME integration tests, and fix functional group fallback This PR enhances the Discovery module with improved compliance, OME integration testing, and functional group handling for Omnia 2.3. - Add network_spec.yml JSON schema validation - Add .lock files to cleanup paths (align with orchestrator) - Add no_log: true on credential tasks - Add return type annotations to Python functions - Fix cross-domain reference in generate_discovery_report.yml - Add architecture documentation - Add OME integration tests (V010-V014): - Verify OME static groups match PXE mapping - Verify devices without static group assignment - Verify ADMIN_MAC matches OME NIC - Verify IB_NIC_NAME matches OME IB NIC - Verify PXE mapping CSV format for Orchestrator - Add PXE mapping validation tests (V001-V009) - Add OME domain credentials support to setup_env.sh - Fix NFT test order to prevent credential removal before execute - Add nosec comments for Bandit B108 in test code - Accept valid default groups in OME static group check - Enforce OS version suffix pattern: <prefix>_rhel_<major>_<minor>_<arch> - Fallback to default functional group for invalid OME groups - Support any custom prefix with required rhel version suffix - Improve compliance score from 93.68 to 96.84 - Add Bandit B108 nosec comments for test temp directories Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> * Fix security issues: SSRF in OME inventory and weak credential policy Issue #1153 (SSRF): - Add same-origin validation for OME navigation links - Add allow_redirects=False to prevent redirect-based token leak Issue #1146 (Weak Credentials): - Change re.match to re.fullmatch in Discovery and Utils - Raise ome_password minimum length from 1 to 8 - Replace no-op pattern with length-enforcing pattern - Remove 30 unused credential fields from Discovery Addresses security findings: - [OMN-SEC] [MEDIUM] 2720853351 (SSRF) - [OMN-SEC] [MEDIUM] 1362293893 (weak credentials) Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> * Fix insufficient IP validation on ome_ip for SSRF prevention Add link-local, reserved, multicast, and unspecified address checks to ome_ip validation in discovery_validation_flow.py. Blocks requests to cloud metadata endpoints (169.254.169.254) and other non-routable addresses while preserving private network IPs used by OME appliances. Addresses security finding [OMN-SEC] [MEDIUM] D-SRC-H1 Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> * Add OME password validation to credential collection Add validate_credentials call for ome_password in Discovery credential update flow to enforce the 8-character minimum length requirement from credential_rules.json. Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> * Remove Discovery UT tests for removed credential fields Remove tests for bmc_username, bmc_password, and provision_password from test_input_validation_schema.py since these fields were removed from credential_rules.json as part of security fix (Discovery only validates ome_username and ome_password). Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> * Move SSL warning suppression to conditional in OME client Remove global urllib3.disable_warnings at module level. Move warning suppression inside OMEClient.__init__ so it only applies when verify_ssl=False, and emit a log warning. This prevents process-wide suppression of TLS warnings for other modules in the same Ansible run. CA certificate support will be addressed in a separate Q4 BR. Addresses security finding [OMN-SEC] [MEDIUM] D-SRC-H2 Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> * Move SSL warning suppression to conditional in OME client Remove global urllib3.disable_warnings at module level. Move warning suppression inside OMEClient.__init__ so it only applies when verify_ssl=False, and emit a log warning. This prevents process-wide suppression of TLS warnings for other modules in the same Ansible run. CA certificate support will be addressed in a separate Q4 BR. Addresses security finding [OMN-SEC] [MEDIUM] D-SRC-H2 Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> * reverted the minimum length password change Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> * Fix Discovery domain security issues (SSRF, credential validation, IP validation, SSL warning) Issue #1153: SSRF in OME Inventory (CVE Finding 2720853351) - Add urljoin validation for navigation links with same-origin check - Add allow_redirects=False to prevent redirect-based token leak Issue #1146: Weak Credential Policy (CVE Finding 1362293893) - Change re.match to re.fullmatch in Discovery and Utils validate_credentials - Remove 30 unused credential fields from Discovery credential_rules.json - Password minimum length remains at 1 to match documentation Issue #1125: SSRF via ome_ip (CVE Finding D-SRC-H1) - Add validation to reject link-local, reserved, multicast, unspecified addresses Issue #1126: SSL/TLS Certificate Verification Warning (CVE Finding D-SRC-H2) - Remove global urllib3.disable_warnings - Move warning suppression to conditional (only when verify_ssl=False) - Add module.warn() for task output warning Test Fix: - Remove tests for removed credential fields Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> --------- Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> Signed-off-by: SOWJANYAJAGADISH123 <sowjanya.jagadish@dell.com> Signed-off-by: saifudheene <saifudheen.e@dell.com> Signed-off-by: Sujit Jadhav <sujit.jadhav@dell.com> Signed-off-by: Rohith Ravut <rohith.ravut@dellteam.com> Signed-off-by: Abhishek S A <abhishek.sa3@dell.com> Signed-off-by: Mithilesh Reddy <mithilesh.reddy@dell.com> Signed-off-by: sayuri <sayuri.kamble@dell.com> Signed-off-by: Rajeshkumar S <rajeshkumar.s2@dell.com> Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com> Signed-off-by: balajikumaran.cs <balajikumaran.cs@dell.com> Signed-off-by: Narthan_S <narthan.s@dell.com> Signed-off-by: snarthan <narthan.s@dell.com> Signed-off-by: esaifudheen <esaifudheen@dell.com> Signed-off-by: Nagachandan-P <Nagachandan.p@dell.com> Signed-off-by: venu <236371043+Venu-p1@users.noreply.github.com> Signed-off-by: Venu-p1 <236371043+Venu-p1@users.noreply.github.com> Signed-off-by: VirajPatil-1500 <VirajPatil-1500@users.noreply.github.com> Signed-off-by: Katakam-Rakesh <katakam.rakesh@dell.com> Signed-off-by: Nagachandan P <Nagachandan.p@dell.com> Signed-off-by: VirajPatil-1500 <Viraj.Patil@dellteam.com> Signed-off-by: Vrinda_Marwah <Vrinda.Marwah@dell.com> Signed-off-by: Sujal27 <sujal.sooryavamshi@dell.com> Signed-off-by: priti_parate <priti_parate@dell.com> Signed-off-by: priti-parate <140157516+priti-parate@users.noreply.github.com> Signed-off-by: Kratika_Patidar <Kratika.Patidar@dell.com> Signed-off-by: mithileshreddy04 <mithilesh.reddy@dell.com> Signed-off-by: Sushmitha-dell <Sushmitha.u@dell.com> Signed-off-by: SujalS27 <Sujal.Sooryavamshi@dell.com> Signed-off-by: Mithilesh Reddy <mithilesh.reddy@dell.com>" Signed-off-by: Kratika Patidar <Kratika.Patidar@dell.com> Signed-off-by: sushmitha-dell <sushmitha-dell@users.noreply.github.com> Signed-off-by: U, Sushmitha <Sushmitha_U@Dell.com> Signed-off-by: Sushmitha U <Sushmitha.U@dell.com> Signed-off-by: Kratika-P <Kratika.Patidar@dell.com> Signed-off-by: Katakam Rakesh Naga Sai <125246792+Katakam-Rakesh@users.noreply.github.com> Co-authored-by: Abhishek S A <abhishek.sa3@dell.com> Co-authored-by: Rajeshkumar S <rajeshkumar.s2@dell.com> Co-authored-by: SOWJANYAJAGADISH123 <Sowjanya.Jagadish@dell.com> Co-authored-by: saifudheene <Saifudheen.E@dell.com> Co-authored-by: Sujit Jadhav <sujit.jadhav@dell.com> Co-authored-by: Rohith Ravut <rohith.ravut@dellteam.com> Co-authored-by: Mithilesh Reddy <mithilesh.reddy@dell.com> Co-authored-by: Sayuri Kamble <68416000+SAYUK09@users.noreply.github.com> Co-authored-by: balajikumaran.cs <balajikumaran.cs@dell.com> Co-authored-by: snarthan <narthan.s@dell.com> Co-authored-by: esaifudheen <esaifudheen@dell.com> Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: Nagachandan P <Nagachandan.p@dell.com> Co-authored-by: Venu-p1 <236371043+Venu-p1@users.noreply.github.com> Co-authored-by: VirajPatil-1500 <Viraj.Patil@dellteam.com> Co-authored-by: VirajPatil-1500 <VirajPatil-1500@users.noreply.github.com> Co-authored-by: Katakam-Rakesh <katakam.rakesh@dell.com> Co-authored-by: Katakam Rakesh Naga Sai <125246792+Katakam-Rakesh@users.noreply.github.com> Co-authored-by: Vrinda Marwah <vrinda.marwah@dell.com> Co-authored-by: priti-parate <140157516+priti-parate@users.noreply.github.com> Co-authored-by: Kratika Patidar <Kratika.Patidar@dell.com> Co-authored-by: Sushmitha U <Sushmitha.U@dell.com> Co-authored-by: sushmitha-dell <sushmitha-dell@users.noreply.github.com> Co-authored-by: U, Sushmitha <Sushmitha_U@Dell.com> Co-authored-by: sayuri <sayuri.kamble@dell.com>
…dentials from memory Add regex validation for image tag and container registry URL parameters in all three build_images.sh scripts (telemetry, build_stream, orchestrator). Tags are restricted to [a-zA-Z0-9._-] and registries to [a-zA-Z0-9._/:@-], rejecting shell metacharacters before they reach any build command. Fixes: CVE-1324429373 (CWE-20, CWE-250) - Medium severity (CVSS 5.2) Also clear password reference from auth_payload immediately after POST and clear session cookies on logout in ome_server_inventory.py for defense-in-depth against cleartext credential retention in memory. Fixes: D-SRC-M1 (CWE-316) - Low severity (CVSS 1.8) Signed-off-by: sayuri <sayuri.kamble@dell.com>
* Fixes : rename base_os to baseos Signed-off-by: VirajPatil-1500 <Viraj.Patil@dellteam.com> * Fixes : baseos changed in test file and revert varible baseos changes Signed-off-by: VirajPatil-1500 <Viraj.Patil@dellteam.com> * Fixes : baseos rename in test Signed-off-by: VirajPatil-1500 <Viraj.Patil@dellteam.com> --------- Signed-off-by: VirajPatil-1500 <Viraj.Patil@dellteam.com>
* fix(utils) - security fix - Remove password-based SSH authentication from install_os deploy workflow (#5509) Changes: 1. Remove sshpass-based password authentication from iso_delivery/tasks/deploy_os.yml - Removed "Verify SSH access with root password" task - Removed "Warn if passwordless SSH key access failed" task - Removed "Warn if root password SSH access failed" task - Removed "Fail if both SSH key and password authentication fail" task - Simplified to single SSH key-based verification with retries 2. Fix ssh_private_key_path derivation in iso_delivery/vars/main.yml - Fixed regex_replace pattern from '\\.pub$' to '\.pub$' (YAML folded scalar escaping issue) - Added trim filter to handle trailing newlines - Added true flag to default() to handle empty strings correctly - This bug was causing ssh_private_key_path to remain as .pub file (public key instead of private key) 3. Update SSH key validation in validate_install_os_config/tasks/main.yml - Added deploy mode to SSH key validation condition - SSH key is now mandatory for all modes (build_iso, generate_ks, deploy, end-to-end) 4. Update error messages in iso_delivery/vars/main.yml - Removed ssh_password_fail_msg (no longer needed) - Renamed passwordless_ssh_fail_msg to ssh_key_auth_fail_msg - Updated os_install_success_msg to reference "SSH key authentication: verified" 5. Update documentation - iso_delivery/README.md: Added SSH Verification section explaining key-only auth - input-contract.md: Updated ssh_public_key_path description to include deploy flow - input-contract.md: Updated os_root_password description to clarify manual access only 6. Add test coverage - test_playbook.py: Added test_negative_deploy_mode_missing_ssh_key() - Tests that deploy mode fails when SSH key is missing Security improvements: - Eliminated password exposure in process lists and shell history (sshpass removal) - Enforced SSH key-based authentication as the only verification method Signed-off-by: venu <236371043+Venu-p1@users.noreply.github.com> * feat(orchestrator): update PXE mapping file examples with OS-versioned functional groups and also update default catalog as hybrid catalog (#5566) * Update pxe_mapping_file.csv Signed-off-by: Katakam Rakesh Naga Sai <125246792+Katakam-Rakesh@users.noreply.github.com> * Create pxe_mapping_file_slurm_service_k8s_hybrid.csv Signed-off-by: Katakam Rakesh Naga Sai <125246792+Katakam-Rakesh@users.noreply.github.com> * Update pxe_mapping_file.csv Signed-off-by: Katakam Rakesh Naga Sai <125246792+Katakam-Rakesh@users.noreply.github.com> * Update pxe_mapping_file.csv Signed-off-by: Katakam Rakesh Naga Sai <125246792+Katakam-Rakesh@users.noreply.github.com> * Update default catalog to hybrid RHEL 10.2/10.0 multi-version layout - Restructure catalog_rhel.json to hybrid catalog: RHEL 10.2 x86_64 (Slurm control, K8s control-plane, K8s node) + RHEL 10.0 x86_64 (Slurm login) + RHEL 10.0 aarch64 (Slurm node, login-compiler), without VAST - Split baseos_group into versioned baseos_group_10.0 and baseos_group_10.2 to support mixed OS versions - Add new os_rhel_10_0_x86_64 functional layer in catalog - Update pxe_mapping_file.csv: align functional group names as per the hybrid catalog and add new os_rhel_10_0_aarch64 entry - Update docs (omnia-env.md, samples/README.md, main/README.md) to reflect hybrid catalog structure Signed-off-by: Katakam-Rakesh <katakam.rakesh@dell.com> * Update pxe_mapping_file_slurm_service_k8s_hybrid.csv Signed-off-by: Katakam Rakesh Naga Sai <125246792+Katakam-Rakesh@users.noreply.github.com> * Update login_node from rhel_10_0 to rhel_10_2 in hybrid catalogs - Rename login_node_rhel_10_0_x86_64 to login_node_rhel_10_2_x86_64 in all 8 hybrid catalogs and the default catalog_rhel.json - Update baseos_group_10.0 to baseos_group_10.2 in the login node functional layer - Align pxe_mapping_file.csv and hybrid PXE example with the updated functional group name - Update docs (omnia-env.md, samples/README.md) and catalog description to reflect login node on RHEL 10.2 Signed-off-by: Katakam-Rakesh <katakam.rakesh@dell.com> * Update hybrid catalog descriptions to reflect login node on RHEL 10.2 - Update description field in all 8 hybrid catalogs to list login node under RHEL 10.2 instead of RHEL 10.0 Signed-off-by: Katakam-Rakesh <katakam.rakesh@dell.com> * Update login_node from rhel_10_0 to rhel_10_2 in aarch64 hybrid catalogs - Rename login_node_rhel_10_0_aarch64 to login_node_rhel_10_2_aarch64 in slurm_hybrid_10_2_10_0_aarch64 and its no_vast variant - Update baseos_group_10.0 to baseos_group_10.2 in the login node functional layer - Update catalog descriptions accordingly Signed-off-by: Katakam-Rakesh <katakam.rakesh@dell.com> --------- Signed-off-by: Katakam Rakesh Naga Sai <125246792+Katakam-Rakesh@users.noreply.github.com> Signed-off-by: Katakam-Rakesh <katakam.rakesh@dell.com> * fix(discovery): Improve compliance, add OME integration tests, and fix functional group fallback (#5565) * fix(discovery): Improve compliance, add OME integration tests, and fix functional group fallback This PR enhances the Discovery module with improved compliance, OME integration testing, and functional group handling for Omnia 2.3. ## Changes ### Source Code (src/discovery) - Add network_spec.yml JSON schema validation - Add .lock files to cleanup paths (align with orchestrator) - Add no_log: true on credential tasks - Add return type annotations to Python functions - Fix cross-domain reference in generate_discovery_report.yml - Add architecture documentation ### Test Automation (test/discovery) - Add OME integration tests (V010-V014): - Verify OME static groups match PXE mapping - Verify devices without static group assignment - Verify ADMIN_MAC matches OME NIC - Verify IB_NIC_NAME matches OME IB NIC - Verify PXE mapping CSV format for Orchestrator - Add PXE mapping validation tests (V001-V009) - Add OME domain credentials support to setup_env.sh - Fix NFT test order to prevent credential removal before execute - Add nosec comments for Bandit B108 in test code - Accept valid default groups in OME static group check ### Functional Group Handling - Enforce OS version suffix pattern: <prefix>_rhel_<major>_<minor>_<arch> - Fallback to default functional group for invalid OME groups - Support any custom prefix with required rhel version suffix ### Compliance Improvements - Improve compliance score from 93.68 to 96.84 - Add Bandit B108 nosec comments for test temp directories Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> * fix(discovery): Improve compliance, add OME integration tests, and fix functional group fallback This PR enhances the Discovery module with improved compliance, OME integration testing, and functional group handling for Omnia 2.3. - Add network_spec.yml JSON schema validation - Add .lock files to cleanup paths (align with orchestrator) - Add no_log: true on credential tasks - Add return type annotations to Python functions - Fix cross-domain reference in generate_discovery_report.yml - Add architecture documentation - Add OME integration tests (V010-V014): - Verify OME static groups match PXE mapping - Verify devices without static group assignment - Verify ADMIN_MAC matches OME NIC - Verify IB_NIC_NAME matches OME IB NIC - Verify PXE mapping CSV format for Orchestrator - Add PXE mapping validation tests (V001-V009) - Add OME domain credentials support to setup_env.sh - Fix NFT test order to prevent credential removal before execute - Add nosec comments for Bandit B108 in test code - Accept valid default groups in OME static group check - Enforce OS version suffix pattern: <prefix>_rhel_<major>_<minor>_<arch> - Fallback to default functional group for invalid OME groups - Support any custom prefix with required rhel version suffix - Improve compliance score from 93.68 to 96.84 - Add Bandit B108 nosec comments for test temp directories Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> --------- Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> Co-authored-by: Abhishek S A <abhishek.sa3@dell.com> * fix(image_build_manager): allow watcher container transition (#5570) Refresh the managed SELinux compatibility module when its source changes and install it before any Podman access so watcher-triggered builds retain system hardening while entering the container runtime domain. Signed-off-by: SOWJANYAJAGADISH123 <sowjanya.jagadish@dell.com> Co-authored-by: Rajeshkumar S <rajeshkumar.s2@dell.com> * fix(build_stream): only mark ImageGroup FAILED on validate failure (#5573) Deploy and restart stage failures are job-level failures (e.g. configuration issues, queue submission errors) and do not indicate anything wrong with the images themselves. Previously, ImageGroup status was incorrectly transitioned to FAILED on deploy or restart playbook failures. Changes: - Remove _on_deploy_failure() call and method from ResultPoller - Remove _on_restart_failure() call and method from ResultPoller - Keep _on_restart_completed() for node_results/failed_nodes persistence - Update STATUS_FLOW on_failure to None for deploy and restart - Only validate failure transitions ImageGroup to FAILED - Update existing unit tests to match new behavior Job state, stage state, resume/retry flows, and artifact persistence are all unaffected by this change. Signed-off-by: SOWJANYAJAGADISH123 <sowjanya.jagadish@dell.com> Co-authored-by: Rajeshkumar S <rajeshkumar.s2@dell.com> * fix(orchestrator): renew OpenCHAMI tokens before provision (#5571) When running provision to add a node after the cluster has been running for more than 24 hours, the playbook fails with OpenCHAMI authentication errors. This is because metadata-service and boot-service exchange a bootstrap token at startup for a refresh token family that expires after 24 hours. Once expired, the services can no longer authenticate with SMD. This workaround restarts metadata-service, boot-service, and HAProxy in the provision preamble before OpenCHAMI authentication. The restart forces the services to re-exchange their bootstrap tokens and obtain a fresh refresh token family with a new 24-hour window. The restart is also added at the end of deploy_openchami for consistency, but this may be redundant since containers already start fresh during deployment. If the team agrees, this can be removed in a follow-up. This is a bounded Omnia recovery workaround until the upstream OpenCHAMI packaging provides a durable credential lifecycle (e.g., systemd timer for periodic token refresh). Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com> Co-authored-by: Sujit Jadhav <sujit.jadhav@dell.com> * fix(orchestrator): skip framework CUDA trees in HPC tools binary scan and drop buildstream marker The source-only binary scan in _hpc_benchmarks_helpers walked every executable under the staged hpc_tools paths, so CUDA trees staged under /hpc_tools/platforms/<os>/<ver>/<arch>/cuda were reported as tool binaries. The find command now prunes the framework-owned subdirectories (_STAGING_EXCLUDED_SUBDIRS) at any depth, and the scan path is shell-quoted. Remove the buildstream marker from the OpenMPI and HPC benchmarks source-only delivery tests so they are not selected in BuildStream runs. Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com> * fix(repo_manager): catalog add/delete merge sources for multi-arch catalogs and align AI skills with code (#5579) * Fix catalog add/delete for multi-arch catalogs and align AI skills with code Remove the buildstream marker from the HPC benchmarks source-only delivery test (V206). repo_manager catalog tooling: - add merges package sources per architecture and OS version instead of replacing the entry, so updates no longer drop other platforms; an identical re-add is a no-op - repeated package keys in one input file merge instead of last-wins - delete logs the functional layers affected when an emptied group is removed - legacy 1.0 catalogs get a "run transform first" message - unit tests for the merge behaviour BuildStream AI skills: - bulk-edit: full RHEL version switch (layer names, sources, base_os, metadata) and allowed_root on writes; worked example re-verified - catalog-editing: merge semantics, every-layer additions on hybrid catalogs, delete side effect, runtime catalogs out of scope - catalog-generation: repo_manager rules schema validation misses (layer naming, single OS type, source coverage); hybrid login note - master reference: tables regenerated from current catalogs and repo config (RHEL 10.2 sources, image tags, moved template paths) - catalog-diff: exit code 2 without --schema, HTML needs jinja2, example numbers updated - remove stale cross-skill claims; align manifests and READMEs - reference-integrity test now checks every src/ path, relative refs, manifest dependencies and description drift Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com> * docs(build_stream): replace 'offline' with specific terminology in AI skills Replace ambiguous 'offline' references with more specific terminology across BuildStream AI skills to clarify when online sources are unavailable and the master reference file is used as fallback. Changes: - 'offline gaps' → 'gaps when online sources are unavailable' - 'offline fallback' → 'fallback' or 'master reference file data' - 'online/offline' → 'online source lookup and fallback' - 'operates entirely offline' → 'operates without live dependency resolution' - 'offline mode' → 'when using the master reference file' This addresses reviewer feedback that 'offline' was being introduced in multiple skills and could cause confusion. --------- Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com> Co-authored-by: Abhishek S A <abhishek.sa3@dell.com> * fix(test): improve CUDA path detection with platform-aware checks Update CUDA path detection to use dynamic platform detection with proper validation: - Remove unnecessary backslash escapes that prevented variable expansion - Use . instead of source for consistency with other commands - Check for .done_cuda marker to verify complete installation - Check for executable nvcc instead of just file existence - Update lock check to use platform-specific .nfs_lock_cuda directory - Remove hardcoded /hpc_tools/cuda path from test function output - Dynamically report the detected CUDA path in test results This ensures tests work across different RHEL versions and architectures while properly validating that CUDA installation is complete. Signed-off-by: sayuri <sayuri.kamble@dell.com> * docs(ai_skills): fix catalog skills documentation issues and remove internal ER references (#5586) * fix(orchestrator): remove buildstream and sanity markers from failed test cases Remove @pytest.mark.buildstream and @pytest.mark.sanity markers from three test cases that are currently failing, so they are not selected in BuildStream or sanity test runs: - test_slurm_reconfigure (V029) - Slurm reconfiguration test - test_dcgm_cuda_login_compiler (V507) - CUDA toolkit on login_compiler - test_apptainer_concurrent_jobs (V080) - Concurrent container jobs These tests will be re-enabled with the markers once the underlying issues are resolved. Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com> * fix(repo-manager): priority value for epel (#5588) * epel priotity setting externally Signed-off-by: Nagachandan P <Nagachandan.p@dell.com> * Update repo_manager_config.yml fixing lint issues Signed-off-by: Nagachandan P <Nagachandan.p@dell.com> --------- Signed-off-by: Nagachandan P <Nagachandan.p@dell.com> * Remove sanity markers from failing PowerVault and HPC benchmark tests (#5591) * Update test_negative.py Signed-off-by: balajikumaran.cs <balajikumaran.cs@dell.com> * Update test_contracts.py Signed-off-by: balajikumaran.cs <balajikumaran.cs@dell.com> * Fix pylint errors in test_negative.py and remove sanity markers Fixed verify_pxeboot function calls to use correct signature with TestLogger and test case from TC. Removed @pytest.mark.sanity markers from test_neg_powervault_gpt_missing_label, test_neg_powervault_duplicate_fstab, and test_hpc_benchmarks_source_only_delivery as requested in PR #5591. Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com> --------- Signed-off-by: balajikumaran.cs <balajikumaran.cs@dell.com> Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com> * Discovery Security fixes (#5596) * fix(discovery): Improve compliance, add OME integration tests, and fix functional group fallback This PR enhances the Discovery module with improved compliance, OME integration testing, and functional group handling for Omnia 2.3. ## Changes ### Source Code (src/discovery) - Add network_spec.yml JSON schema validation - Add .lock files to cleanup paths (align with orchestrator) - Add no_log: true on credential tasks - Add return type annotations to Python functions - Fix cross-domain reference in generate_discovery_report.yml - Add architecture documentation ### Test Automation (test/discovery) - Add OME integration tests (V010-V014): - Verify OME static groups match PXE mapping - Verify devices without static group assignment - Verify ADMIN_MAC matches OME NIC - Verify IB_NIC_NAME matches OME IB NIC - Verify PXE mapping CSV format for Orchestrator - Add PXE mapping validation tests (V001-V009) - Add OME domain credentials support to setup_env.sh - Fix NFT test order to prevent credential removal before execute - Add nosec comments for Bandit B108 in test code - Accept valid default groups in OME static group check ### Functional Group Handling - Enforce OS version suffix pattern: <prefix>_rhel_<major>_<minor>_<arch> - Fallback to default functional group for invalid OME groups - Support any custom prefix with required rhel version suffix ### Compliance Improvements - Improve compliance score from 93.68 to 96.84 - Add Bandit B108 nosec comments for test temp directories Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> * fix(discovery): Improve compliance, add OME integration tests, and fix functional group fallback This PR enhances the Discovery module with improved compliance, OME integration testing, and functional group handling for Omnia 2.3. - Add network_spec.yml JSON schema validation - Add .lock files to cleanup paths (align with orchestrator) - Add no_log: true on credential tasks - Add return type annotations to Python functions - Fix cross-domain reference in generate_discovery_report.yml - Add architecture documentation - Add OME integration tests (V010-V014): - Verify OME static groups match PXE mapping - Verify devices without static group assignment - Verify ADMIN_MAC matches OME NIC - Verify IB_NIC_NAME matches OME IB NIC - Verify PXE mapping CSV format for Orchestrator - Add PXE mapping validation tests (V001-V009) - Add OME domain credentials support to setup_env.sh - Fix NFT test order to prevent credential removal before execute - Add nosec comments for Bandit B108 in test code - Accept valid default groups in OME static group check - Enforce OS version suffix pattern: <prefix>_rhel_<major>_<minor>_<arch> - Fallback to default functional group for invalid OME groups - Support any custom prefix with required rhel version suffix - Improve compliance score from 93.68 to 96.84 - Add Bandit B108 nosec comments for test temp directories Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> * Fix security issues: SSRF in OME inventory and weak credential policy Issue #1153 (SSRF): - Add same-origin validation for OME navigation links - Add allow_redirects=False to prevent redirect-based token leak Issue #1146 (Weak Credentials): - Change re.match to re.fullmatch in Discovery and Utils - Raise ome_password minimum length from 1 to 8 - Replace no-op pattern with length-enforcing pattern - Remove 30 unused credential fields from Discovery Addresses security findings: - [OMN-SEC] [MEDIUM] 2720853351 (SSRF) - [OMN-SEC] [MEDIUM] 1362293893 (weak credentials) Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> * Fix insufficient IP validation on ome_ip for SSRF prevention Add link-local, reserved, multicast, and unspecified address checks to ome_ip validation in discovery_validation_flow.py. Blocks requests to cloud metadata endpoints (169.254.169.254) and other non-routable addresses while preserving private network IPs used by OME appliances. Addresses security finding [OMN-SEC] [MEDIUM] D-SRC-H1 Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> * Add OME password validation to credential collection Add validate_credentials call for ome_password in Discovery credential update flow to enforce the 8-character minimum length requirement from credential_rules.json. Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> * Remove Discovery UT tests for removed credential fields Remove tests for bmc_username, bmc_password, and provision_password from test_input_validation_schema.py since these fields were removed from credential_rules.json as part of security fix (Discovery only validates ome_username and ome_password). Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> * Move SSL warning suppression to conditional in OME client Remove global urllib3.disable_warnings at module level. Move warning suppression inside OMEClient.__init__ so it only applies when verify_ssl=False, and emit a log warning. This prevents process-wide suppression of TLS warnings for other modules in the same Ansible run. CA certificate support will be addressed in a separate Q4 BR. Addresses security finding [OMN-SEC] [MEDIUM] D-SRC-H2 Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> * Move SSL warning suppression to conditional in OME client Remove global urllib3.disable_warnings at module level. Move warning suppression inside OMEClient.__init__ so it only applies when verify_ssl=False, and emit a log warning. This prevents process-wide suppression of TLS warnings for other modules in the same Ansible run. CA certificate support will be addressed in a separate Q4 BR. Addresses security finding [OMN-SEC] [MEDIUM] D-SRC-H2 Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> * reverted the minimum length password change Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> * Fix Discovery domain security issues (SSRF, credential validation, IP validation, SSL warning) Issue #1153: SSRF in OME Inventory (CVE Finding 2720853351) - Add urljoin validation for navigation links with same-origin check - Add allow_redirects=False to prevent redirect-based token leak Issue #1146: Weak Credential Policy (CVE Finding 1362293893) - Change re.match to re.fullmatch in Discovery and Utils validate_credentials - Remove 30 unused credential fields from Discovery credential_rules.json - Password minimum length remains at 1 to match documentation Issue #1125: SSRF via ome_ip (CVE Finding D-SRC-H1) - Add validation to reject link-local, reserved, multicast, unspecified addresses Issue #1126: SSL/TLS Certificate Verification Warning (CVE Finding D-SRC-H2) - Remove global urllib3.disable_warnings - Move warning suppression to conditional (only when verify_ssl=False) - Add module.warn() for task output warning Test Fix: - Remove tests for removed credential fields Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> --------- Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> Signed-off-by: SOWJANYAJAGADISH123 <sowjanya.jagadish@dell.com> Signed-off-by: saifudheene <saifudheen.e@dell.com> Signed-off-by: Sujit Jadhav <sujit.jadhav@dell.com> Signed-off-by: Rohith Ravut <rohith.ravut@dellteam.com> Signed-off-by: Abhishek S A <abhishek.sa3@dell.com> Signed-off-by: Mithilesh Reddy <mithilesh.reddy@dell.com> Signed-off-by: sayuri <sayuri.kamble@dell.com> Signed-off-by: Rajeshkumar S <rajeshkumar.s2@dell.com> Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com> Signed-off-by: balajikumaran.cs <balajikumaran.cs@dell.com> Signed-off-by: Narthan_S <narthan.s@dell.com> Signed-off-by: snarthan <narthan.s@dell.com> Signed-off-by: esaifudheen <esaifudheen@dell.com> Signed-off-by: Nagachandan-P <Nagachandan.p@dell.com> Signed-off-by: venu <236371043+Venu-p1@users.noreply.github.com> Signed-off-by: Venu-p1 <236371043+Venu-p1@users.noreply.github.com> Signed-off-by: VirajPatil-1500 <VirajPatil-1500@users.noreply.github.com> Signed-off-by: Katakam-Rakesh <katakam.rakesh@dell.com> Signed-off-by: Nagachandan P <Nagachandan.p@dell.com> Signed-off-by: VirajPatil-1500 <Viraj.Patil@dellteam.com> Signed-off-by: Vrinda_Marwah <Vrinda.Marwah@dell.com> Signed-off-by: Sujal27 <sujal.sooryavamshi@dell.com> Signed-off-by: priti_parate <priti_parate@dell.com> Signed-off-by: priti-parate <140157516+priti-parate@users.noreply.github.com> Signed-off-by: Kratika_Patidar <Kratika.Patidar@dell.com> Signed-off-by: mithileshreddy04 <mithilesh.reddy@dell.com> Signed-off-by: Sushmitha-dell <Sushmitha.u@dell.com> Signed-off-by: SujalS27 <Sujal.Sooryavamshi@dell.com> Signed-off-by: Mithilesh Reddy <mithilesh.reddy@dell.com>" Signed-off-by: Kratika Patidar <Kratika.Patidar@dell.com> Signed-off-by: sushmitha-dell <sushmitha-dell@users.noreply.github.com> Signed-off-by: U, Sushmitha <Sushmitha_U@Dell.com> Signed-off-by: Sushmitha U <Sushmitha.U@dell.com> Signed-off-by: Kratika-P <Kratika.Patidar@dell.com> Signed-off-by: Katakam Rakesh Naga Sai <125246792+Katakam-Rakesh@users.noreply.github.com> Co-authored-by: Abhishek S A <abhishek.sa3@dell.com> Co-authored-by: Rajeshkumar S <rajeshkumar.s2@dell.com> Co-authored-by: SOWJANYAJAGADISH123 <Sowjanya.Jagadish@dell.com> Co-authored-by: saifudheene <Saifudheen.E@dell.com> Co-authored-by: Sujit Jadhav <sujit.jadhav@dell.com> Co-authored-by: Rohith Ravut <rohith.ravut@dellteam.com> Co-authored-by: Mithilesh Reddy <mithilesh.reddy@dell.com> Co-authored-by: Sayuri Kamble <68416000+SAYUK09@users.noreply.github.com> Co-authored-by: balajikumaran.cs <balajikumaran.cs@dell.com> Co-authored-by: snarthan <narthan.s@dell.com> Co-authored-by: esaifudheen <esaifudheen@dell.com> Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: Nagachandan P <Nagachandan.p@dell.com> Co-authored-by: Venu-p1 <236371043+Venu-p1@users.noreply.github.com> Co-authored-by: VirajPatil-1500 <Viraj.Patil@dellteam.com> Co-authored-by: VirajPatil-1500 <VirajPatil-1500@users.noreply.github.com> Co-authored-by: Katakam-Rakesh <katakam.rakesh@dell.com> Co-authored-by: Katakam Rakesh Naga Sai <125246792+Katakam-Rakesh@users.noreply.github.com> Co-authored-by: Vrinda Marwah <vrinda.marwah@dell.com> Co-authored-by: priti-parate <140157516+priti-parate@users.noreply.github.com> Co-authored-by: Kratika Patidar <Kratika.Patidar@dell.com> Co-authored-by: Sushmitha U <Sushmitha.U@dell.com> Co-authored-by: sushmitha-dell <sushmitha-dell@users.noreply.github.com> Co-authored-by: U, Sushmitha <Sushmitha_U@Dell.com> Co-authored-by: sayuri <sayuri.kamble@dell.com> * merge conflict for catalog Signed-off-by: priti_parate <priti_parate@dell.com> --------- Signed-off-by: venu <236371043+Venu-p1@users.noreply.github.com> Signed-off-by: Katakam Rakesh Naga Sai <125246792+Katakam-Rakesh@users.noreply.github.com> Signed-off-by: Katakam-Rakesh <katakam.rakesh@dell.com> Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> Signed-off-by: SOWJANYAJAGADISH123 <sowjanya.jagadish@dell.com> Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com> Signed-off-by: sayuri <sayuri.kamble@dell.com> Signed-off-by: Nagachandan P <Nagachandan.p@dell.com> Signed-off-by: balajikumaran.cs <balajikumaran.cs@dell.com> Signed-off-by: saifudheene <saifudheen.e@dell.com> Signed-off-by: Sujit Jadhav <sujit.jadhav@dell.com> Signed-off-by: Rohith Ravut <rohith.ravut@dellteam.com> Signed-off-by: Abhishek S A <abhishek.sa3@dell.com> Signed-off-by: Mithilesh Reddy <mithilesh.reddy@dell.com> Signed-off-by: Rajeshkumar S <rajeshkumar.s2@dell.com> Signed-off-by: Narthan_S <narthan.s@dell.com> Signed-off-by: snarthan <narthan.s@dell.com> Signed-off-by: esaifudheen <esaifudheen@dell.com> Signed-off-by: Nagachandan-P <Nagachandan.p@dell.com> Signed-off-by: Venu-p1 <236371043+Venu-p1@users.noreply.github.com> Signed-off-by: VirajPatil-1500 <VirajPatil-1500@users.noreply.github.com> Signed-off-by: VirajPatil-1500 <Viraj.Patil@dellteam.com> Signed-off-by: Vrinda_Marwah <Vrinda.Marwah@dell.com> Signed-off-by: Sujal27 <sujal.sooryavamshi@dell.com> Signed-off-by: priti_parate <priti_parate@dell.com> Signed-off-by: priti-parate <140157516+priti-parate@users.noreply.github.com> Signed-off-by: Kratika_Patidar <Kratika.Patidar@dell.com> Signed-off-by: mithileshreddy04 <mithilesh.reddy@dell.com> Signed-off-by: Sushmitha-dell <Sushmitha.u@dell.com> Signed-off-by: SujalS27 <Sujal.Sooryavamshi@dell.com> Signed-off-by: Mithilesh Reddy <mithilesh.reddy@dell.com>" Signed-off-by: Kratika Patidar <Kratika.Patidar@dell.com> Signed-off-by: sushmitha-dell <sushmitha-dell@users.noreply.github.com> Signed-off-by: U, Sushmitha <Sushmitha_U@Dell.com> Signed-off-by: Sushmitha U <Sushmitha.U@dell.com> Signed-off-by: Kratika-P <Kratika.Patidar@dell.com> Co-authored-by: Abhishek S A <abhishek.sa3@dell.com> Co-authored-by: Venu-p1 <236371043+Venu-p1@users.noreply.github.com> Co-authored-by: Katakam Rakesh Naga Sai <125246792+Katakam-Rakesh@users.noreply.github.com> Co-authored-by: SujalS27 <Sujal.Sooryavamshi@dell.com> Co-authored-by: SOWJANYAJAGADISH123 <Sowjanya.Jagadish@dell.com> Co-authored-by: Rajeshkumar S <rajeshkumar.s2@dell.com> Co-authored-by: balajikumaran.cs <balajikumaran.cs@dell.com> Co-authored-by: Sujit Jadhav <sujit.jadhav@dell.com> Co-authored-by: sayuri <sayuri.kamble@dell.com> Co-authored-by: Nagachandan P <Nagachandan.p@dell.com> Co-authored-by: saifudheene <Saifudheen.E@dell.com> Co-authored-by: Rohith Ravut <rohith.ravut@dellteam.com> Co-authored-by: Mithilesh Reddy <mithilesh.reddy@dell.com> Co-authored-by: Sayuri Kamble <68416000+SAYUK09@users.noreply.github.com> Co-authored-by: snarthan <narthan.s@dell.com> Co-authored-by: esaifudheen <esaifudheen@dell.com> Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: VirajPatil-1500 <Viraj.Patil@dellteam.com> Co-authored-by: VirajPatil-1500 <VirajPatil-1500@users.noreply.github.com> Co-authored-by: Katakam-Rakesh <katakam.rakesh@dell.com> Co-authored-by: Vrinda Marwah <vrinda.marwah@dell.com> Co-authored-by: Kratika Patidar <Kratika.Patidar@dell.com> Co-authored-by: Sushmitha U <Sushmitha.U@dell.com> Co-authored-by: sushmitha-dell <sushmitha-dell@users.noreply.github.com> Co-authored-by: U, Sushmitha <Sushmitha_U@Dell.com>
- Wrap session.post() in try...finally so credential cleanup executes even on request exceptions (ome_server_inventory.py) - Enforce OCI 128-char tag length limit in TAG_PATTERN - Remove @ from REGISTRY_PATTERN (invalid in registry host/path) - Remove CVE number from code comments in all build scripts Signed-off-by: sayuri <sayuri.kamble@dell.com>
…idation, prevent CSV formula injection, and sanitize error messages to prevent information leakage in logs. (#5598) * Fix Discovery security issues (path traversal, CSV injection, exception exposure) Issue #1127: Path Traversal in validate_credentials.py - Add os.path.realpath() to resolve path traversal sequences (../) - Add FileNotFoundError to exception handler Issue #1129: CSV Formula Injection in generate_pxe_mapping.py - Add _sanitize_csv_value() function to prefix dangerous characters with single quote - Apply sanitizer to all string values in CSV rows to prevent Excel formula injection Issue #1132: Generic Exception in ome_server_inventory.py - Import specific exception types from requests.exceptions - Replace generic Exception handler with specific handlers - Provide sanitized, actionable error messages without exposing internal details Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> * Add UT test for CSV sanitizer (DISC_UT_049) Add single unit test for _sanitize_csv_value() function in test_input_validation_schema.py to verify: - Dangerous characters (=, +, -, @, \t, \r) are prefixed with single quote - Normal values (hostnames, IPs, MAC addresses) remain unchanged - Empty inputs are handled correctly Test registered as DISC_UT_049 in ut_test_case_vars.py. Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> * Fix PATH hijack risk in validate_system_environment.py Use absolute paths for hostname and ip commands to prevent PATH hijacking. - hostname: /usr/bin/hostname (RHEL 10/10.2 standard path) - ip: /usr/sbin/ip (RHEL 10/10.2 standard path) Addresses issue #1131 (D-SRC-M4) by eliminating PATH resolution for hardcoded read-only diagnostic commands. Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> --------- Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com>
…he poisoning (CWE-494/CWE-349) in telemetry (#5597) * Fix Unquoted Jinja variables and Unsafe Jinja functions Convert ~260 ansible.builtin.command tasks from folded-scalar string form to safe argv list form to eliminate shell parsing via execve(). Add | quote to ~20 shell tasks with Jinja interpolation. Refactor 2 Python inline scripts to use environment blocks. Audit all 28 lookup() instances; all are benign env/file/password lookups with strict defaults (no pipe/template). Signed-off-by: Mithilesh Reddy <mithilesh.reddy@dell.com> * Telemetry jinja template fixes * fix TLS issue * Update helm_install_karavi.yml Signed-off-by: Mithilesh Reddy <mithilesh.reddy@dell.com> * Revert unnecessary argv changes Signed-off-by: Mithilesh Reddy <mithilesh.reddy@dell.com> * fix(security): remediate clone-cache poisoning in telemetry build_images.sh * correct shell interpolation and error handling --------- Signed-off-by: Mithilesh Reddy <mithilesh.reddy@dell.com>
…nd-cleartext-memory security(build,discovery): Validate tag/registry inputs in build scripts and clear credentials from memory
…ator errors completely (#5602) * Fixes :Input validator error paths: broad exception handlers truncate error reporting and can leave vaulted files decrypted Signed-off-by: VirajPatil-1500 <Viraj.Patil@dellteam.com> * Fixes : handle subprocess timeout explicitly in run_subprocess (review feedback) Signed-off-by: VirajPatil-1500 <Viraj.Patil@dellteam.com> --------- Signed-off-by: VirajPatil-1500 <Viraj.Patil@dellteam.com>
Signed-off-by: Nagachandan-P <Nagachandan.p@dell.com>
…5582) Fixes the repo manager portion of issue #1148 (CWE-214: Secrets passed on process argv). Removed 4 dead auth templates from pulp_commands.py that used --password on argv: - create_container_remote_auth - update_container_remote_auth - create_container_remote_for_digest_auth - update_remote_for_digest_auth Removed username and password parameters from build_container_remote_command(). All production callers now use reconcile_authenticated_container_remote() which sends credentials via Pulp REST API (HTTPS body), never on process argv. Updated unit test to verify no password appears in generated command. Verified: Harbor registry remote created successfully with credentials stored in Pulp, but no password appeared in /proc/*/cmdline or log files. Signed-off-by: Katakam-Rakesh <katakam.rakesh@dell.com>
* fix(callback): redact sensitive keys from result dict Add _SENSITIVE_RESULT_KEYS and _redact_sensitive_keys() method to filter out invocation, ansible_facts, ansible_module_name, ansible_module_args from result dict before serialization in _format_result_msg. Prevents module_args exposure in logs and stdout when tasks fail with multi-line error messages. Applied to 6 domains: Discovery, Orchestrator, Utils, Image Build Manager, BuildStream, Telemetry. Repo Manager already had this fix. Addresses issue #1155 (CVE Finding 3437343473). Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> * fix(callback): add type checking and module_args redaction - Add 'module_args' to _SENSITIVE_RESULT_KEYS to handle modules that store raw inputs under top-level module_args without invocation wrapper - Add type checking in _redact_sensitive_keys to handle non-dict inputs (None, string, etc.) to prevent AttributeError crashes in rare module crashes or transport timeouts - Updated all 6 domains: discovery, orchestrator, utils, image_build_manager, build_stream, telemetry Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> --------- Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com>
…5589) Disable automatic NFS share data cleanup for Kubernetes and Slurm components to prevent failures when clusters are running. Files locked by active pods, containers, or daemons cause cleanup to fail. Users must now manually clean up NFS data while the playbook handles storage unmount and fstab cleanup. Changes: - Set cleanup_nfs_server: false in K8s and Slurm component specs - Update documentation with manual cleanup instructions for both services - Add warning messages at playbook end when K8s or Slurm are selected - Update cleanup descriptions to reflect manual NFS data cleanup Signed-off-by: sayuri <sayuri.kamble@dell.com>
Signed-off-by: Abhishek S A <abhishek.sa3@dell.com>
…5796) Telemetry cleanup skipped every removal when kube_vip could not be resolved, leaving the validator log and generated output files behind. cleanup --all then aborted in its preflight on validate_telemetry_input.log under telemetry/log. Move the log, output, and stray iDRAC report removal outside the kube_vip gate. These are local files with no cluster dependency. Keep input removal inside the gate so staged configuration is not deleted while sink volumes may still remain in the cluster. The input directory is already allowed by the cleanup preflight. Revert the preflight exclusion for validate_telemetry_input.log added in the previous commit, since the log is now removed by cleanup. Output removal deletes telemetry_status.yml when Delete_sinks_volume is set. output-contract.md still says this file is preserved and needs an update to match. Signed-off-by: Mithilesh Reddy <mithilesh.reddy@dell.com> Co-authored-by: priti-parate <140157516+priti-parate@users.noreply.github.com>
…nary entry for hybrid catalog support (#5800) Add os_version to the compute image cache hash and dictionary entry to support hybrid catalogs where functional groups may span multiple OS versions (e.g., RHEL 10.0 and 10.2). - Gap 1: Include os_version in cache_input for both x86_64 and aarch64 compute image builds. This ensures different OS versions produce different hashes even when packages and repo configs are identical, preventing cache collisions in hybrid catalogs. - Gap 3: Store os_version in the dictionary upsert entry. This allows the S3 artifact discovery loop to look in the correct release subfolder (e.g., 10.2/) for cached groups instead of always falling back to rhel_tag. The dictionary key {arch}:{fg_name}:{hash} naturally disambiguates entries because the hash now includes os_version (Gap 2 is resolved by Gap 1). Note: This change invalidates existing caches (one-time rebuild) but is necessary to fix the hybrid catalog collision bug. Signed-off-by: SOWJANYAJAGADISH123 <sowjanya.jagadish@dell.com>
…V016–V021 (#5797) * Update common_vars.py Added the same input logic as other domains Signed-off-by: SujalS27 <Sujal.Sooryavamshi@dell.com> * fix(discovery): exclude InfiniBand NICs from ADMIN_MAC test validation The test function get_ome_device_details filtered only iDRAC NICs when selecting the first active NIC MAC, while the production code in ome_server_inventory.py filters both iDRAC and InfiniBand NICs. On GPU/IB nodes where the InfiniBand adapter (e.g. NVIDIA ConnectX-7) appears before the Ethernet NIC in the OME inventory, the test picked the InfiniBand MAC instead of the Ethernet management MAC, causing DISCOVERY_FVT_EXECUTE_V012 (test_admin_mac_validation) to fail. Align the test NIC filter with production by also skipping NICs whose NicId contains "INFINIBAND". Fixes: #1346 Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> * test(discovery): fix V008 IB_IP fallback and add IB validation tests V016-V021 - Fix V008 check_ip_correlation: remove stale IB_IP column fallback (src now produces IB_IPV4), only check IB_IPV4 correlation when IB_NIC_NAME is present on the row - Update _get_network_spec_subnets to also return ipv6_subnet - Add V016 test_ib_conditional_population: verify IB columns populated consistently with IB_NIC_NAME presence and subnet configuration - Add V017 test_ib_ipv4_format: validate every non-empty IB_IPV4 is a valid IPv4 address - Add V018 test_ib_ipv6_format: validate every non-empty IB_IPV6 is a valid IPv6 address - Add V019 test_ib_ip_uniqueness: no duplicate IB IPs, no collision with ADMIN_IP/BMC_IP; skips when no IB subnet configured - Add V020 test_pxe_mapping_all_columns: verify all 12 output contract columns present in PXE mapping CSV header - Add V021 test_bmc_mac_populated: verify every row has non-empty BMC_MAC Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> * test(discovery): add detailed counts to IB validation test output - V016: show exact breakdown (N with IB_NIC_NAME, M without) - V017: show populated vs empty IB_IPV4 counts - V018: show populated vs empty IB_IPV6 counts - V020: list all 12 column names in detail output Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> * test(discovery): skip IB tests V016-V018 when no IB subnet configured - V016: skip when neither ib_subnet nor ipv6_subnet is configured - V017: skip when ib_subnet (IPv4) is not configured - V018: skip when ipv6_subnet is not configured - Prevents misleading "0 populated, 16 empty" results when IB is not configured in network_spec.yml Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> * test(discovery): use tl.skipped() for IB tests when no subnet configured Fix V016-V019 to call tl.skipped() instead of tl.passed() when IB subnet is not configured, so test output correctly shows SKIPPED instead of PASS. Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> --------- Signed-off-by: SujalS27 <Sujal.Sooryavamshi@dell.com> Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> Co-authored-by: Abhishek S A <abhishek.sa3@dell.com> Co-authored-by: snarthan <narthan.s@dell.com> Co-authored-by: balajikumaran.cs <balajikumaran.cs@dell.com>
Slurm and Kubernetes cleanup does not delete data on the NFS server by default. The README stated that cleanup removes shared data, which contradicted the runtime behavior and the cleanup_nfs_server default of false. Document that the server-side data is kept unless cleanup_nfs_server is true, that the NFS share must otherwise be cleaned up manually, and align the tag and extra-variable tables with that behavior. Documentation only. Signed-off-by: esaifudheen <esaifudheen@dell.com>
…avior docs(orchestrator): clarify NFS data cleanup behavior in cleanup README
… remove them after each case (#5802) * fix(telemetry): clean local artifacts when kube_vip is unavailable Telemetry cleanup skipped every removal when kube_vip could not be resolved, leaving the validator log and generated output files behind. cleanup --all then aborted in its preflight on validate_telemetry_input.log under telemetry/log. Move the log, output, and stray iDRAC report removal outside the kube_vip gate. These are local files with no cluster dependency. Keep input removal inside the gate so staged configuration is not deleted while sink volumes may still remain in the cluster. The input directory is already allowed by the cleanup preflight. Revert the preflight exclusion for validate_telemetry_input.log added in the previous commit, since the log is now removed by cleanup. Output removal deletes telemetry_status.yml when Delete_sinks_volume is set. output-contract.md still says this file is preserved and needs an update to match. Signed-off-by: Mithilesh Reddy <mithilesh.reddy@dell.com> * test(orchestrator): redirect IPoIB validation logs and remove them after each case The IPoIB input FVT validates each case against a scratch project named omnia_fvt_ipoib_<hex>. The validator wrote its log to <OMNIA_DATA_PATH>/orchestrator/log/<scratch>/, and the teardown removed only the scratch input and output directories, so every run left a log directory behind that blocked omnia cleanup --all. The validate_orchestrator_input role now takes an optional orchestrator_validation_log_dir variable for the validator's log directory. When it is not set, the default path is used, so deploy, precheck, and validate runs are unchanged. The FVT passes its scratch log directory under report_path (<report_path>/ipoib_inputs/logs/<scratch>) through that variable. The teardown removes that directory with the scratch input and output. The test does not read the log, so removing it does not change any result. Signed-off-by: Mithilesh Reddy <mithilesh.reddy@dell.com> * test(orchestrator): fold IPoIB scratch validation logs into the active project log The IPoIB input FVT validates each case against a scratch project named omnia_fvt_ipoib_<hex>. The validator writes its log under orchestrator/log/<scratch>/, and teardown removed only the scratch input and output, so every case left a new log directory behind that blocked omnia cleanup --all. After each validation, copy that scratch log into the active project's single validation log, orchestrator/log/<project>/orchestrator_validation_<project>.log. The first case of a run rewrites the file and later cases append to it, each with a header naming its scratch project. Teardown removes the scratch input, output, and log directories. The aggregate file is kept, and the next run rewrites it. No product code changes: the validator and role are unchanged. Signed-off-by: Mithilesh Reddy <mithilesh.reddy@dell.com> --------- Signed-off-by: Mithilesh Reddy <mithilesh.reddy@dell.com> Co-authored-by: priti-parate <140157516+priti-parate@users.noreply.github.com> Co-authored-by: Abhishek S A <abhishek.sa3@dell.com>
…cache to NFS, fix firewall test podman zone (#5810) * Update common_vars.py Added the same input logic as other domains Signed-off-by: SujalS27 <Sujal.Sooryavamshi@dell.com> * fix(discovery): exclude InfiniBand NICs from ADMIN_MAC test validation The test function get_ome_device_details filtered only iDRAC NICs when selecting the first active NIC MAC, while the production code in ome_server_inventory.py filters both iDRAC and InfiniBand NICs. On GPU/IB nodes where the InfiniBand adapter (e.g. NVIDIA ConnectX-7) appears before the Ethernet NIC in the OME inventory, the test picked the InfiniBand MAC instead of the Ethernet management MAC, causing DISCOVERY_FVT_EXECUTE_V012 (test_admin_mac_validation) to fail. Align the test NIC filter with production by also skipping NICs whose NicId contains "INFINIBAND". Fixes: #1346 Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> * test(discovery): fix V008 IB_IP fallback and add IB validation tests V016-V021 - Fix V008 check_ip_correlation: remove stale IB_IP column fallback (src now produces IB_IPV4), only check IB_IPV4 correlation when IB_NIC_NAME is present on the row - Update _get_network_spec_subnets to also return ipv6_subnet - Add V016 test_ib_conditional_population: verify IB columns populated consistently with IB_NIC_NAME presence and subnet configuration - Add V017 test_ib_ipv4_format: validate every non-empty IB_IPV4 is a valid IPv4 address - Add V018 test_ib_ipv6_format: validate every non-empty IB_IPV6 is a valid IPv6 address - Add V019 test_ib_ip_uniqueness: no duplicate IB IPs, no collision with ADMIN_IP/BMC_IP; skips when no IB subnet configured - Add V020 test_pxe_mapping_all_columns: verify all 12 output contract columns present in PXE mapping CSV header - Add V021 test_bmc_mac_populated: verify every row has non-empty BMC_MAC Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> * test(discovery): add detailed counts to IB validation test output - V016: show exact breakdown (N with IB_NIC_NAME, M without) - V017: show populated vs empty IB_IPV4 counts - V018: show populated vs empty IB_IPV6 counts - V020: list all 12 column names in detail output Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> * test(discovery): skip IB tests V016-V018 when no IB subnet configured - V016: skip when neither ib_subnet nor ipv6_subnet is configured - V017: skip when ib_subnet (IPv4) is not configured - V018: skip when ipv6_subnet is not configured - Prevents misleading "0 populated, 16 empty" results when IB is not configured in network_spec.yml Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> * test(discovery): use tl.skipped() for IB tests when no subnet configured Fix V016-V019 to call tl.skipped() instead of tl.passed() when IB subnet is not configured, so test output correctly shows SKIPPED instead of PASS. Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> * fix: PowerVault mount options, CUDA dnf cache redirect, firewall test podman zone - setup_iscsi_storage.sh.j2: pass MOUNT_OPTS to mount command so fstab options (noatime, _netdev) are applied on first mount instead of using kernel defaults (relatime) - install_cuda_toolkit.sh.j2: redirect dnf cache to NFS via --setopt=cachedir to avoid filling the LiveOS rootfs overlay (~4 GB savings), clean up cache on success and failure, and call setup_cuda_env in the already-installed-on-VAST code path - prepare_vars.py / network_prepare_func.py: move 5432/tcp and 27779/tcp from host-facing FIREWALL_PORTS to new FIREWALL_PODMAN_ZONE_PORTS checked against the podman zone; drop stale 27778/tcp Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> --------- Signed-off-by: SujalS27 <Sujal.Sooryavamshi@dell.com> Signed-off-by: SujalS27 <sujal.sooryavamshi@dell.com> Co-authored-by: Abhishek S A <abhishek.sa3@dell.com> Co-authored-by: snarthan <narthan.s@dell.com> Co-authored-by: balajikumaran.cs <balajikumaran.cs@dell.com>
…ng on down nodes (#5807) V048/V050/V052 (_check_slurm_role_ldap_jobs) submitted Slurm jobs with sbatch --wait --nodelist targeting each mapped compute. When a target compute node was down, Slurm kept the job PENDING (ReqNodeNotAvail) and sbatch --wait blocked indefinitely, hanging the FVT run for hours. Changes: - Check mapped compute readiness (sinfo) before the submission loop. Nodes that are down, drained, unknown or missing are reported as failures and skipped without submitting. - Drop --wait from the LDAP job submission. The job ID is recorded immediately from sbatch --parsable, and the existing bounded _wait_job_state (180 s) polls for completion. - Add timeout 30 to the scancel cleanup command so a hung scheduler cannot block workspace teardown. - Move mapped_compute_readiness to _workload_helpers so both slurm_jobs and slurm_auth modules can import it without cross-importing private functions. Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com> Co-authored-by: snarthan <narthan.s@dell.com>
|
|
||
|
|
||
| def _emit(text: str) -> None: | ||
| print(f"{datetime.now(timezone.utc):%H:%M:%SZ} {text}", flush=True) |
There was a problem hiding this comment.
@Rajeshkumar-s2 @balajikumaran-c-s Please have a look on this
There was a problem hiding this comment.
False positive, no change.
_emit only prints status text. The "secret" CodeQL traces into it is settings["secret_path"], the OpenBao secret path (for example secret/data/automation), printed in the "reading from OpenBao ()" line in setup_func._creds_openbao. It is flagged because the key name contains "secret", not because it holds one. Credential values are never printed: openbao_func.read_domains emits only the field names and counts, and the JSON goes to the suite's stdin.
I did not rename the public secret_path config key to satisfy the heuristic. A maintainer can dismiss alert 76 as a false positive.
| log(message, "FAIL") | ||
| pytest.exit(message, returncode=1) | ||
|
|
||
| _exit_on_failure(sync_project_to_remote(target), "Project sync failed") |
There was a problem hiding this comment.
@Rajeshkumar-s2 @balajikumaran-c-s Please have a look on this
There was a problem hiding this comment.
False positive, no change.
test/build_stream/conftest.py imports sync_project_to_remote from its own library/functions/host_func.py (test/build_stream/library/functions/host_func.py:186), which is sync_project_to_remote(_host) and takes one argument. The bot resolved the call to the repo_manager function (test/repo_manager/library/functions/host_func.py:136), which takes none. The suggested fix sync_project_to_remote() would raise a TypeError here.
…es.sh Generated with [Devin](https://devin.ai) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Secure Code ReviewAdvisory only: this check never blocks the PR. It scans the lines added by this PR against secure-coding rules derived from recent security fixes in this repository. Result: 12 passed, 5 failed (616 file(s) scanned) Checklist
Failed checks - detailsSEC-05 Ansible shell/command variables are quoted - 3 finding(s)Guideline: Use argv lists or '| quote' for every templated value in shell:/command: lines; prefer ansible.builtin.command with argv.
SEC-06 No unsafe Jinja lookups - 3 finding(s)Guideline: Avoid lookup('pipe'|'env'), eval-style filters and template of untrusted input; load user YAML as literal data (no Jinja evaluation).
SEC-14 Secure temporary files - 6 finding(s)Guideline: Use tempfile.mkstemp/NamedTemporaryFile or 'mktemp' (tmpfs for secrets); no predictable /tmp paths or tempfile.mktemp.
SEC-15 Path inputs validated against traversal - 6 finding(s)Guideline: Resolve user-supplied paths/names (realpath + prefix check, allowlist regex) and reject '..'; do not join raw input into paths.
SEC-18 Python dependencies pinned to exact versions - 1 finding(s)Guideline: Use 'pkg==x.y.z' in requirements and pip install commands.
Manual review focus (based on changed files)
Findings are heuristic; reviewers should confirm each one. Suppress a justified finding with a |
…737 test addresses (#5816) * test(telemetry): use RFC 5737 documentation addresses in iDRAC examples and tests The PR policy blocks 100.x.x.x IPv4 addresses in newly added lines. Three example and test addresses in the telemetry tests used that range: test_config.yml (idrac_nft_bmc_ip example) and test_idrac_bmc_inventory.py (BMC IPs used as test keys). Replace them with 192.0.2.0/24 documentation addresses (RFC 5737). The test logic is unchanged: the addresses are only keys, and the function under test does not check address ranges. Signed-off-by: Mithilesh Reddy <mithilesh.reddy@dell.com> * fix(telemetry): derive pulp OS version from service K8s groups only The telemetry role derived pulp_os_version from the first orchestrator inventory group whose name contains a RHEL version. In hybrid clusters that group can belong to another domain (for example a login or slurm node on RHEL 10.0), so the version resolved to 10.0 while the telemetry packages are synced for 10.2. Strimzi and Victoria Metrics downloads then failed with HTTP 404. Telemetry runs only on the service K8s cluster, so consider only service_kube_* groups when deriving the version. Signed-off-by: Mithilesh Reddy <mithilesh.reddy@dell.com> * docs(orchestrator): state that Slurm and K8s cleanup does not delete NFS server data The cleanup README described server-side NFS deletion as a setting users could enable, and said it was controlled by cleanup_nfs_server. That flag is internal and fixed to false, and the runtime banners already report that NFS share data was not automatically cleaned up and must be removed manually. Align the README with that behaviour: cleanup unmounts the component storage and removes its fstab entries, and NFS server data is removed manually. Remove the instructions for enabling server-side deletion and the descriptions of its behaviour, and drop the "By default" wording that implied a setting. Signed-off-by: Mithilesh Reddy <mithilesh.reddy@dell.com> --------- Signed-off-by: Mithilesh Reddy <mithilesh.reddy@dell.com>
* update secure code review guidline Signed-off-by: priti_parate <priti_parate@dell.com> * update secure code review guideline Signed-off-by: priti_parate <priti_parate@dell.com> * remove duplicate crendential no-log true check Signed-off-by: priti_parate <priti_parate@dell.com> --------- Signed-off-by: priti_parate <priti_parate@dell.com> Co-authored-by: Abhishek S A <abhishek.sa3@dell.com>
…SI inputs, per-cluster inputs file (#5814) * feat(system_test): SVT discovery cleanup, PowerVault and PowerScale CSI inputs, per-cluster inputs file - Cleanup: add the discovery suite (after the orchestrator, only when discovery holds deployed state) to the SVT cleanup steps, settings, report suites and test requirements. - PowerVault: user_inputs.yml gives only ip (a list or comma separated), iscsi_initiator and volume_id; the generator holds the defaults and builds powervault_config in storage_config.yml. Without an ip there is no PowerVault (ignored, an earlier block is removed); with an ip the other two are required. - PowerScale CSI: new setup step "files" (./setup_env.sh --stage-files, last step of a run, and run by the pipeline after the credentials) copies secret.yaml and values.yaml to the paths of omnia_config.yml only when enable_powerscale_csi is true; both must exist, have content and be valid YAML. The paths may use $NAME of omnia.env and default to the orchestrator project input folder. Header-only secret.yaml and values.yaml ship next to user_inputs.yml and are seeded (never replaced) in each cluster folder of the GitLab project. - Pipeline: each cluster picks its inputs file with <CLUSTER>_SVT_USER_INPUTS (svt.user_inputs, default user_inputs.yml). - Fix: a rerun with an empty ib_addr_mode no longer fails once the key was removed. Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com> * feat(system_test): seed the PowerScale CSI files from the product's reference examples - --create adds secret.yaml and values.yaml to each cluster folder of the GitLab project by copying src/orchestrator/examples/powerscale_reference_files/CSI_driver/ (only when missing, never replaced, not by --update-inputs); the user edits them for the array there. - The "files" step fails, naming the file and what to edit, when a staged file is still identical to its reference example: an unedited example would only fail later, at deployment. - Drop the header-only secret.yaml and values.yaml from test/system_test: the examples are the single source, so they cannot drift. Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com> * test: pin the SVT requirements and use private temp paths in two build_stream tests - system_test/requirements.txt: exact versions (pytest 9.2.0, pytest-order 1.5.0, PyYAML 6.0.3, ruamel.yaml 0.19.1), the ones the other suites pin and the SVT .venv already runs. - build_stream nft: the parse_catalog conftest and the invalid-config artifact store test no longer write fixed /tmp paths into their config; they use pytest's tmp_path / the test's temporary directory. - system_test dataset_edit.py: mark the CSV writer as not spreadsheet output (read by the orchestrator). Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com> * test(system_test): name the OIM login variable suffix without 'password' setup_svt_project.py printed the CI variable NAME (for example MULTISUBNET_SVT_TARGET_PASS), never its value, but the name came from a constant called PASSWORD_VAR, which CodeQL's name-based rule reads as a password flowing into print(). Rename the constant to OIM_LOGIN_VAR; the messages, the prompt and the API calls are unchanged. Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com> --------- Signed-off-by: balajikumaran-c-s <balajikumaran.cs@dell.com>
feat(release): sync staging to main for Omnia 2.3 RC3
Description of the Solution
Promotes the accumulated
stagingwork tomainas the Omnia 2.3 RC3 release candidate (100 commits, 1,352 files, +79,139 / -41,150). The sync bundles security hardening and pentest/CodeGuardian remediations across all domains, the RHEL 10.2 (pub/rhel10.2) content, Build Stream pre-/post-PXE separation and path relocation under/opt/omnia, new Orchestrator capabilities (ER-ORCH-005 IPv6 InfiniBand,k8s_bootstrap_token_ttl), and expanded UT/FVT/NFT test automation.Related Issue
Changes
Security hardening (all domains)
main; secure temp files for munge key fetch in telemetry.project_namepath-traversal/length/log-injection checks, Redfish discovery hardware attributes, tag/registry validation in build scripts, fail-closed on unknown tags/unvalidated inputs, line/column info for vault YAML errors.IB_NIC_NAMEcommand-injection remediation (allowlist regex plus multi-line join command check).Build Stream (
src/build_stream/)/opt/omnia; store cadence schedule underbuild_stream_rootand preserve it across restarts.Orchestrator (
src/orchestrator/)k8s_bootstrap_token_ttl; improvedomnia_config.ymldocumentation.RealMemoryin MB; disable automatic NFS data cleanup for K8s/Slurm; security fixes part 1.Telemetry (
src/telemetry/)container_registryfromrepo_urlin offline mode.Repo Manager, Discovery, Utils, Image Build Manager
base_osrenamed tobaseos.Tests and CI (
test/,.github/)system_testand pipeline tests.user_registrydirs replaced with positive tests; Utils test runner aligned.Files Changed
src/build_stream/**(~230 files)src/orchestrator/**(~150 files)src/telemetry/**(~95 files)src/repo_manager/**(~55 files)src/image_build_manager/**(~65 files)src/discovery/**(~30 files)src/utils/**(~50 files)src/main/**(~38 files)omnia.sh,omnia-cli, docs, samplessrc/*/galaxy.yml,CHANGELOG.md,requirements.txt,README.mdtest/**(~600 files).github/scripts/check_credential_no_log.py,.github/scripts/secure_code_review.py(+ tests).github/workflows/credential-no-log.yml,.github/workflows/secure-code-review.yml.github/workflows/README.md,.github/pull_request_template.md.config/requirements.yml,CONTRIBUTING.md,docs/galaxy-testing-guide.md(Grouped by directory because the PR touches 1,352 files; see the Files changed tab for the full list.)
Testing
Backward Compatibility
base_osrenamed tobaseos; Build Stream directories moved under/opt/omnia; Build Stream cadence schedule location changed; automatic NFS data cleanup disabled for K8s/Slurm; Slurm defaultRealMemoryis now in MB; stricter input validation now fails closed on unknown tags/unvalidated inputs.Suggested Reviewers
@VenkateswaraVatam@Rajeshkumar-s2@sujit-jadhav@Venu-p1@priti-parate@RvishankarOMnia,@saifudheene