ROX-29577: VM4VM e2e tests by vikin91 · Pull Request #19660 · stackrox/stackrox

vikin91 · 2026-03-27T17:48:59Z

Description

User request (feature): "Design the end-to-end tests for the VM scanning feature."

Add a Go test_e2e VM-scanning suite that provisions RHEL 9 and RHEL 10 KubeVirt VMs, installs and runs roxagent, validates Central scan visibility, and captures progressive diagnostics for flaky-path triage.
Wire a dedicated OpenShift CI job and e2e scripts with CNV/VSOCK prerequisite checks and activation/repo2cpe guardrails.

AI-assisted: this commit was partially generated with AI assistance.
Made-with: Cursor

User-facing documentation

CHANGELOG.md is updated OR update is not needed
documentation PR is created and is linked above OR is not needed

Testing and quality

the change is production ready: the change is GA, or otherwise the functionality is gated by a feature flag
CI results are inspected

Automated testing

How I validated my change

Last status when running locally (the issue was that VM_SCAN_REQUIRE_ACTIVATION was set to false instead of true):

$ ./test.sh

# github.com/stackrox/rox/tests/vmhelpers.test
ld: warning: '/private/var/folders/hs/_m7s0nc54v74zxz1kzfdntdm0000gn/T/go-link-3064997500/000013.o' has malformed LC_DYSYMTAB, expected 98 undefined symbols to start at index 1626, found 95 undefined symbols starting at index 1626
=== RUN   TestHasReportedComponents_TrueForNComponents
--- PASS: TestHasReportedComponents_TrueForNComponents (0.00s)
=== RUN   TestHasReportedComponents_FalseWhenNilScan
--- PASS: TestHasReportedComponents_FalseWhenNilScan (0.00s)
=== RUN   TestAllComponentsScanned
=== RUN   TestAllComponentsScanned/true_when_no_UNSCANNED_notes
=== RUN   TestAllComponentsScanned/false_when_UNSCANNED_present
=== RUN   TestAllComponentsScanned/false_when_empty_components
--- PASS: TestAllComponentsScanned (0.00s)
    --- PASS: TestAllComponentsScanned/true_when_no_UNSCANNED_notes (0.00s)
    --- PASS: TestAllComponentsScanned/false_when_UNSCANNED_present (0.00s)
    --- PASS: TestAllComponentsScanned/false_when_empty_components (0.00s)
=== RUN   TestRawListQueryNamespaceAndName
--- PASS: TestRawListQueryNamespaceAndName (0.00s)
=== RUN   TestWaitForVMPresentInCentral_UsesListVirtualMachines
--- PASS: TestWaitForVMPresentInCentral_UsesListVirtualMachines (0.00s)
=== RUN   TestWaitForVMPresentInCentral_PlanSignatureUsesDefaults
--- PASS: TestWaitForVMPresentInCentral_PlanSignatureUsesDefaults (0.00s)
=== RUN   TestWaitForVMScanTimestamp
--- PASS: TestWaitForVMScanTimestamp (0.01s)
=== RUN   TestCentralWaitWithOptionsStubClients
=== RUN   TestCentralWaitWithOptionsStubClients/WaitForVMIdentityFieldsWithOptions
=== RUN   TestCentralWaitWithOptionsStubClients/WaitForVMRunningInCentralWithOptions
=== RUN   TestCentralWaitWithOptionsStubClients/WaitForVMScanNonNilWithOptions
=== RUN   TestCentralWaitWithOptionsStubClients/WaitForVMComponentsReportedWithOptions
=== RUN   TestCentralWaitWithOptionsStubClients/WaitForAllVMComponentsScannedWithOptions
--- PASS: TestCentralWaitWithOptionsStubClients (0.03s)
    --- PASS: TestCentralWaitWithOptionsStubClients/WaitForVMIdentityFieldsWithOptions (0.01s)
    --- PASS: TestCentralWaitWithOptionsStubClients/WaitForVMRunningInCentralWithOptions (0.01s)
    --- PASS: TestCentralWaitWithOptionsStubClients/WaitForVMScanNonNilWithOptions (0.01s)
    --- PASS: TestCentralWaitWithOptionsStubClients/WaitForVMComponentsReportedWithOptions (0.01s)
    --- PASS: TestCentralWaitWithOptionsStubClients/WaitForAllVMComponentsScannedWithOptions (0.01s)
=== RUN   TestActivationFromSubscriptionManagerOutput_Current
=== PAUSE TestActivationFromSubscriptionManagerOutput_Current
=== RUN   TestActivationFromSubscriptionManagerOutput_NotCurrent
=== PAUSE TestActivationFromSubscriptionManagerOutput_NotCurrent
=== RUN   TestActivationFromSubscriptionManagerOutput_Empty
=== PAUSE TestActivationFromSubscriptionManagerOutput_Empty
=== RUN   TestActivationFromSubscriptionManagerOutput_CurrentCaseInsensitive
=== PAUSE TestActivationFromSubscriptionManagerOutput_CurrentCaseInsensitive
=== RUN   TestBuildDnfHistoryPrimingCommand_ReinstallsBasePackage
=== PAUSE TestBuildDnfHistoryPrimingCommand_ReinstallsBasePackage
=== RUN   TestIsSudoPasswordPromptError
=== PAUSE TestIsSudoPasswordPromptError
=== RUN   TestFormatGuestCommandOutputForError
=== PAUSE TestFormatGuestCommandOutputForError
=== RUN   TestDetectRateLimitedNACK_TrueForCentralReason
--- PASS: TestDetectRateLimitedNACK_TrueForCentralReason (0.00s)
=== RUN   TestDetectRateLimitedNACK_FalseForOtherReasons
--- PASS: TestDetectRateLimitedNACK_FalseForOtherReasons (0.00s)
=== RUN   TestContainsCentralRateLimiterMarker
--- PASS: TestContainsCentralRateLimiterMarker (0.00s)
=== RUN   TestContainsCentralRateLimiterMarker_Negative
--- PASS: TestContainsCentralRateLimiterMarker_Negative (0.00s)
=== RUN   TestRateLimitSignalsFromText_BothPaths
--- PASS: TestRateLimitSignalsFromText_BothPaths (0.00s)
=== RUN   TestRateLimitSignalsFromText_Clean
--- PASS: TestRateLimitSignalsFromText_Clean (0.00s)
=== RUN   TestVMIndexMetricSeriesMirrorProductDefinitions
--- PASS: TestVMIndexMetricSeriesMirrorProductDefinitions (0.00s)
=== RUN   TestCentralVirtualMachineIndexProcessedSelector
--- PASS: TestCentralVirtualMachineIndexProcessedSelector (0.00s)
=== RUN   TestSensorIndexReportsSentCentralNotReadySelector
--- PASS: TestSensorIndexReportsSentCentralNotReadySelector (0.00s)
=== RUN   TestIsRateLimitedNACKReasonExportedMatchesCentralsensor
--- PASS: TestIsRateLimitedNACKReasonExportedMatchesCentralsensor (0.00s)
=== RUN   TestParsePrometheusCounter
--- PASS: TestParsePrometheusCounter (0.00s)
=== RUN   TestSummarizeVMIndexEscalation_RateLimitPriority
--- PASS: TestSummarizeVMIndexEscalation_RateLimitPriority (0.00s)
=== RUN   TestSummarizeVMIndexEscalation_VsockMismatch
--- PASS: TestSummarizeVMIndexEscalation_VsockMismatch (0.00s)
=== RUN   TestSummarizeVMIndexEscalation_ComplianceRelaySendFailed
--- PASS: TestSummarizeVMIndexEscalation_ComplianceRelaySendFailed (0.00s)
=== RUN   TestSummarizeVMIndexEscalation_SensorEnqueueBlocked
--- PASS: TestSummarizeVMIndexEscalation_SensorEnqueueBlocked (0.00s)
=== RUN   TestSummarizeVMIndexEscalation_SensorCentralNotReady
--- PASS: TestSummarizeVMIndexEscalation_SensorCentralNotReady (0.00s)
=== RUN   TestSummarizeVMIndexEscalation_SensorForwardError
--- PASS: TestSummarizeVMIndexEscalation_SensorForwardError (0.00s)
=== RUN   TestSummarizeVMIndexEscalation_CentralNotProcessing
--- PASS: TestSummarizeVMIndexEscalation_CentralNotProcessing (0.00s)
=== RUN   TestSummarizeVMIndexEscalation_Unknown
--- PASS: TestSummarizeVMIndexEscalation_Unknown (0.00s)
=== RUN   TestSummarizeVMIndexEscalation_GuestHeuristic
--- PASS: TestSummarizeVMIndexEscalation_GuestHeuristic (0.00s)
=== RUN   TestPodLogsSnapshot_CombinedLogs
--- PASS: TestPodLogsSnapshot_CombinedLogs (0.00s)
=== RUN   TestFlattenVMIndexEscalationSnapshot
--- PASS: TestFlattenVMIndexEscalationSnapshot (0.00s)
=== RUN   TestCollectPodLogsBySelector_Validation
--- PASS: TestCollectPodLogsBySelector_Validation (0.00s)
=== RUN   TestCollectPodLogsBySelector_NoPods
--- PASS: TestCollectPodLogsBySelector_NoPods (0.00s)
=== RUN   TestCollectPodLogsBySelector_WithMatchingPod
--- PASS: TestCollectPodLogsBySelector_WithMatchingPod (0.00s)
=== RUN   TestScrapePodMetricsViaProxy_Validation
--- PASS: TestScrapePodMetricsViaProxy_Validation (0.00s)
=== RUN   TestCollectMetricsFromPodsMatchingSelector_Validation
--- PASS: TestCollectMetricsFromPodsMatchingSelector_Validation (0.00s)
=== RUN   TestSummarizeVMIndexEscalation_RelayAcceptOrRead
--- PASS: TestSummarizeVMIndexEscalation_RelayAcceptOrRead (0.00s)
=== RUN   TestSummarizeVMIndexEscalation_SensorReceivePathStalled
--- PASS: TestSummarizeVMIndexEscalation_SensorReceivePathStalled (0.00s)
=== RUN   TestSummarizeVMIndexEscalation_CentralACKMissing
--- PASS: TestSummarizeVMIndexEscalation_CentralACKMissing (0.00s)
=== RUN   TestSummarizeVMIndexEscalation_CentralVMVisibilityHint
--- PASS: TestSummarizeVMIndexEscalation_CentralVMVisibilityHint (0.00s)
=== RUN   TestSummarizeVMIndexEscalation_CentralScanVisibilityHint
--- PASS: TestSummarizeVMIndexEscalation_CentralScanVisibilityHint (0.00s)
=== RUN   TestScrapePodMetricsViaPortForward_Validation
--- PASS: TestScrapePodMetricsViaPortForward_Validation (0.00s)
=== RUN   TestCanonicalMetricGoSourceFilesDriftGuard
--- PASS: TestCanonicalMetricGoSourceFilesDriftGuard (0.00s)
=== RUN   TestChooseRepo2CPESource
=== PAUSE TestChooseRepo2CPESource
=== RUN   TestRunRoxagentOnce_RejectsUnsupportedInstallPath
=== PAUSE TestRunRoxagentOnce_RejectsUnsupportedInstallPath
=== RUN   TestVerboseOutputLooksLikeReport_FalseForQuietOutput
=== PAUSE TestVerboseOutputLooksLikeReport_FalseForQuietOutput
=== RUN   TestVerboseOutputLooksLikeReport_TrueForSampleReportJSON
=== PAUSE TestVerboseOutputLooksLikeReport_TrueForSampleReportJSON
=== RUN   TestRoxagentInstallHelpers_BinaryPresentAndExecutableChecks
=== PAUSE TestRoxagentInstallHelpers_BinaryPresentAndExecutableChecks
=== RUN   TestRoxagentInstallHelpers_QuotesPathsWithSpaces
=== PAUSE TestRoxagentInstallHelpers_QuotesPathsWithSpaces
=== RUN   TestBuildRoxagentInstallArgs_UsesSudoInstallModeAndPaths
=== PAUSE TestBuildRoxagentInstallArgs_UsesSudoInstallModeAndPaths
=== RUN   TestSSHCommandArgs_UsesIdentityAndNamespace
--- PASS: TestSSHCommandArgs_UsesIdentityAndNamespace (0.00s)
=== RUN   TestBuildVirtctlSSHCommand_QuotesArguments
--- PASS: TestBuildVirtctlSSHCommand_QuotesArguments (0.00s)
=== RUN   TestSCPToArgs_RemoteTargetShape
--- PASS: TestSCPToArgs_RemoteTargetShape (0.00s)
=== RUN   TestRenderCloudInit_InjectsAuthorizedKey
--- PASS: TestRenderCloudInit_InjectsAuthorizedKey (0.00s)
=== RUN   TestRenderCloudInit_IncludesGuestUser
--- PASS: TestRenderCloudInit_IncludesGuestUser (0.00s)
=== RUN   TestRenderCloudInit_GrantsPasswordlessSudo
--- PASS: TestRenderCloudInit_GrantsPasswordlessSudo (0.00s)
=== RUN   TestRenderCloudInit_MissingGuestUser
--- PASS: TestRenderCloudInit_MissingGuestUser (0.00s)
=== RUN   TestRenderCloudInit_MissingSSHPublicKey
--- PASS: TestRenderCloudInit_MissingSSHPublicKey (0.00s)
=== RUN   TestCreateVirtualMachine_RequiresCloudInitFields
--- PASS: TestCreateVirtualMachine_RequiresCloudInitFields (0.00s)
=== RUN   TestCreateVirtualMachine_SetsDefaultMemoryRequest
=== PAUSE TestCreateVirtualMachine_SetsDefaultMemoryRequest
=== RUN   TestVMFailureConditionDetail
=== PAUSE TestVMFailureConditionDetail
=== RUN   TestPollUntil_EscalationFiresOnNthPoll
--- PASS: TestPollUntil_EscalationFiresOnNthPoll (0.02s)
=== RUN   TestPollUntil_EscalationSkipsSuccessfulPolls
--- PASS: TestPollUntil_EscalationSkipsSuccessfulPolls (0.00s)
=== RUN   TestPollUntil_EscalationFiresOnceAfterThreshold
--- PASS: TestPollUntil_EscalationFiresOnceAfterThreshold (0.01s)
=== RUN   TestPollUntil_EscalationSharedMutuallyExclusiveWithEscalation
--- PASS: TestPollUntil_EscalationSharedMutuallyExclusiveWithEscalation (0.00s)
=== RUN   TestPollUntil_SharedEscalationAccumulatesAcrossChainedPolls
--- PASS: TestPollUntil_SharedEscalationAccumulatesAcrossChainedPolls (0.01s)
=== RUN   TestBeginComposedWait_ResetsSharedStateForNewComposedWait
--- PASS: TestBeginComposedWait_ResetsSharedStateForNewComposedWait (0.00s)
=== CONT  TestActivationFromSubscriptionManagerOutput_Current
=== CONT  TestVMFailureConditionDetail
--- PASS: TestActivationFromSubscriptionManagerOutput_Current (0.00s)
=== CONT  TestChooseRepo2CPESource
--- PASS: TestVMFailureConditionDetail (0.00s)
=== CONT  TestRoxagentInstallHelpers_BinaryPresentAndExecutableChecks
=== CONT  TestVerboseOutputLooksLikeReport_TrueForSampleReportJSON
=== CONT  TestBuildRoxagentInstallArgs_UsesSudoInstallModeAndPaths
--- PASS: TestVerboseOutputLooksLikeReport_TrueForSampleReportJSON (0.00s)
--- PASS: TestBuildRoxagentInstallArgs_UsesSudoInstallModeAndPaths (0.00s)
=== CONT  TestRoxagentInstallHelpers_QuotesPathsWithSpaces
=== CONT  TestCreateVirtualMachine_SetsDefaultMemoryRequest
=== RUN   TestChooseRepo2CPESource/last_primary_slot_before_boundary
--- PASS: TestRoxagentInstallHelpers_QuotesPathsWithSpaces (0.00s)
=== CONT  TestFormatGuestCommandOutputForError
=== PAUSE TestChooseRepo2CPESource/last_primary_slot_before_boundary
=== CONT  TestRunRoxagentOnce_RejectsUnsupportedInstallPath
=== RUN   TestChooseRepo2CPESource/boundary_falls_back_when_attemptsMade_equals_max
=== CONT  TestBuildDnfHistoryPrimingCommand_ReinstallsBasePackage
=== PAUSE TestChooseRepo2CPESource/boundary_falls_back_when_attemptsMade_equals_max
--- PASS: TestFormatGuestCommandOutputForError (0.00s)
=== CONT  TestIsSudoPasswordPromptError
=== CONT  TestVerboseOutputLooksLikeReport_FalseForQuietOutput
=== RUN   TestIsSudoPasswordPromptError/terminal_required
=== CONT  TestActivationFromSubscriptionManagerOutput_NotCurrent
=== CONT  TestActivationFromSubscriptionManagerOutput_CurrentCaseInsensitive
=== CONT  TestActivationFromSubscriptionManagerOutput_Empty
--- PASS: TestBuildDnfHistoryPrimingCommand_ReinstallsBasePackage (0.00s)
=== RUN   TestChooseRepo2CPESource/plan_boundary_three_of_three
--- PASS: TestActivationFromSubscriptionManagerOutput_Empty (0.00s)
--- PASS: TestRoxagentInstallHelpers_BinaryPresentAndExecutableChecks (0.00s)
--- PASS: TestRunRoxagentOnce_RejectsUnsupportedInstallPath (0.00s)
--- PASS: TestVerboseOutputLooksLikeReport_FalseForQuietOutput (0.00s)
=== PAUSE TestIsSudoPasswordPromptError/terminal_required
--- PASS: TestActivationFromSubscriptionManagerOutput_NotCurrent (0.00s)
=== RUN   TestIsSudoPasswordPromptError/password_required
=== PAUSE TestIsSudoPasswordPromptError/password_required
=== RUN   TestIsSudoPasswordPromptError/other_sudo_failure
=== PAUSE TestIsSudoPasswordPromptError/other_sudo_failure
=== RUN   TestIsSudoPasswordPromptError/unrelated_error
=== PAUSE TestChooseRepo2CPESource/plan_boundary_three_of_three
=== PAUSE TestIsSudoPasswordPromptError/unrelated_error
=== RUN   TestChooseRepo2CPESource/zero_max_immediate_fallback_on_first_index
=== PAUSE TestChooseRepo2CPESource/zero_max_immediate_fallback_on_first_index
=== CONT  TestIsSudoPasswordPromptError/other_sudo_failure
=== RUN   TestChooseRepo2CPESource/after_boundary_stays_fallback
=== CONT  TestIsSudoPasswordPromptError/password_required
=== CONT  TestIsSudoPasswordPromptError/terminal_required
=== CONT  TestIsSudoPasswordPromptError/unrelated_error
=== PAUSE TestChooseRepo2CPESource/after_boundary_stays_fallback
=== RUN   TestChooseRepo2CPESource/single_primary_slot_then_fallback
=== PAUSE TestChooseRepo2CPESource/single_primary_slot_then_fallback
=== RUN   TestChooseRepo2CPESource/zero_attempts_uses_primary_when_budget_positive
=== PAUSE TestChooseRepo2CPESource/zero_attempts_uses_primary_when_budget_positive
--- PASS: TestIsSudoPasswordPromptError (0.00s)
    --- PASS: TestIsSudoPasswordPromptError/other_sudo_failure (0.00s)
    --- PASS: TestIsSudoPasswordPromptError/password_required (0.00s)
    --- PASS: TestIsSudoPasswordPromptError/unrelated_error (0.00s)
    --- PASS: TestIsSudoPasswordPromptError/terminal_required (0.00s)
=== RUN   TestChooseRepo2CPESource/mid_range_still_primary
=== PAUSE TestChooseRepo2CPESource/mid_range_still_primary
=== CONT  TestChooseRepo2CPESource/last_primary_slot_before_boundary
=== CONT  TestChooseRepo2CPESource/single_primary_slot_then_fallback
=== CONT  TestChooseRepo2CPESource/plan_boundary_three_of_three
=== CONT  TestChooseRepo2CPESource/boundary_falls_back_when_attemptsMade_equals_max
=== CONT  TestChooseRepo2CPESource/zero_max_immediate_fallback_on_first_index
=== CONT  TestChooseRepo2CPESource/after_boundary_stays_fallback
=== CONT  TestChooseRepo2CPESource/mid_range_still_primary
=== CONT  TestChooseRepo2CPESource/zero_attempts_uses_primary_when_budget_positive
--- PASS: TestChooseRepo2CPESource (0.00s)
    --- PASS: TestChooseRepo2CPESource/last_primary_slot_before_boundary (0.00s)
    --- PASS: TestChooseRepo2CPESource/plan_boundary_three_of_three (0.00s)
    --- PASS: TestChooseRepo2CPESource/boundary_falls_back_when_attemptsMade_equals_max (0.00s)
    --- PASS: TestChooseRepo2CPESource/single_primary_slot_then_fallback (0.00s)
    --- PASS: TestChooseRepo2CPESource/zero_max_immediate_fallback_on_first_index (0.00s)
    --- PASS: TestChooseRepo2CPESource/mid_range_still_primary (0.00s)
    --- PASS: TestChooseRepo2CPESource/zero_attempts_uses_primary_when_budget_positive (0.00s)
    --- PASS: TestChooseRepo2CPESource/after_boundary_stays_fallback (0.00s)
--- PASS: TestActivationFromSubscriptionManagerOutput_CurrentCaseInsensitive (0.00s)
--- PASS: TestCreateVirtualMachine_SetsDefaultMemoryRequest (0.01s)
PASS
ok  	github.com/stackrox/rox/tests/vmhelpers	2.316s
=== RUN   TestLoadVMScanConfig_MissingRequired
--- PASS: TestLoadVMScanConfig_MissingRequired (0.00s)
=== RUN   TestLoadVMScanConfig_RequiresActivationInputsWhenEnabled
--- PASS: TestLoadVMScanConfig_RequiresActivationInputsWhenEnabled (0.00s)
=== RUN   TestLoadVMScanConfig_RequiresActivationKeyWhenEnabled
--- PASS: TestLoadVMScanConfig_RequiresActivationKeyWhenEnabled (0.00s)
=== RUN   TestLoadVMScanConfig_RequiresActivationOrgWhenTruthyNonTrue
--- PASS: TestLoadVMScanConfig_RequiresActivationOrgWhenTruthyNonTrue (0.00s)
PASS
ok  	github.com/stackrox/rox/tests	(cached)
=== RUN   TestVMScanning
    common.go:85: Mar 27 20:24:38.385 VM scanning setup: initialize test contexts
    common.go:85: Mar 27 20:24:38.385 Running TestVMScanning with a timeout of 1h30m0s plus 10m0s for cleanup
    common.go:85: Mar 27 20:24:38.385 VM scanning setup: load suite configuration from environment
    common.go:85: Mar 27 20:24:38.385 VM scanning setup: create Kubernetes clients
    common.go:85: Mar 27 20:24:38.385 VM scanning setup: connect to Central gRPC
    common.go:85: Mar 27 20:24:38.386 VM scanning setup: verify central/sensor connectivity and feature gates
    common.go:85: Mar 27 20:24:38.386 Waiting for deployment "sensor" in namespace "stackrox" to be ready
    common.go:80: [DEBUG] GET https://api.piotr-03-23-work.ocp.infra.rox.systems:6443/apis/apps/v1/namespaces/stackrox/deployments/sensor?timeout=30s
    common.go:85: Mar 27 20:24:40.674 deployment "sensor" in namespace "stackrox" READY (1/1 ready replicas)
    common.go:85: Mar 27 20:24:40.674 Waiting until central-sensor connection is in state(s) [HEALTHY]...
    connect_to_central.go:120: gRPC call /v1.ClustersService/GetClusters succeeded in 446.744333ms
    common.go:85: Mar 27 20:24:41.122 Central-sensor connection now in state "HEALTHY".
    connect_to_central.go:120: gRPC call /v1.FeatureFlagService/GetFeatureFlags succeeded in 110.299042ms
    common.go:85: Mar 27 20:24:41.232 VM scanning setup: verify cluster VSOCK readiness
    common.go:80: [DEBUG] GET https://api.piotr-03-23-work.ocp.infra.rox.systems:6443/apis/kubevirt.io/v1/namespaces/openshift-cnv/kubevirts?timeout=30s
    common.go:80: [DEBUG] GET https://api.piotr-03-23-work.ocp.infra.rox.systems:6443/api/v1/namespaces/openshift-cnv/pods?labelSelector=kubevirt.io%3Dvirt-handler
    common.go:85: Mar 27 20:24:41.494 VM scanning setup: resolve SSH identity and configure virtctl
    common.go:85: Mar 27 20:24:41.495 VM scanning setup: provision persistent VMs
    common.go:85: Mar 27 20:24:41.495 provision persistent VMs: creating namespace "vm-scan-e2e-b4f96fed"
    common.go:80: [DEBUG] POST https://api.piotr-03-23-work.ocp.infra.rox.systems:6443/api/v1/namespaces?timeout=30s
    common.go:85: Mar 27 20:24:41.624 provision persistent VMs: creating VM vm-scan-e2e-b4f96fed/vm-rhel9 with image "registry.redhat.io/rhel9/rhel-guest-image@sha256:ab4ec16077fe00e3c7efd0b2f6a77571f3645f5c95befc4d917757dc88b2f423"
    common.go:80: [DEBUG] POST https://api.piotr-03-23-work.ocp.infra.rox.systems:6443/apis/kubevirt.io/v1/namespaces/vm-scan-e2e-b4f96fed/virtualmachines?timeout=30s
    common.go:85: Mar 27 20:24:41.796 provision persistent VMs: creating VM vm-scan-e2e-b4f96fed/vm-rhel10 with image "registry.redhat.io/rhel10/rhel-guest-image@sha256:c03936a065f2e34b05ae7d14a62a5afdb81b31eb97bdfe2633b349b6d4100c67"
    common.go:80: [DEBUG] POST https://api.piotr-03-23-work.ocp.infra.rox.systems:6443/apis/kubevirt.io/v1/namespaces/vm-scan-e2e-b4f96fed/virtualmachines?timeout=30s
    common.go:85: Mar 27 20:24:41.942 provision persistent VMs: waiting for VMI object vm-scan-e2e-b4f96fed/vm-rhel9 (timeout=20m0s)
    common.go:80: [DEBUG] GET https://api.piotr-03-23-work.ocp.infra.rox.systems:6443/apis/kubevirt.io/v1/namespaces/vm-scan-e2e-b4f96fed/virtualmachineinstances/vm-rhel9?timeout=30s
    vm.go:310: [vmhelpers] wait VMI vm-scan-e2e-b4f96fed/vm-rhel9 exists: attempt 1/600 (retries left: 599): vmi now exists
    common.go:85: Mar 27 20:24:42.067 provision persistent VMs: waiting for VMI Running vm-scan-e2e-b4f96fed/vm-rhel9 (timeout=20m0s)
    common.go:80: [DEBUG] GET https://api.piotr-03-23-work.ocp.infra.rox.systems:6443/apis/kubevirt.io/v1/namespaces/vm-scan-e2e-b4f96fed/virtualmachineinstances/vm-rhel9?timeout=30s
    vm.go:371: [vmhelpers] wait VMI vm-scan-e2e-b4f96fed/vm-rhel9 running: attempt 1/600 (retries left: 599): phase="Pending"
    common.go:80: [DEBUG] GET https://api.piotr-03-23-work.ocp.infra.rox.systems:6443/apis/kubevirt.io/v1/namespaces/vm-scan-e2e-b4f96fed/virtualmachineinstances/vm-rhel9?timeout=30s
    common.go:80: [DEBUG] GET https://api.piotr-03-23-work.ocp.infra.rox.systems:6443/apis/kubevirt.io/v1/namespaces/vm-scan-e2e-b4f96fed/virtualmachineinstances/vm-rhel9?timeout=30s
    common.go:80: [DEBUG] GET https://api.piotr-03-23-work.ocp.infra.rox.systems:6443/apis/kubevirt.io/v1/namespaces/vm-scan-e2e-b4f96fed/virtualmachineinstances/vm-rhel9?timeout=30s
    common.go:80: [DEBUG] GET https://api.piotr-03-23-work.ocp.infra.rox.systems:6443/apis/kubevirt.io/v1/namespaces/vm-scan-e2e-b4f96fed/virtualmachineinstances/vm-rhel9?timeout=30s
    vm.go:371: [vmhelpers] wait VMI vm-scan-e2e-b4f96fed/vm-rhel9 running: attempt 5/600 (retries left: 595): phase="Scheduled" ready.status="False" ready.reason="GuestNotRunning" ready.message="Guest VM is not reported as running"
    common.go:80: [DEBUG] GET https://api.piotr-03-23-work.ocp.infra.rox.systems:6443/apis/kubevirt.io/v1/namespaces/vm-scan-e2e-b4f96fed/virtualmachineinstances/vm-rhel9?timeout=30s
    common.go:80: [DEBUG] GET https://api.piotr-03-23-work.ocp.infra.rox.systems:6443/apis/kubevirt.io/v1/namespaces/vm-scan-e2e-b4f96fed/virtualmachineinstances/vm-rhel9?timeout=30s
    common.go:85: Mar 27 20:24:54.189 provision persistent VMs: VMI is running vm-scan-e2e-b4f96fed/vm-rhel9
    common.go:85: Mar 27 20:24:54.189 provision persistent VMs: waiting for VMI object vm-scan-e2e-b4f96fed/vm-rhel10 (timeout=20m0s)
    common.go:80: [DEBUG] GET https://api.piotr-03-23-work.ocp.infra.rox.systems:6443/apis/kubevirt.io/v1/namespaces/vm-scan-e2e-b4f96fed/virtualmachineinstances/vm-rhel10?timeout=30s
    vm.go:310: [vmhelpers] wait VMI vm-scan-e2e-b4f96fed/vm-rhel10 exists: attempt 1/600 (retries left: 599): vmi now exists
    common.go:85: Mar 27 20:24:54.311 provision persistent VMs: waiting for VMI Running vm-scan-e2e-b4f96fed/vm-rhel10 (timeout=20m0s)
    common.go:80: [DEBUG] GET https://api.piotr-03-23-work.ocp.infra.rox.systems:6443/apis/kubevirt.io/v1/namespaces/vm-scan-e2e-b4f96fed/virtualmachineinstances/vm-rhel10?timeout=30s
    vm.go:371: [vmhelpers] wait VMI vm-scan-e2e-b4f96fed/vm-rhel10 running: attempt 1/600 (retries left: 599): phase="Running" ready.status="True"
    common.go:85: Mar 27 20:24:54.435 provision persistent VMs: VMI is running vm-scan-e2e-b4f96fed/vm-rhel10
    common.go:85: Mar 27 20:24:54.435 VM scanning setup: prepare guests (ssh/cloud-init/roxagent/activation)
    common.go:85: Mar 27 20:24:54.435 Waiting for SSH to become reachable on vm-scan-e2e-b4f96fed/vm-rhel9 (timeout=10m0s) ...
    guest.go:130: [vmhelpers] wait SSH vm-scan-e2e-b4f96fed/vm-rhel9 reachable: attempt 1/120 (retries left: 119): ssh not reachable yet (stderr: Internal error occurred: dialing VM: dial tcp 10.131.0.51:22: connect: no route to host
        Connection closed by UNKNOWN port 65535
        exit status 255)
    guest.go:130: [vmhelpers] wait SSH vm-scan-e2e-b4f96fed/vm-rhel9 reachable: attempt 5/120 (retries left: 115): ssh not reachable yet (stderr: Internal error occurred: dialing VM: dial tcp 10.131.0.51:22: connect: no route to host
        Connection closed by UNKNOWN port 65535
        exit status 255)
    guest.go:130: [vmhelpers] wait SSH vm-scan-e2e-b4f96fed/vm-rhel9 reachable: attempt 10/120 (retries left: 110): ssh not reachable yet (stderr: Internal error occurred: dialing VM: dial tcp 10.131.0.51:22: connect: no route to host
        Connection closed by UNKNOWN port 65535
        exit status 255)
    guest.go:130: [vmhelpers] wait SSH vm-scan-e2e-b4f96fed/vm-rhel9 reachable: attempt 15/120 (retries left: 105): ssh not reachable yet (stderr: Internal error occurred: dialing VM: dial tcp 10.131.0.51:22: connect: no route to host
        Connection closed by UNKNOWN port 65535
        exit status 255)
    guest.go:130: [vmhelpers] wait SSH vm-scan-e2e-b4f96fed/vm-rhel9 reachable: attempt 20/120 (retries left: 100): ssh not reachable yet (stderr: Internal error occurred: dialing VM: dial tcp 10.131.0.51:22: connect: no route to host
        Connection closed by UNKNOWN port 65535
        exit status 255)
    guest.go:130: [vmhelpers] wait SSH vm-scan-e2e-b4f96fed/vm-rhel9 reachable: attempt 25/120 (retries left: 95): ssh not reachable yet (stderr: Internal error occurred: dialing VM: dial tcp :22: connect: connection refused
        Connection closed by UNKNOWN port 65535
        exit status 255)
    guest.go:130: [vmhelpers] wait SSH vm-scan-e2e-b4f96fed/vm-rhel9 reachable: attempt 30/120 (retries left: 90): ssh not reachable yet (stderr: Internal error occurred: dialing VM: dial tcp :22: connect: connection refused
        Connection closed by UNKNOWN port 65535
        exit status 255)
    guest.go:130: [vmhelpers] wait SSH vm-scan-e2e-b4f96fed/vm-rhel9 reachable: attempt 35/120 (retries left: 85): ssh not reachable yet (stderr: Internal error occurred: dialing VM: dial tcp :22: connect: connection refused
        Connection closed by UNKNOWN port 65535
        exit status 255)
    guest.go:130: [vmhelpers] wait SSH vm-scan-e2e-b4f96fed/vm-rhel9 reachable: attempt 40/120 (retries left: 80): ssh not reachable yet (stderr: Internal error occurred: dialing VM: dial tcp 10.131.0.51:22: connect: connection refused
        Connection closed by UNKNOWN port 65535
        exit status 255)
    common.go:85: Mar 27 20:28:31.083 SSH is reachable on vm-scan-e2e-b4f96fed/vm-rhel9 in 3m37s
    common.go:85: Mar 27 20:28:31.083 Waiting for cloud-init to finish on vm-scan-e2e-b4f96fed/vm-rhel9 (timeout=10m0s) ...
    common.go:85: Mar 27 20:29:57.216 Cloud-init finished on vm-scan-e2e-b4f96fed/vm-rhel9 in 1m26s
    common.go:85: Mar 27 20:29:57.216 Verifying sudo on vm-scan-e2e-b4f96fed/vm-rhel9 (timeout=10m0s) ...
    common.go:85: Mar 27 20:30:03.167 Sudo verified on vm-scan-e2e-b4f96fed/vm-rhel9 in 6s
    common.go:85: Mar 27 20:30:03.167 Copying roxagent binary on vm-scan-e2e-b4f96fed/vm-rhel9 (timeout=10m0s) ...
    common.go:85: Mar 27 20:31:07.666 Roxagent binary copied on vm-scan-e2e-b4f96fed/vm-rhel9 in 1m4s
    common.go:85: Mar 27 20:31:07.666 Verifying roxagent install path and mode on vm-scan-e2e-b4f96fed/vm-rhel9 (timeout=10m0s) ...
    common.go:85: Mar 27 20:31:23.061 Roxagent install path and mode verified on vm-scan-e2e-b4f96fed/vm-rhel9 in 15s
    common.go:85: Mar 27 20:31:23.061 Checking activation status on vm-scan-e2e-b4f96fed/vm-rhel9 (timeout=10m0s) ...
    vm_scanning_suite_test.go:327:
        	Error Trace:	/Users/prygiels/src/go/src/github.com/stackrox/stackrox/tests/vm_scanning_suite_test.go:327
        	            				/Users/prygiels/src/go/src/github.com/stackrox/stackrox/tests/vm_scanning_suite_test.go:117
        	            				/Users/prygiels/src/go/pkg/mod/github.com/stretchr/testify@v1.11.1/suite/suite.go:211
        	            				/Users/prygiels/src/go/src/github.com/stackrox/stackrox/tests/vm_scanning_suite_test.go:73
        	Error:      	Received unexpected error:
        	            	subscription-manager status on vm-scan-e2e-b4f96fed/vm-rhel9: exit status 1 (output: +-------------------------------------------+
        	            	   System Status Details
        	            	+-------------------------------------------+
        	            	Overall Status: Not registered


        	            	WARNING

        	            	The yum/dnf plugins: /etc/dnf/plugins/subscription-manager.conf, /etc/dnf/plugins/product-id.conf were automatically enabled for the benefit of Red Hat Subscription Management. If not desired, use "subscription-manager config --rhsm.auto_enable_yum_plugins=0" to block this behavior.


        	            	Warning: Permanently added 'vmi.vm-rhel9.vm-scan-e2e-b4f96fed' (ED25519) to the list of known hosts.
        	            	exit status 1)
        	Test:       	TestVMScanning
--- FAIL: TestVMScanning (436.24s)
FAIL
FAIL	github.com/stackrox/rox/tests	437.417s
FAIL

User request (feature): "design the end-to-end tests for the VM scanning feature." User request (branch): "add the changes to the piotr/ROX-29577-VM4VM-e2e-tests branch." Add a Go test_e2e VM-scanning suite that provisions RHEL 9 and RHEL 10 KubeVirt VMs, installs and runs roxagent, validates Central scan visibility, and captures progressive diagnostics for flaky-path triage. Wire a dedicated OpenShift CI job and e2e scripts with CNV/VSOCK prerequisite checks and activation/repo2cpe guardrails. AI-assisted: this commit was partially generated with AI assistance. Made-with: Cursor

vikin91 · 2026-03-27T17:49:01Z

This change is part of the following stack:

ROX-29577: VM4VM e2e tests #19660 ◀

_{Change managed by git-spice.}

openshift-ci · 2026-03-27T17:49:04Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

sourcery-ai

Sorry @github-actions[bot], your pull request is larger than the review limit of 150000 diff characters

rhacs-bot · 2026-03-27T18:06:54Z

Images are ready for the commit at fc20580.

To use with deploy scripts, first export MAIN_IMAGE_TAG=4.11.x-482-gfc20580a6a.

codecov · 2026-03-27T18:22:39Z

Codecov Report

❌ Patch coverage is 41.34119% with 796 lines in your changes missing coverage. Please review.
✅ Project coverage is 49.32%. Comparing base (0e2ec56) to head (fc20580).
⚠️ Report is 3 commits behind head on master.

Files with missing lines	Patch %	Lines
tests/vmhelpers/observability.go	46.48%	246 Missing and 13 partials ⚠️
tests/vmhelpers/vm.go	30.79%	200 Missing and 9 partials ⚠️
tests/vmhelpers/guest.go	15.33%	127 Missing ⚠️
tests/vmhelpers/roxagent.go	22.68%	74 Missing and 1 partial ⚠️
tests/vmhelpers/central.go	61.27%	49 Missing and 18 partials ⚠️
tests/vmhelpers/wait.go	61.25%	22 Missing and 9 partials ⚠️
tests/vmhelpers/virtctl.go	60.56%	26 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #19660      +/-   ##
==========================================
- Coverage   49.34%   49.32%   -0.02%     
==========================================
  Files        2743     2750       +7     
  Lines      207019   208394    +1375     
==========================================
+ Hits       102152   102794     +642     
- Misses      97269    97963     +694     
- Partials     7598     7637      +39

Flag	Coverage Δ
go-unit-tests	`49.32% <41.34%> (-0.02%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

coderabbitai · 2026-03-27T18:34:00Z

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 49330f2f-cfab-474d-9f0d-8c5a5fa0a71d

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch piotr/ROX-29577-VM4VM-e2e-tests

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

vikin91 added 3 commits March 27, 2026 14:29

Tmp: Add spec and impl plan (Superpower artifacts)

1623b76

ROX-29577: relax VM-scanning VSOCK preflight for CNV layouts

5c6c460

openshift-ci bot added the do-not-merge/work-in-progress label Mar 27, 2026

github-actions bot added area/ci ai-review labels Mar 27, 2026

sourcery-ai bot reviewed Mar 27, 2026

View reviewed changes

vikin91 added 2 commits March 27, 2026 19:26

Improve progress reporting and add partial assertions

31e471f

Fix style and stuck SSH access attempt

e7cde69

vikin91 added 7 commits March 27, 2026 19:34

Fix style, fix virtctl ssh

b11511c

Fix virtctl ssh 2

79f2f48

Fix virtctl ssh 3

767d3c4

More informative fail messages

b935833

Fix agent +x perimssion

b94b0d1

Fix activation command. Log wording

5dec33d

Add temporary test.sh to run this from local machine

fc20580

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ROX-29577: VM4VM e2e tests #19660

ROX-29577: VM4VM e2e tests #19660
vikin91 wants to merge 12 commits intomasterfrom
piotr/ROX-29577-VM4VM-e2e-tests

vikin91 commented Mar 27, 2026 •

edited

Loading

Uh oh!

vikin91 commented Mar 27, 2026 •

edited

Loading

Uh oh!

openshift-ci bot commented Mar 27, 2026

Uh oh!

sourcery-ai bot left a comment

Uh oh!

rhacs-bot commented Mar 27, 2026 •

edited

Loading

Uh oh!

codecov bot commented Mar 27, 2026 •

edited

Loading

Uh oh!

coderabbitai bot commented Mar 27, 2026 •

edited

Loading

Review skipped

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

vikin91 commented Mar 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

User-facing documentation

Testing and quality

Automated testing

How I validated my change

Uh oh!

vikin91 commented Mar 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openshift-ci bot commented Mar 27, 2026

Uh oh!

sourcery-ai bot left a comment

Choose a reason for hiding this comment

Uh oh!

rhacs-bot commented Mar 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Mar 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

coderabbitai bot commented Mar 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

vikin91 commented Mar 27, 2026 •

edited

Loading

vikin91 commented Mar 27, 2026 •

edited

Loading

rhacs-bot commented Mar 27, 2026 •

edited

Loading

codecov bot commented Mar 27, 2026 •

edited

Loading

coderabbitai bot commented Mar 27, 2026 •

edited

Loading