ROX-29771: Refactor common nongroovy test code#17374
Conversation
|
Skipping CI for Draft Pull Request. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #17374 +/- ##
=======================================
Coverage 49.40% 49.40%
=======================================
Files 2685 2685
Lines 197716 197716
=======================================
+ Hits 97686 97691 +5
+ Misses 92401 92400 -1
+ Partials 7629 7625 -4
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
Images are ready for the commit at 7bec0eb. To use with deploy scripts, first |
d74dfdc to
e52691d
Compare
|
/retest |
2 similar comments
|
/retest |
|
/retest |
|
Images are ready for the commit at 5a3a9f6. To use with deploy scripts, first |
7bec0eb to
5a3a9f6
Compare
|
/retest |
|
/retest-times 20 gke-nongroovy-e2e-tests |
|
/test gke-nongroovy-e2e-tests |
12 similar comments
|
/test gke-nongroovy-e2e-tests |
|
/test gke-nongroovy-e2e-tests |
|
/test gke-nongroovy-e2e-tests |
|
/test gke-nongroovy-e2e-tests |
|
/test gke-nongroovy-e2e-tests |
|
/test gke-nongroovy-e2e-tests |
|
/test gke-nongroovy-e2e-tests |
|
/test gke-nongroovy-e2e-tests |
|
/test gke-nongroovy-e2e-tests |
|
/test gke-nongroovy-e2e-tests |
|
/test gke-nongroovy-e2e-tests |
|
/test gke-nongroovy-e2e-tests |
|
@vikin91: The following tests failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
|
/retest |
lvalerom
left a comment
There was a problem hiding this comment.
LGTM. Thank you for the improvements! 🙂
…edScopes
This commit addresses flakiness in e2e tests by improving deployment management
and test reliability:
1. **Refactored deployment creation** (common.go):
- Replaced kubectl CLI calls with K8s API client for better control
- Added extensive logging at each step of deployment lifecycle
- Bubbled up namespace parameter (configurable, defaults to 'default')
- Added waitForDeploymentReadyInK8s() to ensure deployments are ready in K8s
before waiting for Central ingestion
- Renamed functions for clarity:
* waitForDeployment -> waitForDeploymentInCentral
* waitForDeploymentCount -> waitForDeploymentCountInCentral
2. **Fixed TestExcludedScopes** (excluded_scopes_test.go):
- Changed image from 'nginx' to 'quay.io/rhacs-eng/qa-multi-arch-nginx:latest'
- This ensures the 'Latest tag' policy is triggered (required for test logic)
- Updated to use new deployment functions with explicit namespace
3. **Updated related tests** for consistency:
- backup_test.go: Use prefetched image
- active_vuln_test.go: Use prefetched images from quay.io to avoid rate limiting
- resourcecollection_test.go: Use renamed waitForDeploymentCountInCentral
All changes maintain backward compatibility while improving debuggability
and reducing flakiness in CI environments.
Partially AI-generated.
Resolves: ROX-29771
Replace undefined waitForDeployment with waitForDeploymentInCentral to fix compilation error in excluded_scopes_test.go. Test now properly waits for deployment to be registered in Central before proceeding. User request: fix compilation issues in TestExcludedScopes test migrated to use prefetcher approach. Partially AI-assisted.
- Use t.Logf instead of log.Infof - Add missing namespace args - Readd removed comment
90e73f3 to
d4d8c77
Compare
Description
(Extracted from #17354, which was extracted from #17216)
🛑 Blocks #17354not anymoreRefactor nongroovy Go e2e test infrastructure to improve robustness, maintainability, and debuggability.
Problems:
TestPodandTestContainerInstances(recently disabled) due to Sensor being restarted in the tests running before them - tests were enabled and fixed in ROX-29771: Unflake TestPods and TestContainerInstances #17216, ROX-29771: Use prefetched images in non-groovy tests #17354, ROX-31331: Require Collector reporting all processes #17551TestDelegatedScanningwas restarting Sensor that affected the other tests. After adding functions that wait for Sensor to be up before starting the test, theTestDelegatedScanningstarted flaking itself (unable to delete a pod).TestDelegatedScanningwas executing kubectl command to delete a pod once and then waited for 30s for the pod to be gone. However, due to the nature of the test, the pod might have not been created yet at the time of executing the command to delete it (race). The test was failing because the pod "refused to die".What has been changed in this PR:
kubectlCLI calls with K8s API client for deployment creationTestDelegatedScanning): Call create deployment and delete pod using k8s API with retries. Before it was ancmd.execwith single attempt.waitForDeployment→waitForDeploymentInCentral)waitForDeploymentReadyInK8sto check deployment status before waiting in Centralt.Logfinstead oflog.InfofUser-facing documentation
Testing and quality
Automated testing
Modified existing e2e tests to use the new refactored helper functions.
How I validated my change
ocp-4-18-nongroovy-e2e-testsandocp-4-19-nongroovy-e2e-tests10 times and made sure that none of the modified tests are failing. I focused onTestDelegatedScanning,TestPodandTestContainerInstances. (TestDelegatedScanning requires an OCP cluster).gke-nongroovy-e2e-testsbut fewer times.AI assistance:
~40% of code was AI-generated (boilerplate K8s API calls, logging statements, error handling patterns). All core logic, function signatures, architectural decisions, and test updates were designed and validated manually. AI-generated code was reviewed line-by-line and modified where needed for correctness and consistency with existing patterns.