Skip to content

chore(build): Upgrade to UBI9#17874

Closed
davdhacs wants to merge 28 commits intomasterfrom
rox-14475-ubi9-update
Closed

chore(build): Upgrade to UBI9#17874
davdhacs wants to merge 28 commits intomasterfrom
rox-14475-ubi9-update

Conversation

@davdhacs
Copy link
Contributor

@davdhacs davdhacs commented Nov 19, 2025

re-run of UBI8->9 testing in #15178 on latest master (post 4.9).

rox-ci-image update from PR stackrox/rox-ci-image#234

BradLugo and others added 13 commits August 13, 2025 09:26
Missed the actually runtime container images in the last commit (only
changed the builder images).
Should resolve the permission issues we're seeing in CI. Regarding why
we do all the certificate business in the first place, I think it's
because we want to update the trusted certificates in the container with
any stackrox-generate certs+any relevant OCP certs. However, there may
be a better way to go about it. Needs further investigation and possibly
roping in other teams (e.g., Install team).
Update `pg_rhel_major` to 9 for the rest of the download scripts.
Don't globber files when restoring. Should resolve the sensor errors. I
suspect this approach won't be the one we ship - just trying to get
everything to work for now.
Turns out we weren't running the `update-ca-trust` command from the last
fixup since we were restoring `/etc/pki/ca-trust/extracted` that was
saved during the container build process. These changes should implement
the original fixup correctly and allow the operator-related CA tests to
pass.
@openshift-ci
Copy link

openshift-ci bot commented Nov 19, 2025

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@davdhacs
Copy link
Contributor Author

/test ?

@openshift-ci
Copy link

openshift-ci bot commented Nov 19, 2025

@davdhacs: The following commands are available to trigger required jobs:

/test gke-nongroovy-e2e-tests
/test gke-ui-e2e-tests

The following commands are available to trigger optional jobs:

/test aks-qa-e2e-tests
/test aro-qa-e2e-tests
/test eks-qa-e2e-tests
/test gke-external-pg-17-qa-e2e-tests
/test gke-latest-nongroovy-e2e-tests
/test gke-latest-operator-e2e-tests
/test gke-latest-qa-e2e-tests
/test gke-latest-ui-e2e-tests
/test gke-nongroovy-compatibility-tests
/test gke-oldest-nongroovy-e2e-tests
/test gke-oldest-operator-e2e-tests
/test gke-oldest-qa-e2e-tests
/test gke-oldest-ui-e2e-tests
/test gke-operator-e2e-tests
/test gke-qa-e2e-tests
/test gke-race-condition-qa-e2e-tests
/test gke-scale-tests
/test gke-scanner-v4-install-tests
/test gke-sensor-integration-tests
/test gke-upgrade-tests
/test gke-version-compatibility-tests
/test ibmcloudz-4-14-qa-e2e-tests
/test ibmcloudz-4-15-qa-e2e-tests
/test ibmcloudz-4-16-qa-e2e-tests
/test ibmcloudz-4-17-qa-e2e-tests
/test ocp-4-12-compliance-e2e-tests
/test ocp-4-12-nongroovy-e2e-tests
/test ocp-4-12-operator-e2e-tests
/test ocp-4-12-qa-e2e-tests
/test ocp-4-12-scanner-v4-install-tests
/test ocp-4-12-sensor-integration-tests
/test ocp-4-12-ui-e2e-tests
/test ocp-4-20-compliance-e2e-tests
/test ocp-4-20-crun-qa-e2e-tests
/test ocp-4-20-fips-qa-e2e-tests
/test ocp-4-20-nongroovy-e2e-tests
/test ocp-4-20-operator-e2e-tests
/test ocp-4-20-qa-e2e-tests
/test ocp-4-20-scanner-v4-install-tests
/test ocp-4-20-sensor-integration-tests
/test ocp-4-20-ui-e2e-tests
/test ocp-dev-preview-compliance-e2e-tests
/test ocp-dev-preview-fips-qa-e2e-tests
/test ocp-dev-preview-nongroovy-e2e-tests
/test ocp-dev-preview-operator-e2e-tests
/test ocp-dev-preview-qa-e2e-tests
/test ocp-dev-preview-scanner-v4-install-tests
/test ocp-dev-preview-sensor-integration-tests
/test ocp-dev-preview-ui-e2e-tests
/test ocp-next-candidate-compliance-e2e-tests
/test ocp-next-candidate-fips-qa-e2e-tests
/test ocp-next-candidate-nongroovy-e2e-tests
/test ocp-next-candidate-operator-e2e-tests
/test ocp-next-candidate-qa-e2e-tests
/test ocp-next-candidate-scanner-v4-install-tests
/test ocp-next-candidate-sensor-integration-tests
/test ocp-next-candidate-ui-e2e-tests
/test ocp-stable-scanner-v4-install-compliance-e2e-tests
/test ocp-stable-scanner-v4-install-nongroovy-e2e-tests
/test ocp-stable-scanner-v4-install-operator-e2e-tests
/test ocp-stable-scanner-v4-install-qa-e2e-tests
/test ocp-stable-scanner-v4-install-scanner-v4-install-tests
/test ocp-stable-scanner-v4-install-sensor-integration-tests
/test ocp-stable-scanner-v4-install-ui-e2e-tests
/test osd-aws-qa-e2e-tests
/test osd-gcp-qa-e2e-tests
/test powervs-4-14-qa-corebpf-e2e-tests
/test powervs-4-15-qa-corebpf-e2e-tests
/test powervs-4-16-qa-corebpf-e2e-tests
/test powervs-4-17-qa-corebpf-e2e-tests
/test powervs-4-18-qa-corebpf-e2e-tests
/test powervs-4-19-qa-corebpf-e2e-tests
/test rosa-fips-qa-e2e-tests
/test rosa-hcp-qa-e2e-tests
/test rosa-qa-e2e-tests

Use /test all to run the following jobs that were automatically triggered:

pull-ci-stackrox-stackrox-master-gke-nongroovy-e2e-tests
pull-ci-stackrox-stackrox-master-gke-operator-e2e-tests
pull-ci-stackrox-stackrox-master-gke-qa-e2e-tests
pull-ci-stackrox-stackrox-master-gke-scanner-v4-install-tests
pull-ci-stackrox-stackrox-master-gke-ui-e2e-tests
pull-ci-stackrox-stackrox-master-gke-upgrade-tests
pull-ci-stackrox-stackrox-master-ocp-4-12-nongroovy-e2e-tests
pull-ci-stackrox-stackrox-master-ocp-4-12-operator-e2e-tests
pull-ci-stackrox-stackrox-master-ocp-4-12-qa-e2e-tests
pull-ci-stackrox-stackrox-master-ocp-4-12-scanner-v4-install-tests
pull-ci-stackrox-stackrox-master-ocp-4-20-nongroovy-e2e-tests
pull-ci-stackrox-stackrox-master-ocp-4-20-operator-e2e-tests
pull-ci-stackrox-stackrox-master-ocp-4-20-qa-e2e-tests
pull-ci-stackrox-stackrox-master-ocp-4-20-scanner-v4-install-tests
Details

In response to this:

/test ?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Copy link
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey there - I've reviewed your changes - here's some feedback:

  • Consider consolidating the repetitive container image version updates across workflows (e.g. via a variable or reusable workflow) to simplify future base image upgrades.
  • Double-check that the updated save-dir-contents paths (pointing to /etc/pki/ca-trust/source) exist and work correctly under UBI9.
  • Clean up or fix the su-exec block in central-entrypoint.sh: either remove the obsolete TODO or make the conditional POSIX-compliant and ensure su-exec is installed.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- Consider consolidating the repetitive container image version updates across workflows (e.g. via a variable or reusable workflow) to simplify future base image upgrades.
- Double-check that the updated save-dir-contents paths (pointing to /etc/pki/ca-trust/source) exist and work correctly under UBI9.
- Clean up or fix the su-exec block in central-entrypoint.sh: either remove the obsolete TODO or make the conditional POSIX-compliant and ensure su-exec is installed.

## Individual Comments

### Comment 1
<location> `image/rhel/static-bin/central-entrypoint.sh:21-25` </location>
<code_context>
         echo >&2 "Warning: failed to change permissions of one or more directories. Startup may fail."
      fi

+    # TODO(DO NOT MERGE): exec: su-exec: not found
+    #  Is this entire conditional still relevant? If so, perhaps we can make it
+    #  better.
+    #  Doesn't seem to work in UBI8 either.
+    #  Also, the conditional itself isn't POSIX-compliant (should be `[ "$(id -u)" = "0" ]`)
      exec su-exec 4000:4000 "$0" "$@"
 fi
</code_context>

<issue_to_address>
**issue:** The TODO block should be removed before merging.

TODOs marked 'DO NOT MERGE' in production code can cause confusion and risk incomplete changes being deployed. Please resolve or remove this block before merging.
</issue_to_address>

### Comment 2
<location> `image/rhel/static-bin/central-entrypoint.sh:26` </location>
<code_context>
+    #  better.
+    #  Doesn't seem to work in UBI8 either.
+    #  Also, the conditional itself isn't POSIX-compliant (should be `[ "$(id -u)" = "0" ]`)
      exec su-exec 4000:4000 "$0" "$@"
 fi

</code_context>

<issue_to_address>
**issue (bug_risk):** su-exec may not be available in UBI9 images.

Consider replacing su-exec with a more widely available alternative like 'gosu' or 'runuser' to prevent runtime failures from missing binaries.
</issue_to_address>

### Comment 3
<location> `scanner/image/scanner/Dockerfile:48` </location>
<code_context>
     # by the script `save-dir-contents` during the image build. The directory
     # contents are then restored by the script `restore-all-dir-contents`
     # during the container start.
-    chown -R 65534:65534 /etc/pki/ca-trust /etc/ssl && save-dir-contents /etc/pki/ca-trust /etc/ssl
+    chown -R 65534:65534 /etc/pki/ca-trust /etc/ssl && save-dir-contents /etc/pki/ca-trust/source /etc/ssl

 # This is equivalent to nobody:nobody.
</code_context>

<issue_to_address>
**issue (bug_risk):** Saving only /etc/pki/ca-trust/source may omit important trust anchors.

Verify whether the application requires the entire /etc/pki/ca-trust directory or just the 'source' subdirectory. Update the logic if additional files are needed.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

# by the script `save-dir-contents` during the image build. The directory
# contents are then restored by the script `restore-all-dir-contents`
# during the container start.
chown -R 65534:65534 /etc/pki/ca-trust /etc/ssl && save-dir-contents /etc/pki/ca-trust /etc/ssl
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (bug_risk): Saving only /etc/pki/ca-trust/source may omit important trust anchors.

Verify whether the application requires the entire /etc/pki/ca-trust directory or just the 'source' subdirectory. Update the logic if additional files are needed.

Copy link
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New security issues found

@rhacs-bot
Copy link
Contributor

rhacs-bot commented Nov 19, 2025

Images are ready for the commit at 095fe95.

To use with deploy scripts, first export MAIN_IMAGE_TAG=4.10.x-644-g095fe956b2.

@codecov
Copy link

codecov bot commented Nov 19, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 49.15%. Comparing base (88cde4d) to head (095fe95).
⚠️ Report is 74 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #17874      +/-   ##
==========================================
- Coverage   49.15%   49.15%   -0.01%     
==========================================
  Files        2739     2739              
  Lines      201205   201205              
==========================================
- Hits        98903    98901       -2     
- Misses      94599    94600       +1     
- Partials     7703     7704       +1     
Flag Coverage Δ
go-unit-tests 49.15% <ø> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New security issues found

@davdhacs
Copy link
Contributor Author

/test gke-nongroovy-e2e-tests gke-qa-e2e-tests

@davdhacs davdhacs changed the title chore(base image): Upgrade to UBI9 chore(build): Upgrade to UBI9 Nov 20, 2025
@davdhacs
Copy link
Contributor Author

/test gke-upgrade-tests

Copy link
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New security issues found

Copy link
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New security issues found

@davdhacs
Copy link
Contributor Author

/test aks-qa-e2e-tests gke-qa-e2e-tests

@davdhacs davdhacs added the ci-all-qa-tests Tells CI to run all API tests (not just BAT). label Nov 26, 2025
@davdhacs
Copy link
Contributor Author

/retest

@gitguardian
Copy link

gitguardian bot commented Dec 1, 2025

⚠️ GitGuardian has uncovered 1 secret following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secret in your pull request
GitGuardian id GitGuardian status Secret Commit Filename
20170519 Triggered JSON Web Token dc5e0ed central/auth/m2m/id_token_test.go View secret
🛠 Guidelines to remediate hardcoded secrets
  1. Understand the implications of revoking this secret by investigating where it is used in your code.
  2. Replace and store your secret safely. Learn here the best practices.
  3. Revoke and rotate this secret.
  4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider


🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

@davdhacs
Copy link
Contributor Author

/test ocp-4-20-compliance-e2e-tests ocp-4-20-operator-e2e-tests ocp-4-20-qa-e2e-tests ocp-next-candidate-fips-qa-e2e-tests ocp-next-candidate-qa-e2e-tests ocp-next-candidate-operator-e2e-tests ocp-next-candidate-compliance-e2e-tests

@davdhacs
Copy link
Contributor Author

/test ocp-4-20-compliance-e2e-tests ocp-4-20-operator-e2e-tests ocp-4-20-qa-e2e-tests ocp-next-candidate-fips-qa-e2e-tests ocp-next-candidate-qa-e2e-tests ocp-next-candidate-operator-e2e-tests ocp-next-candidate-compliance-e2e-tests

@openshift-ci
Copy link

openshift-ci bot commented Dec 11, 2025

@davdhacs: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/gke-operator-e2e-tests 9cd8aab link false /test gke-operator-e2e-tests
ci/prow/gke-scanner-v4-install-tests 9cd8aab link false /test gke-scanner-v4-install-tests
ci/prow/ocp-4-20-scanner-v4-install-tests 9cd8aab link false /test ocp-4-20-scanner-v4-install-tests
ci/prow/ocp-4-12-scanner-v4-install-tests 9cd8aab link false /test ocp-4-12-scanner-v4-install-tests
ci/prow/ocp-4-12-operator-e2e-tests 9cd8aab link false /test ocp-4-12-operator-e2e-tests
ci/prow/ocp-4-20-operator-e2e-tests 3e77142 link false /test ocp-4-20-operator-e2e-tests
ci/prow/ocp-next-candidate-operator-e2e-tests 3e77142 link false /test ocp-next-candidate-operator-e2e-tests

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@davdhacs
Copy link
Contributor Author

davdhacs commented Jan 5, 2026

re-doing on master after rox-ci-image update to 0.5.1: #18364

@davdhacs davdhacs closed this Jan 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants