Skip to content

ROX-31023: arm64 support for operator images#16915

Merged
mclasmeier merged 4 commits intomasterfrom
mc/operator-arm64
Oct 1, 2025
Merged

ROX-31023: arm64 support for operator images#16915
mclasmeier merged 4 commits intomasterfrom
mc/operator-arm64

Conversation

@mclasmeier
Copy link
Contributor

@mclasmeier mclasmeier commented Sep 18, 2025

Description

The GitHub Actions built operator images only work on amd64 currently.
Adding arm64 support would enable us to quickly test CI-built operator images locally also on arm64 machines.

User-facing documentation

No user-facing changes, no changes to production code, no tests.

How I validated my change

Set TAG.

❯ skopeo inspect --raw docker://quay.io/rhacs-eng/stackrox-operator:$TAG  
{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
   "manifests": [
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 741,
         "digest": "sha256:0f8efc05128ef46ca3f4796a5b8eacf0a99ac4cb924cca3a9c1bcd4f932b77b4",
         "platform": {
            "architecture": "amd64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 740,
         "digest": "sha256:347a37201c138dba8e6d1f26c270e2bd391bb72aa224c1730b929e18f67e9644",
         "platform": {
            "architecture": "arm64",
            "os": "linux"
         }
      }
   ]
}
❯ podman run --platform=linux/arm64 --rm quay.io/rhacs-eng/stackrox-operator:$TAG --help
Usage of /usr/local/bin/stackrox-operator:
  -alsologtostderr
    	log to standard error as well as files
  -enable-http2
    	If HTTP/2 should be enabled for the metrics server.
  -health-probe-bind-address string
    	The address the probe endpoint binds to. (default ":8081")
  -kubeconfig string
    	Paths to a kubeconfig. Only required if out-of-cluster.
  -leader-elect
    	Enable leader election for controller manager. Enabling this will ensure there is only one active controller manager.
  -log_backtrace_at value
    	when logging hits line file:N, emit a stack trace
  -log_dir string
    	If non-empty, write log files in this directory
  -log_link string
    	If non-empty, add symbolic links in this directory to the log files
  -logbuflevel int
    	Buffer log messages logged at this level or lower (-1 means don't buffer; 0 means buffer INFO only; ...). Has limited applicability on non-prod platforms.
  -logtostderr
    	log to standard error instead of files
  -metrics-bind-address string
    	The address the metric endpoint binds to. (default "0.0.0.0:8443")
  -stderrthreshold value
    	logs at or above this threshold go to stderr (default 2)
  -v value
    	log level for V logs
  -vmodule value
    	comma-separated list of pattern=N settings for file-filtered logging
  -zap-devel
    	Development Mode defaults(encoder=consoleEncoder,logLevel=Debug,stackTraceLevel=Warn). Production Mode defaults(encoder=jsonEncoder,logLevel=Info,stackTraceLevel=Error) (default true)
  -zap-encoder value
    	Zap log encoding (one of 'json' or 'console')
  -zap-log-level value
    	Zap Level to configure the verbosity of logging. Can be one of 'debug', 'info', 'error', 'panic'or any integer value > 0 which corresponds to custom debug levels of increasing verbosity
  -zap-stacktrace-level value
    	Zap Level at and above which stacktraces are captured (one of 'info', 'error', 'panic').
  -zap-time-encoding value
    	Zap time encoding (one of 'epoch', 'millis', 'nano', 'iso8601', 'rfc3339' or 'rfc3339nano'). Defaults to 'epoch'.
❯ podman create --name stackrox-operator-tmp quay.io/rhacs-eng/stackrox-operator:$TAG
c0d75eecebf6b1f75afcafd0591368a4510a6ea9f2848112bdb434ffe6aa9a2e
❯ docker cp stackrox-operator-tmp:/usr/local/bin/stackrox-operator /tmp/stackrox-operator
Successfully copied 104MB to /tmp/stackrox-operator
❯ file /tmp/stackrox-operator
/tmp/stackrox-operator: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, BuildID[sha1]=3e186f28a7da0365bf67245e0075c90f8f84b249, stripped

Alternatively: can be deployed with roxie.

@openshift-ci
Copy link

openshift-ci bot commented Sep 18, 2025

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

Copy link
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey there - I've reviewed your changes - here's some feedback:

Blocking issues:

  • An action sourced from a third-party repository on GitHub is not pinned to a full length commit SHA. Pinning an action to a full length commit SHA is currently the only way to use an action as an immutable release. Pinning to a particular SHA helps mitigate the risk of a bad actor adding a backdoor to the action's repository, as they would need to generate a SHA-1 collision for a valid Git object payload. (link)
  • An action sourced from a third-party repository on GitHub is not pinned to a full length commit SHA. Pinning an action to a full length commit SHA is currently the only way to use an action as an immutable release. Pinning to a particular SHA helps mitigate the risk of a bad actor adding a backdoor to the action's repository, as they would need to generate a SHA-1 collision for a valid Git object payload. (link)

General comments:

  • Consider extracting the QEMU and Buildx setup into a shared step or composite action so you don’t need to repeat it across arches and jobs.
  • The new push-operator-manifests job has a lot of overlap with your existing multi-arch manifest push logic—consider refactoring common parts into a reusable workflow or action.
  • Since you now require TARGET_ARCH for local builds via GOARCH, add a default (e.g. defaulting to amd64) so 'make docker-build' still works outside CI without extra env vars.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- Consider extracting the QEMU and Buildx setup into a shared step or composite action so you don’t need to repeat it across arches and jobs.
- The new push-operator-manifests job has a lot of overlap with your existing multi-arch manifest push logic—consider refactoring common parts into a reusable workflow or action.
- Since you now require TARGET_ARCH for local builds via GOARCH, add a default (e.g. defaulting to amd64) so 'make docker-build' still works outside CI without extra env vars.

## Individual Comments

### Comment 1
<location> `operator/Makefile:337` </location>
<code_context>
 docker-build: build/Dockerfile.gen test smuggled-status-sh ## Build docker image with the operator.
 	DOCKER_BUILDKIT=1 BUILDKIT_PROGRESS=plain docker build \
 		-t ${IMG} \
+		--build-arg TARGET_ARCH=$(GOARCH) \
 		-f $< \
 		..
</code_context>

<issue_to_address>
**suggestion (bug_risk):** TARGET_ARCH build argument is passed from GOARCH.

Make sure GOARCH is always set or provide a default to prevent build failures.
</issue_to_address>

### Comment 2
<location> `.github/workflows/build.yaml:603` </location>
<code_context>
        uses: docker/setup-qemu-action@v3
</code_context>

<issue_to_address>
**security (yaml.github-actions.security.third-party-action-not-pinned-to-commit-sha):** An action sourced from a third-party repository on GitHub is not pinned to a full length commit SHA. Pinning an action to a full length commit SHA is currently the only way to use an action as an immutable release. Pinning to a particular SHA helps mitigate the risk of a bad actor adding a backdoor to the action's repository, as they would need to generate a SHA-1 collision for a valid Git object payload.

*Source: opengrep*
</issue_to_address>

### Comment 3
<location> `.github/workflows/build.yaml:606` </location>
<code_context>
        uses: docker/setup-buildx-action@v3
</code_context>

<issue_to_address>
**security (yaml.github-actions.security.third-party-action-not-pinned-to-commit-sha):** An action sourced from a third-party repository on GitHub is not pinned to a full length commit SHA. Pinning an action to a full length commit SHA is currently the only way to use an action as an immutable release. Pinning to a particular SHA helps mitigate the risk of a bad actor adding a backdoor to the action's repository, as they would need to generate a SHA-1 collision for a valid Git object payload.

*Source: opengrep*
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@rhacs-bot
Copy link
Contributor

rhacs-bot commented Sep 18, 2025

Images are ready for the commit at bff2128.

To use with deploy scripts, first export MAIN_IMAGE_TAG=4.9.x-927-gbff21281d8.

@codecov
Copy link

codecov bot commented Sep 18, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 48.80%. Comparing base (91010e2) to head (bff2128).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master   #16915   +/-   ##
=======================================
  Coverage   48.80%   48.80%           
=======================================
  Files        2707     2707           
  Lines      202201   202201           
=======================================
+ Hits        98679    98683    +4     
+ Misses      95752    95747    -5     
- Partials     7770     7771    +1     
Flag Coverage Δ
go-unit-tests 48.80% <ø> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@mclasmeier mclasmeier changed the title Mc/operator arm64 arm64 support for operator images Sep 19, 2025
@mclasmeier mclasmeier marked this pull request as ready for review September 19, 2025 20:36
@mclasmeier mclasmeier requested review from a team as code owners September 19, 2025 20:36
@mclasmeier mclasmeier requested review from porridge and removed request for a team September 19, 2025 20:36
sourcery-ai[bot]
sourcery-ai bot previously requested changes Sep 19, 2025
Copy link
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey there - I've reviewed your changes and they look great!

Blocking issues:

  • An action sourced from a third-party repository on GitHub is not pinned to a full length commit SHA. Pinning an action to a full length commit SHA is currently the only way to use an action as an immutable release. Pinning to a particular SHA helps mitigate the risk of a bad actor adding a backdoor to the action's repository, as they would need to generate a SHA-1 collision for a valid Git object payload. (link)
  • An action sourced from a third-party repository on GitHub is not pinned to a full length commit SHA. Pinning an action to a full length commit SHA is currently the only way to use an action as an immutable release. Pinning to a particular SHA helps mitigate the risk of a bad actor adding a backdoor to the action's repository, as they would need to generate a SHA-1 collision for a valid Git object payload. (link)
Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location> `.github/workflows/build.yaml:654` </location>
<code_context>
+            if [[ "${{ github.event_name }}" == "push" && "${{ github.ref_name }}" == "master" ]]; then
+              push_context="merge-to-master"
+            fi
+            push_operator_image_set "$push_context" "${{ env.ROX_PRODUCT_BRANDING }}" "${{ matrix.arch }}"
+
+      - name: Push bundle image
</code_context>

<issue_to_address>
**suggestion:** Custom push logic replaces previous make-based push for operator images.

Please verify that error handling and logging in the custom push function are sufficient, particularly for multi-arch pushes, and that any failures are clearly reported in CI.

Suggested implementation:

```
            echo "Starting operator image push for context: $push_context, branding: ${{ env.ROX_PRODUCT_BRANDING }}, arch: ${{ matrix.arch }}"
            set -o pipefail
            if ! push_operator_image_set "$push_context" "${{ env.ROX_PRODUCT_BRANDING }}" "${{ matrix.arch }}"; then
              echo "ERROR: push_operator_image_set failed for context: $push_context, branding: ${{ env.ROX_PRODUCT_BRANDING }}, arch: ${{ matrix.arch }}" >&2
              exit 1
            fi
            echo "Operator image push completed successfully for context: $push_context, branding: ${{ env.ROX_PRODUCT_BRANDING }}, arch: ${{ matrix.arch }}"

```

- Ensure that the `push_operator_image_set` function in `./scripts/ci/lib.sh` itself logs errors and outputs relevant information for multi-arch pushes.
- If the function does not already print detailed error messages, update it to do so.
- If the function does not return a non-zero exit code on failure, update it to ensure CI can detect failures.
</issue_to_address>

### Comment 2
<location> `scripts/ci/lib.sh:324` </location>
<code_context>
     fi
 }

+push_operator_image_set() {
+    info "Pushing stackrox-operator image"
+
</code_context>

<issue_to_address>
**suggestion:** Function expects exactly three arguments; consider validating input types.

Currently, only the number of arguments is checked. Please add validation for argument values to ensure they are not empty or invalid.
</issue_to_address>

### Comment 3
<location> `.github/workflows/build.yaml:603` </location>
<code_context>
        uses: docker/setup-qemu-action@v3
</code_context>

<issue_to_address>
**security (yaml.github-actions.security.third-party-action-not-pinned-to-commit-sha):** An action sourced from a third-party repository on GitHub is not pinned to a full length commit SHA. Pinning an action to a full length commit SHA is currently the only way to use an action as an immutable release. Pinning to a particular SHA helps mitigate the risk of a bad actor adding a backdoor to the action's repository, as they would need to generate a SHA-1 collision for a valid Git object payload.

*Source: opengrep*
</issue_to_address>

### Comment 4
<location> `.github/workflows/build.yaml:606` </location>
<code_context>
        uses: docker/setup-buildx-action@v3
</code_context>

<issue_to_address>
**security (yaml.github-actions.security.third-party-action-not-pinned-to-commit-sha):** An action sourced from a third-party repository on GitHub is not pinned to a full length commit SHA. Pinning an action to a full length commit SHA is currently the only way to use an action as an immutable release. Pinning to a particular SHA helps mitigate the risk of a bad actor adding a backdoor to the action's repository, as they would need to generate a SHA-1 collision for a valid Git object payload.

*Source: opengrep*
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Copy link
Contributor

@porridge porridge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please replace the placeholders from PR description.

@mclasmeier
Copy link
Contributor Author

@porridge I don't understand yet where this zombie jobs is coming from

image

@porridge
Copy link
Contributor

The stackrox/stackrox repo branch protection configuration specifies that a GH check of that name must pass for the change to be mergeable. I guess this needs to be relaxed for a moment while the name changes.

@mclasmeier
Copy link
Contributor Author

The stackrox/stackrox repo branch protection configuration specifies that a GH check of that name must pass for the change to be mergeable. I guess this needs to be relaxed for a moment while the name changes.

Will contact automation team once this is PR is code-wise ready.

@mclasmeier mclasmeier requested a review from porridge September 25, 2025 11:46
@mclasmeier mclasmeier changed the title arm64 support for operator images ROX-31023: arm64 support for operator images Sep 25, 2025
@mclasmeier mclasmeier force-pushed the mc/operator-arm64 branch 3 times, most recently from a89ebe7 to e5aaeb0 Compare September 26, 2025 08:50
@mclasmeier
Copy link
Contributor Author

/ai-review

@mclasmeier mclasmeier dismissed sourcery-ai[bot]’s stale review September 26, 2025 08:52

For consistency reasons with the existing code this PR continues using tags in the image refs.

@mclasmeier mclasmeier requested review from a team and tommartensen September 26, 2025 12:33
@mclasmeier mclasmeier changed the title ROX-31023: arm64 support for operator images ROX-31023: arm64 support for operator images (don't merge) Sep 28, 2025
@mclasmeier mclasmeier removed request for a team and tommartensen September 28, 2025 20:30
@mclasmeier mclasmeier changed the title ROX-31023: arm64 support for operator images (don't merge) ROX-31023: arm64 support for operator images Sep 28, 2025
@mclasmeier mclasmeier requested review from a team and tommartensen September 28, 2025 20:38
Copy link
Contributor

@porridge porridge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a few nitpicks inline.

@openshift-ci
Copy link

openshift-ci bot commented Sep 30, 2025

@mclasmeier: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/gke-operator-e2e-tests bff2128 link false /test gke-operator-e2e-tests
ci/prow/ocp-4-19-operator-e2e-tests bff2128 link false /test ocp-4-19-operator-e2e-tests
ci/prow/gke-qa-e2e-tests bff2128 link false /test gke-qa-e2e-tests

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@mclasmeier mclasmeier merged commit 85aa9d2 into master Oct 1, 2025
97 of 101 checks passed
@mclasmeier mclasmeier deleted the mc/operator-arm64 branch October 1, 2025 07:44
vladbologa added a commit that referenced this pull request Oct 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants