Skip to content

ROX-26601: Use downstream image repos in operator bundle#13154

Merged
mclasmeier merged 10 commits intomasterfrom
mc/konflux-redhat.io-registries
Nov 12, 2024
Merged

ROX-26601: Use downstream image repos in operator bundle#13154
mclasmeier merged 10 commits intomasterfrom
mc/konflux-redhat.io-registries

Conversation

@mclasmeier
Copy link
Contributor

@mclasmeier mclasmeier commented Oct 29, 2024

Description

This PR modifies the image references in the Konflux-built operator-bundle to point to the expected downstream image registries (registry.redhat.io/advanced-cluster-security/...) instead of quay.io/rhacs-eng/....

For convenience during development an ImageContentSourcePolicy has been provided which allows testing built operator-bundles prior to a release.

User-facing documentation

  • CHANGELOG update is not needed
  • documentation PR is not needed

Testing and quality

  • the change is production ready
  • CI results are inspected

Automated testing

We have no automated testing for downstream bundles artifacts.

How I validated my change

We can easily deploy standalone bundles using a recent (unreleased) version of operator-sdk.

  1. Update operator-sdk
    • Apply this commit
    • (cd operator; rm .gotools/bin/operator-sdk; make operator-sdk)
  2. Prepare OpenShift cluster.
    • Connect to cluster.
    • NS=bundle-test
    • kubectl create namespace bundle-test
    • ./deploy/common/pull-secret.sh quay-ips quay.io | kubectl -n $NS apply -f -
  3. Deploy operator-bundle built by this PR:
    • cd operator/
    • BUNDLE_TAG=v4.7.0-60-g8d142cef9f
    • `make which-operator-sdk` run bundle quay.io/rhacs-eng/stackrox-operator-bundle:$BUNDLE_TAG --pull-secret-name quay-ips --service-account default --namespace $NS

Observe that it fails to deploy, because the images refer to the downstream registry where the images do not exist:

INFO[0021] Creating a File-Based Catalog of the bundle "quay.io/rhacs-eng/stackrox-operator-bundle:v4.7.0-60-g667b0e2c4c-fast" 
INFO[0027] Generated a valid File-Based Catalog         
INFO[0031] Created registry pod: o-rhacs-eng-stackrox-operator-bundle-v4-7-0-60-g667b0e2c4c-fast 
INFO[0032] Created CatalogSource: rhacs-operator-catalog 
INFO[0032] OperatorGroup "operator-sdk-og" created      
INFO[0032] Created Subscription: rhacs-operator-v4-7-0-60-g667b0e2c4c-fast-sub 
INFO[0051] Approved InstallPlan install-rbfkb for the Subscription: rhacs-operator-v4-7-0-60-g667b0e2c4c-fast-sub 
INFO[0051] Waiting for ClusterServiceVersion "bundle-test/rhacs-operator.v4.7.0-60-g667b0e2c4c-fast" to reach 'Succeeded' phase 
INFO[0053]   Found ClusterServiceVersion "bundle-test/rhacs-operator.v4.7.0-60-g667b0e2c4c-fast" phase: Pending 
INFO[0054]   Found ClusterServiceVersion "bundle-test/rhacs-operator.v4.7.0-60-g667b0e2c4c-fast" phase: Installing
FATA[0120] Failed to run bundle: error waiting for CSV to install: deployment rhacs-operator-controller-manager has error: client rate limiter Wait returned an error: context deadline exceeded

Describing the rhacs-operator-controller-manager pod:

 Warning  Failed          2s (x2 over 17s)  kubelet            Failed to pull image "registry.redhat.io/advanced-cluster-security/rhacs-rhel8-operator@sha256:e237979e3264e70c02acf3a84b8f49c7c58056978a8d62abe620130015b0decc": reading manifest sha256:e237979e3264e70c02acf3a84b8f49c7c58056978a8d62abe620130015b0decc in registry.redhat.io/advanced-cluster-security/rhacs-rhel8-operator: manifest unknown

as expected.

Second test:

  1. Delete previous bundle deployment:
    • cd operator/
    • `make which-operator-sdk` cleanup --delete-all rhacs-operator
  2. Apply the following ImageContentSourcePolicy the the cluster:
    apiVersion: operator.openshift.io/v1alpha1
    kind: ImageContentSourcePolicy
    metadata:
      name: acs-redhat-io
    spec:
      repositoryDigestMirrors:
      - source: registry.redhat.io/advanced-cluster-security/rhacs-operator-bundle
        mirrors:
        - quay.io/rhacs-eng/stackrox-operator-bundle
      - source: registry.redhat.io/advanced-cluster-security/rhacs-rhel8-operator
        mirrors:
        - quay.io/rhacs-eng/stackrox-operator
      - source: registry.redhat.io/advanced-cluster-security/rhacs-main-rhel8
        mirrors:
        - quay.io/rhacs-eng/main
      - source: registry.redhat.io/advanced-cluster-security/rhacs-scanner-rhel8
        mirrors:
        - quay.io/rhacs-eng/scanner
      - source: registry.redhat.io/advanced-cluster-security/rhacs-scanner-slim-rhel8
        mirrors:
        - quay.io/rhacs-eng/scanner-slim
      - source: registry.redhat.io/advanced-cluster-security/rhacs-scanner-db-rhel8
        mirrors:
        - quay.io/rhacs-eng/scanner-db
      - source: registry.redhat.io/advanced-cluster-security/rhacs-scanner-db-slim-rhel8
        mirrors:
        - quay.io/rhacs-eng/scanner-db-slim
      - source: registry.redhat.io/advanced-cluster-security/rhacs-collector-slim-rhel8
        mirrors:
        - quay.io/rhacs-eng/collector-slim
      - source: registry.redhat.io/advanced-cluster-security/rhacs-collector-rhel8
        mirrors:
        - quay.io/rhacs-eng/collector
      - source: registry.redhat.io/advanced-cluster-security/rhacs-roxctl-rhel8
        mirrors:
        - quay.io/rhacs-eng/roxctl
      - source: registry.redhat.io/advanced-cluster-security/rhacs-central-db-rhel8
        mirrors:
        - quay.io/rhacs-eng/central-db
      - source: registry.redhat.io/advanced-cluster-security/rhacs-scanner-v4-db-rhel8
        mirrors:
        - quay.io/rhacs-eng/scanner-v4-db
      - source: registry.redhat.io/advanced-cluster-security/rhacs-scanner-v4-rhel8
        mirrors:
        - quay.io/rhacs-eng/scanner-v4
    
  3. Due to a limitation of ImageContentSourcePolicies we need a workaround:
    • Follow the OpenShift docs to inject valid pull credentials for quay.io/rhacs-eng into the global default image pull secrets.
  4. Re-deploy bundle:
    • `make which-operator-sdk` cleanup --delete-all rhacs-operator
    • `make which-operator-sdk` run bundle quay.io/rhacs-eng/stackrox-operator-bundle:$BUNDLE_TAG --pull-secret-name quay-ips --service-account default --namespace $NS

Observe that it deployed alright with images being fetched from quay.io:

INFO[0023] Creating a File-Based Catalog of the bundle "quay.io/rhacs-eng/stackrox-operator-bundle:v4.7.0-60-g667b0e2c4c-fast" 
INFO[0025] Generated a valid File-Based Catalog         
INFO[0029] Created registry pod: o-rhacs-eng-stackrox-operator-bundle-v4-7-0-60-g667b0e2c4c-fast 
INFO[0029] Created CatalogSource: rhacs-operator-catalog 
INFO[0030] OperatorGroup "operator-sdk-og" created      
INFO[0030] Created Subscription: rhacs-operator-v4-7-0-60-g667b0e2c4c-fast-sub 
INFO[0044] Approved InstallPlan install-nzbkd for the Subscription: rhacs-operator-v4-7-0-60-g667b0e2c4c-fast-sub 
INFO[0044] Waiting for ClusterServiceVersion "bundle-test/rhacs-operator.v4.7.0-60-g667b0e2c4c-fast" to reach 'Succeeded' phase 
INFO[0046]   Found ClusterServiceVersion "bundle-test/rhacs-operator.v4.7.0-60-g667b0e2c4c-fast" phase: Installing 
INFO[0069]   Found ClusterServiceVersion "bundle-test/rhacs-operator.v4.7.0-60-g667b0e2c4c-fast" phase: Succeeded 
INFO[0069] OLM has successfully installed "rhacs-operator.v4.7.0-60-g667b0e2c4c-fast"

Pods:

❯ kc get pods
NAME                                                              READY   STATUS      RESTARTS   AGE
171bd037bc85e2f208d4d295bd4b83276375bad0ad5bc728f84c650a897sqzq   0/1     Completed   0          103s
o-rhacs-eng-stackrox-operator-bundle-v4-7-0-60-g667b0e2c4c-fast   1/1     Running     0          111s
rhacs-operator-controller-manager-7c6ff78ddf-ww29n                1/1     Running     0          90s

Applying a Central CR causes central to come up healthy:

❮ kc get pod -l app.kubernetes.io/part-of=stackrox-central-services
NAME                                  READY   STATUS    RESTARTS   AGE
central-cbbb46958-jshmt               1/1     Running   0          49s
central-db-b5858cf4b-24sk8            1/1     Running   0          16m
config-controller-c6dc9cc4f-vdhwd     1/1     Running   0          16m
scanner-bc7fc468-btwvz                1/1     Running   0          16m
scanner-db-799779969d-b48cw           1/1     Running   0          16m
scanner-v4-db-96f55b846-lgmkh         1/1     Running   0          50s
scanner-v4-indexer-6f8f8dbf96-7jrk9   1/1     Running   0          50s
scanner-v4-indexer-6f8f8dbf96-fbgb8   1/1     Running   0          50s
scanner-v4-indexer-6f8f8dbf96-w7t6v   1/1     Running   0          50s
scanner-v4-matcher-54b8df8db9-j2vck   1/1     Running   0          50s
scanner-v4-matcher-54b8df8db9-kbgk5   1/1     Running   0          50s

After having produced an init-bundle in the shape of a Kubernetes secret and applied it we can proceed with deploying a SecuredCluster.

❯ kc get pod -l app.kubernetes.io/part-of=stackrox-secured-cluster-services
NAME                                 READY   STATUS    RESTARTS   AGE
admission-control-57f6ccc7f4-cp6k7   1/1     Running   0          2m46s
admission-control-57f6ccc7f4-jf8g9   1/1     Running   0          2m46s
admission-control-57f6ccc7f4-wvgmw   1/1     Running   0          2m46s
collector-bl868                      3/3     Running   0          2m46s
collector-k4q8j                      3/3     Running   0          2m47s
collector-mnh9d                      3/3     Running   0          2m47s
collector-tvtkr                      3/3     Running   0          2m47s
collector-xxdxz                      3/3     Running   0          2m46s
sensor-85ffc69b97-tmksf              1/1     Running   0          2m46s

@openshift-ci
Copy link

openshift-ci bot commented Oct 29, 2024

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@rhacs-bot
Copy link
Contributor

rhacs-bot commented Oct 29, 2024

Images are ready for the commit at 36bf4c4.

To use with deploy scripts, first export MAIN_IMAGE_TAG=4.7.x-94-g36bf4c4646.

@codecov
Copy link

codecov bot commented Oct 29, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 48.56%. Comparing base (b80256b) to head (36bf4c4).
Report is 13 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master   #13154   +/-   ##
=======================================
  Coverage   48.55%   48.56%           
=======================================
  Files        2467     2467           
  Lines      177807   177824   +17     
=======================================
+ Hits        86339    86354   +15     
- Misses      84538    84539    +1     
- Partials     6930     6931    +1     
Flag Coverage Δ
go-unit-tests 48.56% <ø> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@mclasmeier mclasmeier force-pushed the mc/konflux-redhat.io-registries branch from 87642af to caf58f9 Compare November 5, 2024 11:21
@mclasmeier mclasmeier marked this pull request as ready for review November 7, 2024 07:36
@mclasmeier mclasmeier requested a review from a team as a code owner November 7, 2024 07:36
@mclasmeier mclasmeier marked this pull request as draft November 7, 2024 11:15
@mclasmeier mclasmeier force-pushed the mc/konflux-redhat.io-registries branch from caf58f9 to e86e1a0 Compare November 7, 2024 11:21
@mclasmeier mclasmeier force-pushed the mc/konflux-redhat.io-registries branch from 8d142ce to 667b0e2 Compare November 8, 2024 10:37
@mclasmeier mclasmeier marked this pull request as ready for review November 8, 2024 12:27
@mclasmeier mclasmeier requested a review from a team as a code owner November 8, 2024 12:27
@mclasmeier mclasmeier requested review from GrimmiMeloni and removed request for a team November 8, 2024 12:27
@mclasmeier mclasmeier requested review from msugakov and porridge and removed request for GrimmiMeloni November 8, 2024 12:27
@mclasmeier mclasmeier added the backport-for-4.6-konflux-release https://redhat-internal.slack.com/archives/C05TS9N0S7L/p1730134914487439 label Nov 11, 2024
@openshift-ci
Copy link

openshift-ci bot commented Nov 11, 2024

@mclasmeier: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/ocp-4-17-operator-e2e-tests 14773a1 link false /test ocp-4-17-operator-e2e-tests
ci/prow/ocp-4-17-qa-e2e-tests 14773a1 link false /test ocp-4-17-qa-e2e-tests
ci/prow/ocp-4-12-operator-e2e-tests 14773a1 link false /test ocp-4-12-operator-e2e-tests
ci/prow/ocp-4-12-qa-e2e-tests 14773a1 link false /test ocp-4-12-qa-e2e-tests
ci/prow/gke-operator-e2e-tests 14773a1 link false /test gke-operator-e2e-tests

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@mclasmeier mclasmeier force-pushed the mc/konflux-redhat.io-registries branch from 14773a1 to 2d7820f Compare November 11, 2024 11:41
@mclasmeier mclasmeier requested a review from msugakov November 11, 2024 11:47
@mclasmeier
Copy link
Contributor Author

PipelineRun for operator-bundle succeeded: https://github.com/stackrox/stackrox/pull/13154/checks?check_run_id=32808387556.

Copy link
Contributor

@msugakov msugakov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the new descriptions.

@mclasmeier mclasmeier merged commit 857e3d2 into master Nov 12, 2024
@mclasmeier mclasmeier deleted the mc/konflux-redhat.io-registries branch November 12, 2024 10:42
aaa5kameric pushed a commit that referenced this pull request Nov 14, 2024
Co-authored-by: Moritz Clasmeier <mclasmeier@redhat.com>
ajheflin pushed a commit that referenced this pull request Jun 24, 2025
Co-authored-by: Moritz Clasmeier <mclasmeier@redhat.com>
msugakov pushed a commit that referenced this pull request Jun 25, 2025
Co-authored-by: Moritz Clasmeier <mclasmeier@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/operator backport-for-4.6-konflux-release https://redhat-internal.slack.com/archives/C05TS9N0S7L/p1730134914487439

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants