Skip to content

ROX-26949: Address EC violations when retagging images#13309

Merged
msugakov merged 43 commits intomasterfrom
misha/try-retag-without-components
Nov 18, 2024
Merged

ROX-26949: Address EC violations when retagging images#13309
msugakov merged 43 commits intomasterfrom
misha/try-retag-without-components

Conversation

@msugakov
Copy link
Contributor

@msugakov msugakov commented Nov 12, 2024

Description

Long intro

This change follows up #13079 to address new EC violations of this kind

✕ [Violation] slsa_build_scripted_build.build_task_image_results_found
  ImageRef: quay.io/rhacs-eng/collector-slim@sha256:01d4a964e831a4b87138c4c585ea773a121a6531dd4569644ceb7febb7f3255c
  Reason: Build task not found

✕ [Violation] tasks.required_tasks_found
  ImageRef: quay.io/rhacs-eng/collector-slim@sha256:01d4a964e831a4b87138c4c585ea773a121a6531dd4569644ceb7febb7f3255c
  Reason: One of "buildah", "buildah-10gb", "buildah-6gb", "buildah-8gb", "buildah-remote", "buildah-oci-ta",
  "buildah-remote-oci-ta" tasks is missing

✕ [Violation] tasks.required_tasks_found
  ImageRef: quay.io/rhacs-eng/collector-slim@sha256:01d4a964e831a4b87138c4c585ea773a121a6531dd4569644ceb7febb7f3255c
  Reason: One of "sast-snyk-check", "sast-snyk-check-oci-ta" tasks is missing

✕ [Violation] tasks.required_tasks_found
  ImageRef: quay.io/rhacs-eng/collector-slim@sha256:01d4a964e831a4b87138c4c585ea773a121a6531dd4569644ceb7febb7f3255c
  Reason: One of "source-build", "source-build-oci-ta" tasks is missing

✕ [Violation] tasks.required_tasks_found
  ImageRef: quay.io/rhacs-eng/collector-slim@sha256:01d4a964e831a4b87138c4c585ea773a121a6531dd4569644ceb7febb7f3255c
  Reason: Required task "clair-scan" is missing

✕ [Violation] tasks.required_tasks_found
  ImageRef: quay.io/rhacs-eng/collector-slim@sha256:01d4a964e831a4b87138c4c585ea773a121a6531dd4569644ceb7febb7f3255c
  Reason: Required task "clamav-scan" is missing

✕ [Violation] tasks.required_tasks_found
  ImageRef: quay.io/rhacs-eng/collector-slim@sha256:01d4a964e831a4b87138c4c585ea773a121a6531dd4569644ceb7febb7f3255c
  Reason: Required task "deprecated-image-check" is missing

✕ [Violation] tasks.required_tasks_found
  ImageRef: quay.io/rhacs-eng/collector-slim@sha256:01d4a964e831a4b87138c4c585ea773a121a6531dd4569644ceb7febb7f3255c
  Reason: Required task "init" is missing

✕ [Violation] tasks.required_tasks_found
  ImageRef: quay.io/rhacs-eng/collector-slim@sha256:01d4a964e831a4b87138c4c585ea773a121a6531dd4569644ceb7febb7f3255c
  Reason: Required task "rpms-signature-scan" is missing

My explanation is that the shape of the retag-pipeline is recorded as one of cosign artifacts during the build. At the later EC check, this pipeline is analyzed and found invalid. We simply cannot make this pipeline valid without completely changing the approach and I don't have an idea how.

As I learned through a number of trials, this recording happens when either or both of the following are present:

  1. On the retag-image task - IMAGE_DIGEST and IMAGE_URL results are provided. It's even possible when these result names are substrings because there's some weird matching logic (I did not check the code of the corresponding controllers though).
  2. On the retag-pipeline - IMAGE_DIGEST and IMAGE_URL results.

The pipeline information overwrites the original one attached during builds in Collector and Scanner repos which passes EC checks, therefore we need to make the original pipeline information not overridden.

Unfortunately, this happens externally to the pipeline where we don't have control. I believe this is done by a Tekton Chains instance deployed in Konflux. https://tekton.dev/docs/chains/slsa-provenance/#image_url--image_digest

The solution I propose here is to just not produce these results in task and pipeline.

The negative consequence is that the pipelines without IMAGE_DIGEST and IMAGE_URL results and without Tekton Chains signing can't update Snapshots (this code looks relevant):

    Snapshot creation status
        Failed to create snapshot. Error: Missing info IMAGE_URL from pipelinerun collector-slim-retagged-on-push-rf2k2

No Snapshot updates means no following EC execution and no need in -retagged Components. Therefore, the retag pipelines become Component-less.

Recap

  1. IMAGE_DIGEST and IMAGE_URL results cause Tekton Chains signing.
  2. Tekton Chains signing in retag pipelines attaches retag pipeline info to image. That fails EC checks.
  3. I found no way to prevent attaching pipeline info other than to stop passing IMAGE_DIGEST and IMAGE_URL.
  4. No IMAGE_DIGEST and IMAGE_URL -> no Tekton Chains signing -> no Snapshot updates -> no need in -retagged Components.

Consequences

See here: https://docs.google.com/document/d/1GDmhs4hHRXoNBXt8Wzu5YyLPPhDFfKMbJmH5deNm_xI/edit?tab=t.0#heading=h.hwb5423ut62k

User-facing documentation

  • CHANGELOG is updated OR update is not needed
  • documentation PR is created and is linked above OR is not needed

Testing and quality

  • the change is production ready: the change is GA or otherwise the functionality is gated by a feature flag
  • CI results are inspected

Automated testing

No change there.

How I validated my change

Verified resulting images with ec cli.
Note that was done for images retagged with modified COLLECTOR_VERSION and SCANNER_VERSION. The reason is that I messed up the original ones during my experiments and they are unrepairably failing EC checks. That'll resolve once COLLECTOR_VERSION and SCANNER_VERSION get updated in stackrox master.

@msugakov msugakov added the konflux-build Run Konflux in PR. Push commit to trigger it. label Nov 12, 2024
@openshift-ci
Copy link

openshift-ci bot commented Nov 12, 2024

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@msugakov msugakov changed the title Try whiteout component and app info for collector-slim retag Try white out component and app info for collector-slim retag Nov 12, 2024
@rhacs-bot
Copy link
Contributor

rhacs-bot commented Nov 12, 2024

Images are ready for the commit at fe73f67.

To use with deploy scripts, first export MAIN_IMAGE_TAG=4.7.x-168-gfe73f67a51.

@codecov
Copy link

codecov bot commented Nov 12, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 48.53%. Comparing base (be035c3) to head (fe73f67).
Report is 8 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #13309      +/-   ##
==========================================
+ Coverage   48.50%   48.53%   +0.03%     
==========================================
  Files        2470     2470              
  Lines      178320   178403      +83     
==========================================
+ Hits        86487    86592     +105     
+ Misses      84900    84878      -22     
  Partials     6933     6933              
Flag Coverage Δ
go-unit-tests 48.53% <ø> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@github-actions github-actions bot added the ci-all-qa-tests Tells CI to run all API tests (not just BAT). label Nov 12, 2024
@msugakov msugakov force-pushed the misha/try-retag-without-components branch from 790cc94 to 955fe15 Compare November 12, 2024 15:39
@msugakov
Copy link
Contributor Author

/retest collector-slim-retag

@openshift-ci

This comment was marked as outdated.

@msugakov msugakov changed the title Try white out component and app info for collector-slim retag ROX-24468: Address EC violations when retagging images Nov 12, 2024
@msugakov msugakov changed the title ROX-24468: Address EC violations when retagging images ROX-26949: Address EC violations when retagging images Nov 13, 2024
@msugakov msugakov mentioned this pull request Nov 13, 2024
9 tasks
@stackrox stackrox deleted a comment from openshift-ci bot Nov 15, 2024
@stackrox stackrox deleted a comment from openshift-ci bot Nov 15, 2024
@msugakov
Copy link
Contributor Author

/retest central-db-on-push

@msugakov
Copy link
Contributor Author

/retest main-on-push

@stackrox stackrox deleted a comment from openshift-ci bot Nov 15, 2024
@stackrox stackrox deleted a comment from openshift-ci bot Nov 15, 2024
@msugakov
Copy link
Contributor Author

msugakov commented Nov 15, 2024

EC check results

tl;dr: all clean. The detected [Violation] slsa_source_correlated.source_code_reference_provided is a red herring, per our conclusion (weak reference, I think we made it in a meeting).

collector-full
KUBECONFIG=$HOME/.kube/appstudio-releng.config ec validate image --image quay.io/rhacs-eng/collector:4.7.x-164-gd9dd2cb811-fast --policy registry-rh-acs --ignore-rekor | grep -F -A3 '[Violation]'
ERRO[0027] Parsing PURL "pkg:golang/../" failed: purl is missing name 
ERRO[0031] new digest: unsupported digest algorithm:     rego=ec.oci.image_manifest
ERRO[0031] new digest: unsupported digest algorithm:     rego=ec.oci.image_manifest
ERRO[0036] new digest: unsupported digest algorithm:     rego=ec.oci.image_manifest
ERRO[0040] Parsing PURL "pkg:golang/../" failed: purl is missing name 
ERRO[0055] new digest: unsupported digest algorithm:     rego=ec.oci.image_manifest
ERRO[0057] new digest: unsupported digest algorithm:     rego=ec.oci.image_manifest
ERRO[0057] Parsing PURL "pkg:golang/../" failed: purl is missing name 
ERRO[0063] Parsing PURL "pkg:golang/../" failed: purl is missing name 
✕ [Violation] slsa_source_correlated.source_code_reference_provided
  ImageRef: quay.io/rhacs-eng/collector@sha256:7d655ae0c6743c5b1cf4f70c913a5f247b811ca30d1b674b3b7e61dc098e268f
  Reason: Expected source code reference was not provided for verification

✕ [Violation] slsa_source_correlated.source_code_reference_provided
  ImageRef: quay.io/rhacs-eng/collector@sha256:75bc921aa069dc5f295866fba8f2ff3b2230b6984874fb6c2e6ca587d1342ef8
  Reason: Expected source code reference was not provided for verification

✕ [Violation] slsa_source_correlated.source_code_reference_provided
  ImageRef: quay.io/rhacs-eng/collector@sha256:412d86d4fd5dc29806bb786f06a5a41f8df18364b552a7abc997c0178015ec10
  Reason: Expected source code reference was not provided for verification

✕ [Violation] slsa_source_correlated.source_code_reference_provided
  ImageRef: quay.io/rhacs-eng/collector@sha256:01d4a964e831a4b87138c4c585ea773a121a6531dd4569644ceb7febb7f3255c
  Reason: Expected source code reference was not provided for verification

✕ [Violation] slsa_source_correlated.source_code_reference_provided
  ImageRef: quay.io/rhacs-eng/collector@sha256:000e0c3a7c070a737724be33b1bd8b6ad71eaf15836c7824d054cfdd23523ca1
  Reason: Expected source code reference was not provided for verification

Error: success criteria not met
collector-slim
KUBECONFIG=$HOME/.kube/appstudio-releng.config ec validate image --image quay.io/rhacs-eng/collector-slim:4.7.x-164-gd9dd2cb811-fast --policy registry-rh-acs --ignore-rekor | grep -F -A3 '[Violation]'
ERRO[0028] Parsing PURL "pkg:golang/../" failed: purl is missing name 
ERRO[0031] new digest: unsupported digest algorithm:     rego=ec.oci.image_manifest
ERRO[0032] Parsing PURL "pkg:golang/../" failed: purl is missing name 
ERRO[0035] Parsing PURL "pkg:golang/../" failed: purl is missing name 
ERRO[0054] Parsing PURL "pkg:golang/../" failed: purl is missing name 
ERRO[0056] new digest: unsupported digest algorithm:     rego=ec.oci.image_manifest
ERRO[0064] new digest: unsupported digest algorithm:     rego=ec.oci.image_manifest
ERRO[0068] new digest: unsupported digest algorithm:     rego=ec.oci.image_manifest
ERRO[0068] new digest: unsupported digest algorithm:     rego=ec.oci.image_manifest
✕ [Violation] slsa_source_correlated.source_code_reference_provided
  ImageRef: quay.io/rhacs-eng/collector-slim@sha256:7d655ae0c6743c5b1cf4f70c913a5f247b811ca30d1b674b3b7e61dc098e268f
  Reason: Expected source code reference was not provided for verification

✕ [Violation] slsa_source_correlated.source_code_reference_provided
  ImageRef: quay.io/rhacs-eng/collector-slim@sha256:75bc921aa069dc5f295866fba8f2ff3b2230b6984874fb6c2e6ca587d1342ef8
  Reason: Expected source code reference was not provided for verification

✕ [Violation] slsa_source_correlated.source_code_reference_provided
  ImageRef: quay.io/rhacs-eng/collector-slim@sha256:412d86d4fd5dc29806bb786f06a5a41f8df18364b552a7abc997c0178015ec10
  Reason: Expected source code reference was not provided for verification

✕ [Violation] slsa_source_correlated.source_code_reference_provided
  ImageRef: quay.io/rhacs-eng/collector-slim@sha256:01d4a964e831a4b87138c4c585ea773a121a6531dd4569644ceb7febb7f3255c
  Reason: Expected source code reference was not provided for verification

✕ [Violation] slsa_source_correlated.source_code_reference_provided
  ImageRef: quay.io/rhacs-eng/collector-slim@sha256:000e0c3a7c070a737724be33b1bd8b6ad71eaf15836c7824d054cfdd23523ca1
  Reason: Expected source code reference was not provided for verification

Error: success criteria not met
scanner
KUBECONFIG=$HOME/.kube/appstudio-releng.config ec validate image --image quay.io/rhacs-eng/scanner:4.7.x-164-gd9dd2cb811-fast --policy registry-rh-acs --ignore-rekor | grep -F -A3 '[Violation]'
Error: success criteria not met
✕ [Violation] slsa_source_correlated.source_code_reference_provided
  ImageRef: quay.io/rhacs-eng/scanner@sha256:e259773bd581457cba25a77b8cda401db4cbe6cd947c119703dd736cab145af7
  Reason: Expected source code reference was not provided for verification

✕ [Violation] slsa_source_correlated.source_code_reference_provided
  ImageRef: quay.io/rhacs-eng/scanner@sha256:dd9b2fa2684359965aa6aaa844de2cf43c9c1f2202fcade689d2fa3fee316d8c
  Reason: Expected source code reference was not provided for verification

✕ [Violation] slsa_source_correlated.source_code_reference_provided
  ImageRef: quay.io/rhacs-eng/scanner@sha256:8fcba8f9214549febb91c7591d2d50f1ccfb23ea769d4e8f4f4ab5a920375bde
  Reason: Expected source code reference was not provided for verification

✕ [Violation] slsa_source_correlated.source_code_reference_provided
  ImageRef: quay.io/rhacs-eng/scanner@sha256:81aa2c2d4b9a1accc014885395048360034b0be8d2f435e6f70bb08a2f7bc71b
  Reason: Expected source code reference was not provided for verification

✕ [Violation] slsa_source_correlated.source_code_reference_provided
  ImageRef: quay.io/rhacs-eng/scanner@sha256:2ba6beed6358fe6383f3ef1e954950ef42dfc6b9af9d8aa89b11ce7bd530c729
  Reason: Expected source code reference was not provided for verification
scanner-db
KUBECONFIG=$HOME/.kube/appstudio-releng.config ec validate image --image quay.io/rhacs-eng/scanner-db:4.7.x-164-gd9dd2cb811-fast --policy registry-rh-acs --ignore-rekor | grep -F -A3 '[Violation]'
✕ [Violation] slsa_source_correlated.source_code_reference_provided
  ImageRef: quay.io/rhacs-eng/scanner-db@sha256:fa2ff32ad363a4274b11e7a302fff5d4ef7b84dc1e65d91c6ec962431fbcd174
  Reason: Expected source code reference was not provided for verification

✕ [Violation] slsa_source_correlated.source_code_reference_provided
  ImageRef: quay.io/rhacs-eng/scanner-db@sha256:8ddc6460f590408883d97e71d75518a3d6c565f5df54967b7a8cb6ae234f2183
  Reason: Expected source code reference was not provided for verification

✕ [Violation] slsa_source_correlated.source_code_reference_provided
  ImageRef: quay.io/rhacs-eng/scanner-db@sha256:88bb7cd876cff7b65c342569d4abddcd2f4ea5fc3cca55d6ee9be3d27e785885
  Reason: Expected source code reference was not provided for verification

✕ [Violation] slsa_source_correlated.source_code_reference_provided
  ImageRef: quay.io/rhacs-eng/scanner-db@sha256:3fec0c9053a3e3a0a088d7675e4ae6fd924a8d10dd6f9a18d0083f6ccaa07dd4
  Reason: Expected source code reference was not provided for verification

✕ [Violation] slsa_source_correlated.source_code_reference_provided
  ImageRef: quay.io/rhacs-eng/scanner-db@sha256:3da01a17beb6906aaaea00d05771ff80d19292eaaa383ec027d5d1dff632f501
  Reason: Expected source code reference was not provided for verification

Error: success criteria not met
scanner-db-slim
KUBECONFIG=$HOME/.kube/appstudio-releng.config ec validate image --image quay.io/rhacs-eng/scanner-db-slim:4.7.x-164-gd9dd2cb811-fast --policy registry-rh-acs --ignore-rekor | grep -F -A3 '[Violation]'
✕ [Violation] slsa_source_correlated.source_code_reference_provided
  ImageRef: quay.io/rhacs-eng/scanner-db-slim@sha256:da7962b19d773a1a1703bd857a3f2aae6316c18dd7d30e99eedbf1807258ac25
  Reason: Expected source code reference was not provided for verification

✕ [Violation] slsa_source_correlated.source_code_reference_provided
  ImageRef: quay.io/rhacs-eng/scanner-db-slim@sha256:af9e00eac004efbe9928f0170ec1781dc8b8ac5bcba9fad54eaed6378eeaab2c
  Reason: Expected source code reference was not provided for verification

✕ [Violation] slsa_source_correlated.source_code_reference_provided
  ImageRef: quay.io/rhacs-eng/scanner-db-slim@sha256:7d811b1a95b386e752859494c6f96c2a01a4b83d3a126d3f9aefbb5646999889
  Reason: Expected source code reference was not provided for verification

✕ [Violation] slsa_source_correlated.source_code_reference_provided
  ImageRef: quay.io/rhacs-eng/scanner-db-slim@sha256:627f1790693767acc8f7356315873535c2d533ea2d04e5179d28238ad52bde99
  Reason: Expected source code reference was not provided for verification

✕ [Violation] slsa_source_correlated.source_code_reference_provided
  ImageRef: quay.io/rhacs-eng/scanner-db-slim@sha256:58894dc737d324dd2e1eaec2156f2ed246dd8babaa2e8e29f32ec14bc3aa6d67
  Reason: Expected source code reference was not provided for verification

Error: success criteria not met
scanner-slim
KUBECONFIG=$HOME/.kube/appstudio-releng.config ec validate image --image quay.io/rhacs-eng/scanner-slim:4.7.x-164-gd9dd2cb811-fast --policy registry-rh-acs --ignore-rekor | grep -F -A3 '[Violation]' 
✕ [Violation] slsa_source_correlated.source_code_reference_provided
  ImageRef: quay.io/rhacs-eng/scanner-slim@sha256:ebca0702e190203033b7182ffec3f2f2da831715119f9170e44ef687eae8b730
  Reason: Expected source code reference was not provided for verification

✕ [Violation] slsa_source_correlated.source_code_reference_provided
  ImageRef: quay.io/rhacs-eng/scanner-slim@sha256:53bb4d3f42357ac0504bfedcc8d9f6088fae0476476a244c7a8bf97d789b7b55
  Reason: Expected source code reference was not provided for verification

✕ [Violation] slsa_source_correlated.source_code_reference_provided
  ImageRef: quay.io/rhacs-eng/scanner-slim@sha256:33122aeecfc5fa9b92404df451143ddd3762028c7ab0b610d550db7a47afbc98
  Reason: Expected source code reference was not provided for verification

✕ [Violation] slsa_source_correlated.source_code_reference_provided
  ImageRef: quay.io/rhacs-eng/scanner-slim@sha256:2a7addba798842f74a462d61484149b2690866fdf9fdb09b7880d9b065d755aa
  Reason: Expected source code reference was not provided for verification

✕ [Violation] slsa_source_correlated.source_code_reference_provided
  ImageRef: quay.io/rhacs-eng/scanner-slim@sha256:280c3814851d93c02abaf37ef00efc175b075542090a856b878524058f36d452
  Reason: Expected source code reference was not provided for verification

Error: success criteria not met

@msugakov msugakov marked this pull request as ready for review November 15, 2024 16:35
@msugakov msugakov requested a review from a team as a code owner November 15, 2024 16:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-all-qa-tests Tells CI to run all API tests (not just BAT). konflux-build Run Konflux in PR. Push commit to trigger it.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants