Skip to content

ROX-12057: collect operator e2e test artifacts#2732

Merged
porridge merged 3 commits intomasterfrom
porridge/ROX-12057-collect-artifacts
Aug 24, 2022
Merged

ROX-12057: collect operator e2e test artifacts#2732
porridge merged 3 commits intomasterfrom
porridge/ROX-12057-collect-artifacts

Conversation

@porridge
Copy link
Contributor

@porridge porridge commented Aug 18, 2022 •
edited
Loading

Description

  • Make checking of central API status optional, even though wait_for_central_api is run_with_best_effort, it causes the overall test to fail for operator jobs
  • Change the kuttl output directory for the upgrade test, to make it easier to gather results from both kuttl runs, this might be relevant to @ivan-degtiarenko 's PR
  • Use the PostClusterTest and FinalPost to gather the kuttl artifacts and the various service logs

/cc @gavin-stackrox @ivan-degtiarenko @mtesseract

Checklist

  • Investigated and inspected CI test results
  • Unit test and regression tests added
  • Evaluated and added CHANGELOG entry if required
  • Determined and documented upgrade steps
  • Documented user facing changes (create PR based on openshift/openshift-docs and merge into rhacs-docs)

Testing Performed

Relying on CI

@openshift-ci openshift-ci bot requested a review from gavin-stackrox August 18, 2022 09:31
@openshift-ci
Copy link

openshift-ci bot commented Aug 18, 2022

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@openshift-ci
Copy link

openshift-ci bot commented Aug 18, 2022

@porridge: The /test command needs one or more targets.
The following commands are available to trigger required jobs:

  • /test go-postgres-tests
  • /test go-unit-tests
  • /test go-unit-tests-release
  • /test grouped-static-checks
  • /test images
  • /test integration-unit-tests
  • /test mitre-bundles-checks
  • /test policy-checks
  • /test push-images
  • /test shell-unit-tests
  • /test stackrox_branding-images
  • /test stackrox_branding-push-images
  • /test style-checks
  • /test ui-unit-tests

The following commands are available to trigger optional jobs:

  • /test eks-qa-e2e-tests
  • /test gke-kernel-qa-e2e-tests
  • /test gke-nongroovy-e2e-tests
  • /test gke-postgres-qa-e2e-tests
  • /test gke-postgres-scale-tests
  • /test gke-qa-e2e-tests
  • /test gke-race-condition-qa-e2e-tests
  • /test gke-scale-tests
  • /test gke-ui-e2e-tests
  • /test gke-upgrade-tests
  • /test local-roxctl-tests
  • /test openshift-4-operator-e2e-tests
  • /test openshift-4-qa-e2e-tests
  • /test osd-aws-qa-e2e-tests
  • /test osd-gcp-qa-e2e-tests

Use /test all to run the following jobs that were automatically triggered:

  • pull-ci-stackrox-stackrox-master-gke-kernel-qa-e2e-tests
  • pull-ci-stackrox-stackrox-master-gke-nongroovy-e2e-tests
  • pull-ci-stackrox-stackrox-master-gke-postgres-qa-e2e-tests
  • pull-ci-stackrox-stackrox-master-gke-postgres-scale-tests
  • pull-ci-stackrox-stackrox-master-gke-qa-e2e-tests
  • pull-ci-stackrox-stackrox-master-gke-race-condition-qa-e2e-tests
  • pull-ci-stackrox-stackrox-master-gke-scale-tests
  • pull-ci-stackrox-stackrox-master-gke-ui-e2e-tests
  • pull-ci-stackrox-stackrox-master-gke-upgrade-tests
  • pull-ci-stackrox-stackrox-master-go-postgres-tests
  • pull-ci-stackrox-stackrox-master-go-unit-tests
  • pull-ci-stackrox-stackrox-master-go-unit-tests-release
  • pull-ci-stackrox-stackrox-master-grouped-static-checks
  • pull-ci-stackrox-stackrox-master-images
  • pull-ci-stackrox-stackrox-master-integration-unit-tests
  • pull-ci-stackrox-stackrox-master-local-roxctl-tests
  • pull-ci-stackrox-stackrox-master-mitre-bundles-checks
  • pull-ci-stackrox-stackrox-master-openshift-4-operator-e2e-tests
  • pull-ci-stackrox-stackrox-master-policy-checks
  • pull-ci-stackrox-stackrox-master-push-images
  • pull-ci-stackrox-stackrox-master-shell-unit-tests
  • pull-ci-stackrox-stackrox-master-stackrox_branding-images
  • pull-ci-stackrox-stackrox-master-stackrox_branding-push-images
  • pull-ci-stackrox-stackrox-master-style-checks
  • pull-ci-stackrox-stackrox-master-ui-unit-tests
Details

In response to this:

/cc @gavin-stackrox

Gavin, it's not clear to me whether the FinalPost bit is required/useful here? All users of post_test=PostClusterTest seem to be using it as well, so I cargo-culted.

/test

I suppose this will not trigger the operator e2e tests.. I wonder if there is a way to trigger it using a label like it was on CircleCI?

Description

A detailed explanation of the changes in your PR.

Feel free to remove this section if it is overkill for your PR, and the title of your PR is sufficiently descriptive.

Checklist

  • Investigated and inspected CI test results
  • Unit test and regression tests added
  • Evaluated and added CHANGELOG entry if required
  • Determined and documented upgrade steps
  • Documented user facing changes (create PR based on openshift/openshift-docs and merge into rhacs-docs)

If any of these don't apply, please comment below.

Testing Performed

TODO(replace-me)
Use this space to explain how you tested your PR, or, if you didn't test it, why
you did not do so. Valid reasons include, for example, "CI is sufficient",
"No testable changes". Feel free to attach JSON snippets, curl commands,
screenshots.

In addition to reviewing your code, reviewers must also review your testing
instructions and make sure they are sufficient.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@gavin-stackrox
Copy link
Contributor

gavin-stackrox commented Aug 18, 2022

@porridge FinalPost is useful for:

  • fixup_artifacts_content_type and make_artifacts_help which are related to how we store artifacts in a private GCS bucket in CI.
  • handle_e2e_progress_failures uses breadcrumb files to determine root cause and provide a better error in Prow. Right now it just looks for ACS deployment which might be useful for operator e2e?

The private GCS bucket is questionable, that method for storing artifacts was initially created due to concerns about log security with public CI and lieaked credentials, but OpenShift CI elides credential values. So you might be just as well storing artifacts in ARTIFACT_DIR.

@porridge
Copy link
Contributor Author

porridge commented Aug 19, 2022

/test openshift-4-operator-e2e-tests

@porridge
Copy link
Contributor Author

porridge commented Aug 19, 2022

/test images
/test push-images
/test shell-unit-tests
/test style-checks

@porridge porridge force-pushed the porridge/ROX-12057-collect-artifacts branch from 457362f to f5a0342 Compare August 19, 2022 06:57
@porridge porridge marked this pull request as ready for review August 19, 2022 07:03
@ghost
Copy link

ghost commented Aug 19, 2022

Images are ready for the commit at f5a0342.

To use with deploy scripts, first export MAIN_IMAGE_TAG=3.71.x-323-gf5a03423d3.

@porridge
Copy link
Contributor Author

porridge commented Aug 22, 2022

Looks like this works well enough, please review @gavin-stackrox

gavin-stackrox
gavin-stackrox approved these changes Aug 22, 2022
View reviewed changes
Copy link
Contributor

@gavin-stackrox gavin-stackrox left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@porridge
Copy link
Contributor Author

porridge commented Aug 22, 2022

/retest-required

@porridge porridge enabled auto-merge (squash) August 22, 2022 11:37
@porridge
Copy link
Contributor Author

porridge commented Aug 23, 2022

/retest-required

@porridge
Copy link
Contributor Author

porridge commented Aug 23, 2022

/retest

1 similar comment
@porridge
Copy link
Contributor Author

porridge commented Aug 23, 2022

/retest

@porridge
Copy link
Contributor Author

porridge commented Aug 23, 2022

Reported as an outage.

@openshift-ci
Copy link

openshift-ci bot commented Aug 23, 2022 •
edited
Loading

@porridge: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/gke-postgres-qa-e2e-tests f5a0342 link false /test gke-postgres-qa-e2e-tests
ci/prow/openshift-newest-operator-e2e-tests f5a0342 link false /test openshift-newest-operator-e2e-tests
ci/prow/openshift-oldest-operator-e2e-tests f5a0342 link false /test openshift-oldest-operator-e2e-tests

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@porridge porridge force-pushed the porridge/ROX-12057-collect-artifacts branch from f5a0342 to a8238a0 Compare August 24, 2022 04:57
@porridge porridge merged commit 48ab48c into master Aug 24, 2022
@porridge porridge deleted the porridge/ROX-12057-collect-artifacts branch August 24, 2022 05:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@gavin-stackrox gavin-stackrox gavin-stackrox approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

area/operator

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@porridge @gavin-stackrox