Skip to content

ROX-12037: Support Quay Robot Accounts for registry integration#2578

Merged
theencee merged 4 commits intomasterfrom
nc/quay-robot-registry-int
Sep 6, 2022
Merged

ROX-12037: Support Quay Robot Accounts for registry integration#2578
theencee merged 4 commits intomasterfrom
nc/quay-robot-registry-int

Conversation

@theencee
Copy link
Contributor

@theencee theencee commented Aug 2, 2022

Description

When integrating Quay registry you can now optionally use robot account instead of just OAuth tokens. In fact this is Quay's recommended integration credentials. Unfortunately scanner still requires a token.

Existing integrations should remain as-is.

Checklist

  • Investigated and inspected CI test results
  • Unit test and regression tests added
  • Evaluated and added CHANGELOG entry if required
    ~[ ] Determined and documented upgrade steps
  • Documented user facing changes (create PR based on openshift/openshift-docs and merge into rhacs-docs)

If any of these don't apply, please comment below.

Testing Performed

Auomated: See files
Manual:

Validated that on upgrade the existing integrations remain as is (with or without token).

Then just using the /test endpoint:
Registry:

  1. No Auth - should pass OK
$ roxcurl v1/imageintegrations/test -X POST --data-raw '{"id":"","name":"quay test","categories":["REGISTRY"],"quay":{"endpoint":"quay.io","insecure":false},"autogenerated":false,"clusterId":"","clusters":[],"skipTestIntegration":false,"type":"quay"}' -sk | jq .
{}
  1. Token Only - should pass OK
$ roxcurl v1/imageintegrations/test -X POST --data-raw '{"id":"","name":"quay test","categories":["REGISTRY"],"quay":{"endpoint":"quay.io","oauthToken":"REDACTED","insecure":false},"autogenerated":false,"clusterId":"","clusters":[],"skipTestIntegration":false,"type":"quay"}' -sk | jq .
{}
  1. Robot Only - should pass OK
$ roxcurl v1/imageintegrations/test -X POST --data-raw '{"id":"","name":"quay test","categories":["REGISTRY"],"quay":{"endpoint":"quay.io","registryRobotCredentials":{"username":"REDACTED","password":"REDACTED"},"insecure":false},"autogenerated":false,"clusterId":"","clusters":[],"skipTestIntegration":false,"type":"quay"}' -sk | jq .
{}
  1. Token + Robot - should fail OK
$ roxcurl v1/imageintegrations/test -X POST --data-raw '{"id":"","name":"quay test","categories":["REGISTRY"],"quay":{"endpoint":"quay.io","oauthToken":"REDACTED","registryRobotCredentials":{"username":"REDACTED","password":"REDACTED"},"insecure":false},"autogenerated":false,"clusterId":"","clusters":[],"skipTestIntegration":false,"type":"quay"}' -sk | jq .
{
  "error": "registry integration: Quay registry integration should use robot credentials or robot credentials but not both: invalid arguments: invalid arguments",
  "code": 3,
  "message": "registry integration: Quay registry integration should use robot credentials or robot credentials but not both: invalid arguments: invalid arguments",
  "details": []
}
  1. Robot missing username - should fail OK
$ roxcurl v1/imageintegrations/test -X POST --data-raw '{"id":"","name":"quay test","categories":["REGISTRY"],"quay":{"endpoint":"quay.io","registryRobotCredentials":{"password":"REDACTED"},"insecure":false},"autogenerated":false,"clusterId":"","clusters":[],"skipTestIntegration":false,"type":"quay"}' -sk | jq .
{
  "error": "registry integration: If using Quay robot credentials, both username and password must be provided: invalid arguments: invalid arguments",
  "code": 3,
  "message": "registry integration: If using Quay robot credentials, both username and password must be provided: invalid arguments: invalid arguments",
  "details": []
}
  1. Robot missing password - should fail OK
$ roxcurl v1/imageintegrations/test -X POST --data-raw '{"id":"","name":"quay test","categories":["REGISTRY"],"quay":{"endpoint":"quay.io","registryRobotCredentials":{"username":"REDACTED"},"insecure":false},"autogenerated":false,"clusterId":"","clusters":[],"skipTestIntegration":false,"type":"quay"}' -sk | jq .
{
  "error": "registry integration: If using Quay robot credentials, both username and password must be provided: invalid arguments: invalid arguments",
  "code": 3,
  "message": "registry integration: If using Quay robot credentials, both username and password must be provided: invalid arguments: invalid arguments",
  "details": []
}

Scanner:

  1. No Auth - should pass OK
$ roxcurl v1/imageintegrations/test -X POST --data-raw '{"id":"","name":"quay test","categories":["SCANNER"],"quay":{"endpoint":"quay.io","insecure":false},"autogenerated":false,"clusterId":"","clusters":[],"skipTestIntegration":false,"type":"quay"}' -sk | jq .
{}
  1. Token Only - should pass OK
$ roxcurl v1/imageintegrations/test -X POST --data-raw '{"id":"","name":"quay test","categories":["SCANNER"],"quay":{"endpoint":"quay.io","oauthToken":"REDACTED","insecure":false},"autogenerated":false,"clusterId":"","clusters":[],"skipTestIntegration":false,"type":"quay"}' -sk | jq .
{}
  1. Robot Only - should fail OK
$ roxcurl v1/imageintegrations/test -X POST --data-raw '{"id":"","name":"quay test","categories":["SCANNER"],"quay":{"endpoint":"quay.io","registryRobotCredentials":{"username":"REDACTED","password":"REDACTED"},"insecure":false},"autogenerated":false,"clusterId":"","clusters":[],"skipTestIntegration":false,"type":"quay"}' -sk | jq .
{
  "error": "image scanner integration: Quay scanner integration cannot use robot credentials: invalid arguments: invalid arguments",
  "code": 3,
  "message": "image scanner integration: Quay scanner integration cannot use robot credentials: invalid arguments: invalid arguments",
  "details": []
}
  1. Token + Robot - should fail
$ roxcurl v1/imageintegrations/test -X POST --data-raw '{"id":"","name":"quay test","categories":["SCANNER"],"quay":{"endpoint":"quay.io","oauthToken":"REDACTED","registryRobotCredentials":{"username":"REDACTED","password":"REDACTED"},"insecure":false},"autogenerated":false,"clusterId":"","clusters":[],"skipTestIntegration":false,"type":"quay"}' -sk | jq .
{
  "error": "image scanner integration: Quay scanner integration cannot use robot credentials: invalid arguments: invalid arguments",
  "code": 3,
  "message": "image scanner integration: Quay scanner integration cannot use robot credentials: invalid arguments: invalid arguments",
  "details": []
}

  1. Robot missing username - should fail
$ roxcurl v1/imageintegrations/test -X POST --data-raw '{"id":"","name":"quay test","categories":["SCANNER"],"quay":{"endpoint":"quay.io","registryRobotCredentials":{"password":"REDACTED"},"insecure":false},"autogenerated":false,"clusterId":"","clusters":[],"skipTestIntegration":false,"type":"quay"}' -sk | jq .
{
  "error": "image scanner integration: Quay scanner integration cannot use robot credentials: invalid arguments: invalid arguments",
  "code": 3,
  "message": "image scanner integration: Quay scanner integration cannot use robot credentials: invalid arguments: invalid arguments",
  "details": []
}
  1. Robot missing password - should fail OK
$ roxcurl v1/imageintegrations/test -X POST --data-raw '{"id":"","name":"quay test","categories":["SCANNER"],"quay":{"endpoint":"quay.io","registryRobotCredentials":{"username":"REDACTED"},"insecure":false},"autogenerated":false,"clusterId":"","clusters":[],"skipTestIntegration":false,"type":"quay"}' -sk | jq .
{
  "error": "image scanner integration: Quay scanner integration cannot use robot credentials: invalid arguments: invalid arguments",
  "code": 3,
  "message": "image scanner integration: Quay scanner integration cannot use robot credentials: invalid arguments: invalid arguments",
  "details": []
}

Registry + Scanner:

  1. No Auth - should pass
$ roxcurl v1/imageintegrations/test -X POST --data-raw '{"id":"","name":"quay test","categories":["REGISTRY","SCANNER"],"quay":{"endpoint":"quay.io","insecure":false},"autogenerated":false,"clusterId":"","clusters":[],"skipTestIntegration":false,"type":"quay"}' -sk | jq .
{}
  1. Token Only - should pass OK
$ roxcurl v1/imageintegrations/test -X POST --data-raw '{"id":"","name":"quay test","categories":["REGISTRY","SCANNER"],"quay":{"endpoint":"quay.io","oauthToken":"REDACTED","insecure":false},"autogenerated":false,"clusterId":"","clusters":[],"skipTestIntegration":false,"type":"quay"}' -sk | jq .
{}
  1. Robot Only - should fail OK
$ roxcurl v1/imageintegrations/test -X POST --data-raw '{"id":"","name":"quay test","categories":["REGISTRY","SCANNER"],"quay":{"endpoint":"quay.io","registryRobotCredentials":{"username":"REDACTED","password":"REDACTED"},"insecure":false},"autogenerated":false,"clusterId":"","clusters":[],"skipTestIntegration":false,"type":"quay"}' -sk | jq .
{
  "error": "registry integration: Quay scanner integration cannot use robot credentials: invalid arguments: invalid arguments",
  "code": 3,
  "message": "registry integration: Quay scanner integration cannot use robot credentials: invalid arguments: invalid arguments",
  "details": []
}
  1. Token + Robot - should pass OK
$ roxcurl v1/imageintegrations/test -X POST --data-raw '{"id":"","name":"quay test","categories":["REGISTRY","SCANNER"],"quay":{"endpoint":"quay.io","oauthToken":"REDACTED","registryRobotCredentials":{"username":"REDACTED","password":"REDACTED"},"insecure":false},"autogenerated":false,"clusterId":"","clusters":[],"skipTestIntegration":false,"type":"quay"}' -sk | jq .
{}
  1. Robot missing username - should fail OK
$ roxcurl v1/imageintegrations/test -X POST --data-raw '{"id":"","name":"quay test","categories":["REGISTRY","SCANNER"],"quay":{"endpoint":"quay.io","registryRobotCredentials":{"password":"REDACTED"},"insecure":false},"autogenerated":false,"clusterId":"","clusters":[],"skipTestIntegration":false,"type":"quay"}' -sk | jq .
{
  "error": "registry integration: Quay scanner integration cannot use robot credentials: invalid arguments: invalid arguments",
  "code": 3,
  "message": "registry integration: Quay scanner integration cannot use robot credentials: invalid arguments: invalid arguments",
  "details": []
}
  1. Robot missing password - should fail OK
$ roxcurl v1/imageintegrations/test -X POST --data-raw '{"id":"","name":"quay test","categories":["REGISTRY","SCANNER"],"quay":{"endpoint":"quay.io","registryRobotCredentials":{"username":"REDACTED"},"insecure":false},"autogenerated":false,"clusterId":"","clusters":[],"skipTestIntegration":false,"type":"quay"}' -sk | jq .
{
  "error": "registry integration: Quay scanner integration cannot use robot credentials: invalid arguments: invalid arguments",
  "code": 3,
  "message": "registry integration: Quay scanner integration cannot use robot credentials: invalid arguments: invalid arguments",
  "details": []
}

And then by actually creating some integrations and using it:

  1. Add registry with token only
$ roxcurl v1/imageintegrations -X POST --data-raw '{"id":"","name":"quay test","categories":["REGISTRY"],"quay":{"endpoint":"quay.io","oauthToken":"REDACTED","insecure":false},"autogenerated":false,"clusterId":"","clusters":[],"skipTestIntegration":false,"type":"quay"}' -sk | jq .
{
  "id": "94dca201-6f91-4396-bca5-f48be015e27d",
  "name": "quay test",
  "type": "quay",
  "categories": [
    "REGISTRY"
  ],
  "quay": {
    "endpoint": "quay.io",
    "oauthToken": "REDACTED",
    "insecure": false,
    "registryRobotCredentials": null
  },
  [..snip..]
}

$ roxctl image scan -i quay.io/rhacs-eng/qa:struts-app -f | jq '.metadata.dataSource, .metadata.v1.layers[0], .scan.components[0].name, .scan.dataSource'
{
  "id": "94dca201-6f91-4396-bca5-f48be015e27d",
  "name": "quay test"
}
{
  "instruction": "ADD",
  "value": "file:9c48682ff75c756544d4491472081a078edf5dd0bb5038d1cb850a1f9c480e3e in /",
  "created": "2017-06-20T20:13:32.945602158Z"
}
"readline6"
{
  "id": "169b0d3f-8277-4900-bbce-1127077defae",
  "name": "Stackrox Scanner"
}
  1. Add registry with robot only
$ roxcurl v1/imageintegrations -X POST --data-raw '{"id":"","name":"quay test","categories":["REGISTRY"],"quay":{"endpoint":"quay.io","registryRobotCredentials":{"username":"REDACTED","password":"REDACTED"},"insecure":false},"autogenerated":false,"clusterId":"","clusters":[],"skipTestIntegration":false,"type":"quay"}' -sk | jq .
{
  "id": "69b8a1d5-289b-428b-b6ec-3d11d7a7247c",
  "name": "quay test",
  "type": "quay",
  "categories": [
    "REGISTRY"
  ],
  "quay": {
    "endpoint": "quay.io",
    "oauthToken": "",
    "insecure": false,
    "registryRobotCredentials": {
      "username": "REDACTED",
      "password": "REDACTED"
    }
  },
  [..snip..]
}

$ roxctl image scan -i quay.io/rhacs-eng/qa:struts-app -f | jq '.metadata.dataSource, .metadata.v1.layers[0], .scan.components[0].name, .scan.dataSource'
{
  "id": "69b8a1d5-289b-428b-b6ec-3d11d7a7247c",
  "name": "quay test"
}
{
  "instruction": "ADD",
  "value": "file:9c48682ff75c756544d4491472081a078edf5dd0bb5038d1cb850a1f9c480e3e in /",
  "created": "2017-06-20T20:13:32.945602158Z"
}
"libvorbis"
{
  "id": "169b0d3f-8277-4900-bbce-1127077defae",
  "name": "Stackrox Scanner"
}
  1. Add registry + scanner with token only
$ roxcurl v1/imageintegrations -X POST --data-raw '{"id":"","name":"quay test","categories":["REGISTRY","SCANNER"],"quay":{"endpoint":"quay.io","oauthToken":"REDACTED","insecure":false},"autogenerated":false,"clusterId":"","clusters":[],"skipTestIntegration":false,"type":"quay"}' -sk | jq .
{
  "id": "226ed458-45db-4c7b-afe0-be52f0ffefef",
  "name": "quay test",
  "type": "quay",
  "categories": [
    "REGISTRY",
    "SCANNER"
  ],
  "quay": {
    "endpoint": "quay.io",
    "oauthToken": "REDACTED",
    "insecure": false,
    "registryRobotCredentials": null
  },
  [..snip..]
}

$ roxctl image scan -i quay.io/rhacs-eng/qa:struts-app -f | jq '.metadata.dataSource, .metadata.v1.layers[0], .scan.components[0].name, .scan.dataSource'
{
  "id": "226ed458-45db-4c7b-afe0-be52f0ffefef",
  "name": "quay test"
}
{
  "instruction": "ADD",
  "value": "file:9c48682ff75c756544d4491472081a078edf5dd0bb5038d1cb850a1f9c480e3e in /",
  "created": "2017-06-20T20:13:32.945602158Z"
}
"libogg0"
{
  "id": "226ed458-45db-4c7b-afe0-be52f0ffefef",
  "name": "quay test"
}
  1. Add registry + scanner with token & robot
$ roxcurl v1/imageintegrations -X POST --data-raw '{"id":"","name":"quay test","categories":["REGISTRY","SCANNER"],"quay":{"endpoint":"quay.io","oauthToken":"REDACTED","registryRobotCredentials":{"username":"REDACTED","password":"REDACTED"},"insecure":false},"autogenerated":false,"clusterId":"","clusters":[],"skipTestIntegration":false,"type":"quay"}' -sk | jq .
{
  "id": "5e750920-a650-4fe6-9973-e5ca765f3984",
  "name": "quay test",
  "type": "quay",
  "categories": [
    "REGISTRY",
    "SCANNER"
  ],
  "quay": {
    "endpoint": "quay.io",
    "oauthToken": "REDACTED",
    "insecure": false,
    "registryRobotCredentials": {
      "username": "REDACTED",
      "password": "REDACTED"
    }
  },
  [..snip..]
}

$ roxctl image scan -i quay.io/rhacs-eng/qa:struts-app -f | jq '.metadata.dataSource, .metadata.v1.layers[0], .scan.components[0].name, .scan.dataSource'
{
  "id": "5e750920-a650-4fe6-9973-e5ca765f3984",
  "name": "quay test"
}
{
  "instruction": "ADD",
  "value": "file:9c48682ff75c756544d4491472081a078edf5dd0bb5038d1cb850a1f9c480e3e in /",
  "created": "2017-06-20T20:13:32.945602158Z"
}
"org.apache.myfaces.core:myfaces-impl"
{
  "id": "5e750920-a650-4fe6-9973-e5ca765f3984",
  "name": "quay test"
}

@openshift-ci
Copy link

openshift-ci bot commented Aug 2, 2022

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@theencee
Copy link
Contributor Author

theencee commented Aug 2, 2022

/test all

@ghost
Copy link

ghost commented Aug 2, 2022

Images are ready for the commit at 57decb5.

To use with deploy scripts, first export MAIN_IMAGE_TAG=3.71.x-497-g57decb5a98.

@theencee theencee force-pushed the nc/quay-robot-registry-int branch from 9be3b70 to 62f53ce Compare August 2, 2022 17:30
@theencee
Copy link
Contributor Author

theencee commented Aug 2, 2022

/test images push-images

@theencee
Copy link
Contributor Author

theencee commented Aug 2, 2022

/test stackrox_branding-images stackrox_branding-push-images

@theencee theencee force-pushed the nc/quay-robot-registry-int branch from 62f53ce to ca47b95 Compare August 3, 2022 00:17
@theencee
Copy link
Contributor Author

theencee commented Aug 3, 2022

/test images push-images stackrox_branding-images stackrox_branding-push-images

@theencee theencee force-pushed the nc/quay-robot-registry-int branch from ca47b95 to 26e894b Compare August 3, 2022 18:03
@theencee theencee force-pushed the nc/quay-robot-registry-int branch from 26e894b to 2c0aaee Compare August 11, 2022 00:01
@theencee
Copy link
Contributor Author

/test images push-images stackrox_branding-images stackrox_branding-push-images

@theencee theencee added the turbo-build Uses a faster path to images label Aug 11, 2022
@theencee theencee force-pushed the nc/quay-robot-registry-int branch 2 times, most recently from 3ceb13c to 481eee7 Compare August 17, 2022 14:51
@theencee
Copy link
Contributor Author

/test images push-images stackrox_branding-images stackrox_branding-push-images

@theencee theencee force-pushed the nc/quay-robot-registry-int branch from 481eee7 to 6665662 Compare August 27, 2022 03:19
@theencee theencee force-pushed the nc/quay-robot-registry-int branch from 6665662 to 7eee886 Compare August 27, 2022 03:22
@theencee
Copy link
Contributor Author

/test images push-images stackrox_branding-images stackrox_branding-push-images gke-qa-e2e-tests gke-postgres-qa-e2e-tests

@theencee theencee force-pushed the nc/quay-robot-registry-int branch from 7eee886 to e0413e8 Compare August 29, 2022 07:00
@theencee
Copy link
Contributor Author

/test images push-images stackrox_branding-images stackrox_branding-push-images gke-qa-e2e-tests gke-postgres-qa-e2e-tests

@theencee theencee force-pushed the nc/quay-robot-registry-int branch from e0413e8 to d669434 Compare August 29, 2022 10:06
@theencee
Copy link
Contributor Author

/test images push-images stackrox_branding-images stackrox_branding-push-images gke-qa-e2e-tests gke-postgres-qa-e2e-tests

@theencee theencee removed the turbo-build Uses a faster path to images label Aug 29, 2022
@theencee theencee force-pushed the nc/quay-robot-registry-int branch from d669434 to a4bf56a Compare August 29, 2022 17:09
@theencee
Copy link
Contributor Author

/test images push-images stackrox_branding-images stackrox_branding-push-images gke-qa-e2e-tests gke-postgres-qa-e2e-tests

@theencee theencee force-pushed the nc/quay-robot-registry-int branch from a4bf56a to a5ed9be Compare August 29, 2022 17:34
@theencee
Copy link
Contributor Author

/test images push-images stackrox_branding-images stackrox_branding-push-images gke-qa-e2e-tests gke-postgres-qa-e2e-tests

@theencee theencee force-pushed the nc/quay-robot-registry-int branch from a5ed9be to 57c1542 Compare August 29, 2022 21:41
@theencee
Copy link
Contributor Author

theencee commented Sep 1, 2022

/test images push-images stackrox_branding-images stackrox_branding-push-images gke-qa-e2e-tests gke-postgres-qa-e2e-tests go-unit-tests style-checks

@theencee theencee force-pushed the nc/quay-robot-registry-int branch from e67b841 to bdefd5f Compare September 1, 2022 03:31
@theencee
Copy link
Contributor Author

theencee commented Sep 1, 2022

/test images push-images stackrox_branding-images stackrox_branding-push-images gke-qa-e2e-tests gke-postgres-qa-e2e-tests go-unit-tests style-checks

@theencee theencee marked this pull request as ready for review September 1, 2022 15:45
@theencee theencee force-pushed the nc/quay-robot-registry-int branch 3 times, most recently from 2a73264 to 4b4eb71 Compare September 3, 2022 00:50
@stackrox stackrox deleted a comment from openshift-ci bot Sep 3, 2022
@stackrox stackrox deleted a comment from openshift-ci bot Sep 3, 2022
@theencee
Copy link
Contributor Author

theencee commented Sep 3, 2022

/retest

1 similar comment
@theencee
Copy link
Contributor Author

theencee commented Sep 3, 2022

/retest

@theencee theencee force-pushed the nc/quay-robot-registry-int branch from 4b4eb71 to 2f4e9a6 Compare September 3, 2022 15:06
@theencee theencee force-pushed the nc/quay-robot-registry-int branch 2 times, most recently from cb1bece to 9bceffc Compare September 6, 2022 09:30
Copy link
Contributor

@pedrottimark pedrottimark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Solid from frontend viewpoint.

Thank you for clear comments in code and example requests in description.

Copy link
Contributor

@RTann RTann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mostly nits otherwise LGTM

if quay.GetRegistryRobotCredentials() != nil && quay.GetOauthToken() != "" {
return errors.New("Quay registry integration should use robot credentials or robot credentials but not both")
}
} else {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it possible for len(categories) == 0 here? I think being inside this else assumes len(categories) == 2, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It gets validated elsewhere

username = config.GetRegistryRobotCredentials().GetUsername()
password = config.GetRegistryRobotCredentials().GetPassword()
} else if config.GetOauthToken() != "" {
username = oauthTokenString
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: maybe just always do this check like before and just overwrite it with robot creds if provided? I think that will remove the need for these elses

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kept it this way to make deleting the feature flag easier. Can just delete the else condition in that case.

@theencee theencee merged commit 5b0df09 into master Sep 6, 2022
@theencee theencee deleted the nc/quay-robot-registry-int branch September 6, 2022 22:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants