Skip to content

Use ubi8-micro image for operator#1220

Merged
janisz merged 5 commits intomasterfrom
tj/use_scratch_image_for_operator
Apr 11, 2022
Merged

Use ubi8-micro image for operator#1220
janisz merged 5 commits intomasterfrom
tj/use_scratch_image_for_operator

Conversation

@janisz
Copy link
Copy Markdown
Contributor

@janisz janisz commented Apr 7, 2022

Description

Currently we are using ubi8-minimal image but we only put operator there that could be build without CGO.
Operator should not require any other binaries or libs so there is no need to use system image and we can use scratch image.

Results

  • smaller image
  • faster builds (we don't need to update dependencies since we don't have any)
  • more secure image (again no deps)

Checklist

  • Investigated and inspected CI test results
  • Unit test and regression tests added
  • Evaluated and added CHANGELOG entry if required
  • Determined and documented upgrade steps
  • Documented user facing changes (create PR based on stackrox/openshift-docs and merge into rhacs-docs)

If any of these don't apply, please comment below.

Testing Performed

CI

@ghost
Copy link
Copy Markdown

ghost commented Apr 7, 2022

Tag for build #417071 is 3.69.x-337-ga673a416d8.

💻 For deploying this image using the dev scripts, run the following first:

export MAIN_IMAGE_TAG='3.69.x-337-ga673a416d8'

🕹️ A roxctl binary can be downloaded from the CircleCI artifacts.

Base automatically changed from tj/use_official_golang_image to master April 7, 2022 17:19
Copy link
Copy Markdown
Contributor

@gavin-stackrox gavin-stackrox left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Less is better! But I will need to defer approval to someone that knows CGO.

Copy link
Copy Markdown
Contributor

@msugakov msugakov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what ci-openshift-4-operator-tests label controls but please add it to this PR.


# Build the operator binary.
RUN GOOS=$(go env GOOS) scripts/go-build.sh operator
RUN GOOS=$(go env GOOS) CGO_ENABLED=0 scripts/go-build.sh operator
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Disabling CGO looks rather arbitrary. This will create a mismatch between upstream (this) and downstream builds where CGO is not disabled. Having such mismatch can lead to different behavior which isn't good because downstream images are not as thoroughly tested end-to-end as in upstream CI.

Last time I checked with Malte, there were specific reasons why CGO was enabled. I don't want to be a broken telephone relaying that info, particularly because I will have to research it myself. I believe there's more than "just" rocksdb, there are some tracing concerns that customers have.

Could you please search around for considerations why main and other images are built with CGO?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In main we need rocksb to be linked. Here we don't have any external C libs so we don't need CGO.
The only one drawback I found is when disabling CGO we end up using go network stack and DNS may behave differently.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Tomek but I don't think this info is sufficient. Have you read what I wrote just above?

Last time I checked with Malte, there were specific reasons why CGO was enabled. I don't want to be a broken telephone relaying that info, particularly because I will have to research it myself. I believe there's more than "just" rocksdb, there are some tracing concerns that customers have.

Re-stating the same thing. CGO allows customers better trace the code execution because they can attach certain tools to binaries when they are built with CGO. This is real requirement from real StackRox customers.
I suggest you to research more on this topic. Either of two outcomes are possible:

  • This is still the requirement and we still have to do it for the operator.
  • This is not a requirement any more and we can stop doing that.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it. I'll find answer about this requirement. For now... how about ubi8-micro?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Tomek, sounds good.

@janisz janisz force-pushed the tj/use_scratch_image_for_operator branch from d645ca3 to eca3156 Compare April 8, 2022 08:58
@janisz janisz requested review from SimonBaeumer and msugakov April 8, 2022 09:24
# The following are numeric uid and gid of `nobody` user in UBI.
# We can't use symbolic names because otherwise k8s will fail to start the pod with an error like this:
# Error: container has runAsNonRoot and image has non-numeric user (nobody), cannot verify user is non-root (pod: "stackrox-operator-controller-manager-75bc744454-bkbjr_stackrox-operator-system(49874aae-2695-4d3a-afd3-8723914d2af5)", container: manager)
USER 65534:65534
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@msugakov Why do we need that? I can add /etc/passwd to the image but I'm not sure if that's necessary

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You don't need /etc/passwd. When leveraging numeric uid:gid, the user doesn't need to exist in /etc/passwd.
If USER is not specified, the container will run as root. Maybe we override this in Kubernetes manifests, but still it is good to have run-as-user defined in the Dockerfile.

@janisz janisz changed the title Use scratch image for operator Use ubi8-micro image for operator Apr 8, 2022
Co-authored-by: Misha Sugakov <537715+msugakov@users.noreply.github.com>
@janisz janisz requested a review from gavin-stackrox April 11, 2022 15:14
@janisz janisz enabled auto-merge (squash) April 11, 2022 15:15
@janisz janisz merged commit adbca26 into master Apr 11, 2022
@janisz janisz deleted the tj/use_scratch_image_for_operator branch April 11, 2022 15:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants