Skip to content

ROX-9487: Define Sensor env vars for connecting to local scanner#807

Merged
juanrh merged 5 commits intomasterfrom
juanrh/ROX-9487
Mar 7, 2022
Merged

ROX-9487: Define Sensor env vars for connecting to local scanner#807
juanrh merged 5 commits intomasterfrom
juanrh/ROX-9487

Conversation

@juanrh
Copy link
Contributor

@juanrh juanrh commented Mar 3, 2022

Description

#747 declares env vars for connecting to local scanner, that Sensor uses in sensor/common/scannerclient/singleton.go to connect to local scanner This change defines the values for those variables:

Checklist

  • Investigated and inspected CI test results
  • Unit test and regression tests added
  • Evaluated and added CHANGELOG entry if required
  • Determined and documented upgrade steps

No CHANGELOG entry or upgrade steps required.

Testing Performed

Added helm test.

go test -v  github.com/stackrox/rox/pkg/helm/charts/tests/securedclusterservices -run TestWithHelmtest/testdata/helmtest/scanner-slim.test.yaml

go test -v  github.com/stackrox/rox/pkg/helm/charts/tests/securedclusterservices && \
go test -v github.com/stackrox/rox/central/clusters/zip && \
go test -v github.com/stackrox/rox/roxctl/helm/output

@juanrh juanrh requested review from RTann and SimonBaeumer March 3, 2022 16:28
@ghost
Copy link

ghost commented Mar 3, 2022

Tag for build #278987 is 3.69.x-34-g84f5a95bd6.

💻 For deploying this image using the dev scripts, run the following first:

export MAIN_IMAGE_TAG='3.69.x-34-g84f5a95bd6'

📦 You can also generate an installation bundle with:

docker run -i --rm stackrox/main:3.69.x-34-g84f5a95bd6 central generate interactive > bundle.zip

🕹️ A roxctl binary artifact can be downloaded from CircleCI.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this ensure scanner.stackrox.svc:8443 is used in the Central namespace and scanner-slim.stackrox.svc:8443 is used in the other Secured Clusters?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, because in image/templates/helm/shared/templates/_scanner_init.tpl.htpl the function "srox.scannerInit" does:

  • {{ if or (eq $scannerCfg.mode "") (eq $scannerCfg.mode "full") }} then {{ $_ := set $scannerCfg "name" "scanner" }}
  • {{ else if eq $scannerCfg.mode "slim" }} then {{ $_ := set $scannerCfg "name" "scanner-slim" }}

and here we call "srox.scannerInit" with the parameter $scannerCfg set to$._rox.scanner

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I opened a PR to rename the stackrox resources: #820

Maybe let's wait with change for this PR to be merged or rebasing on top of it?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah let's make sure this is always scanner.<namespace>.svc:8443. One thing to note is the port is technically configurable in the Scanner ConfigMap, but it really shouldn't be changed. Not sure why we offer this, and I wonder if we should just remove that @connorgorman ? Anyway, that's for another time. I think assuming 8443 is safe

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After ROX-9589 this is now {{ printf "scanner.%s.svc:8443" .Release.Namespace }}

@juanrh juanrh requested a review from RTann March 4, 2022 08:37
Copy link
Contributor

@RTann RTann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for now. Let's wait for #820 to be merged first and then update this PR to remove the slim suffix

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this need to be inline with the subsequent - name: ROX_SCANNER_GRPC_ENDPOINT?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see new logic after applying Simon's suggestion above

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah let's make sure this is always scanner.<namespace>.svc:8443. One thing to note is the port is technically configurable in the Scanner ConfigMap, but it really shouldn't be changed. Not sure why we offer this, and I wonder if we should just remove that @connorgorman ? Anyway, that's for another time. I think assuming 8443 is safe

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be eventually be scanner.custom-ns.svc:8443 once #820 is merged

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link
Contributor

@SimonBaeumer SimonBaeumer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM after the if-case is a bit adjusted 👊

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The env var does not need to be set unconditionally because false is the default even if it is not set on the pod.
See here: https://github.com/stackrox/stackrox/blob/master/pkg/env/sensor.go#L20-L20

The reason why you have the nil pointer errors is because of this code which only adds the scanner shape if the feature flag is enabled and not in kubectl output mode:
https://github.com/stackrox/stackrox/blob/master/image/templates/helm/stackrox-secured-cluster/templates/_init.tpl.htpl#L41-L45

Can you add this env variables only:

  • If not in kubectl output AND feature flag enabled in the meta templating stage?
    The check is this: [< if and (not .KubectlOutput) .FeatureFlags.ROX_LOCAL_IMAGE_SCANNING ->]
  • The env variables should be inside the {{- if ._rox.env.openshift }} above.
{{ if ._rox.env.openshift }}
- name: ROX_OPENSHIFT_API
  value: "true"
[<- if and (not .KubectlOutput) .FeatureFlags.ROX_LOCAL_IMAGE_SCANNING >]
- name: ROX_USE_LOCAL_SCANNER
   value: {{ not ._rox.scanner.disable }}
 - name: ROX_SCANNER_GRPC_ENDPOINT
   value: {{ printf "%s.%s.svc:8443" ._rox.scanner.name .Release.Namespace }}
[<- end >]
{{ end }}

In case you get an error because {{ not ._rox.scanner.disable }} was not "false" you need to do a hack and change it to {{ not ._rox.scanner.disable | not | not }} to convert it

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. I had to remove the test to check ROX_SCANNER_GRPC_ENDPOINT is missing, because I cannot manipulate KubectlOutput or FeatureFlags in the test

@juanrh juanrh force-pushed the juanrh/ROX-9487 branch from a46c426 to 84f5a95 Compare March 7, 2022 15:03
@juanrh juanrh requested review from SimonBaeumer March 7, 2022 15:03
@SimonBaeumer SimonBaeumer added this to the 69.0-rc.2 milestone Mar 7, 2022
@juanrh juanrh merged commit 737e37d into master Mar 7, 2022
@juanrh juanrh deleted the juanrh/ROX-9487 branch March 7, 2022 15:25
RTann pushed a commit that referenced this pull request Mar 9, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants