Skip to content

ROX-13493: Add admission controller support for Scale subresource#14296

Merged
clickboo merged 4 commits intomasterfrom
boo-adm-cntrl-scale-subresource
Feb 19, 2025
Merged

ROX-13493: Add admission controller support for Scale subresource#14296
clickboo merged 4 commits intomasterfrom
boo-adm-cntrl-scale-subresource

Conversation

@clickboo
Copy link
Contributor

@clickboo clickboo commented Feb 14, 2025

Description

There exist 3 ways of scaling a deployment in k8s/Openshift:

  1. Update the deployment spec for the replicas property.
  2. Use kubectl scale or oc scale on the deployment.
  3. Use the v1/autoscaling scale subresource (Openshift console uses this)

The admission controller has existing support to run policy detection and enforcement on 1. and 2. above. However, it was missing support for the scale subresource. This PR adds that support.

While I was at it, I also fixed a small product branding miss in the output messages/response of the admission controller when the request is denied.

User-facing documentation

  • CHANGELOG is updated OR update is not needed
  • documentation PR is created and is linked above OR is not needed

Testing and quality

  • the change is production ready: the change is GA or otherwise the functionality is gated by a feature flag
  • CI results are inspected

Automated testing

  • added unit tests
  • added e2e tests
  • added regression tests
  • added compatibility tests
  • modified existing tests

How I validated my change

Testing on an Openshift cluster with admission controller enabled for 3 scenarios:

  1. Use Openshift console to scale a deployment that violates the enforced latest tag policy. Operation is disallowed.
admission-control/manager: 2025/02/17 21:35:20.733920 evaluate_deploytime.go:106: Debug: Evaluating request &AdmissionRequest{UID:3a469625-7ae3-429f-8de3-8c67890ea279,Kind:autoscaling/v1, Kind=Scale,Resource:{apps v1 deployments},SubResource:scale,Name:nginx,Namespace:default,Operation:UPDATE,UserInfo:{kube:admin  [system:cluster-admins system:authenticated] .......,RequestKind:autoscaling/v1, Kind=Scale,RequestResource:apps/v1, Resource=deployments,RequestSubResource:scale,}
admission-control/manager: 2025/02/17 21:35:20.733970 evaluate_deploytime.go:112: Debug: Not bypassing UPDATE request on default/nginx [autoscaling/v1, Kind=Scale]
admission-control/manager: 2025/02/17 21:35:20.734106 evaluate_deploytime.go:137: Debug: Evaluating policies on id:"6a25ee4b-564c-4998-838f-9f884ca3aeba"  name:"nginx"  hash:1732657809438660397  type:"Deployment"  namespace:"default"  namespace_id:"51c8c679-0899-428a-bc64-c2dfbce24588"  replicas:3  labels:{key:"app"  value:"nginx"}  pod_labels:{key:"app"  value:"nginx"}  label_selector:{match_labels:{key:"app"  value:"nginx"}}  created:{seconds:1739825295}  cluster_id:"f3319a39-884e-4465-ac3c-a003136f5f4e"  containers:{id:"6a25ee4b-564c-4998-838f-9f884ca3aeba:nginx"  config:{}  image:{id:"sha256:088eea90c3d0a540ee5686e7d7471acbd4063b6e97eaf49b5e651665eb7f4dc7"  name:{registry:"docker.io"  remote:"library/nginx"  tag:"latest"  full_name:"docker.io/library/nginx:latest"}}  security_context:{}  resources:{}  name:"nginx"  liveness_probe:{}  readiness_probe:{}}  service_account:"default"  service_account_permission_level:NONE  automount_service_account_token:true  state_timestamp:1739827889595449
admission-control/manager: 2025/02/17 21:35:20.734376 evaluate_deploytime.go:176: Debug: Violated policies: 1, rejecting UPDATE request on default/nginx [autoscaling/v1, Kind=Scale]
admission-control/service: 2025/02/17 21:35:20.734486 service.go:125: Debug: Sending admission review: {"kind":"AdmissionReview","apiVersion":"admission.k8s.io/v1","response":{"uid":"3a469625-7ae3-429f-8de3-8c67890ea279","allowed":false,"status":{"metadata":{},"status":"Failure","message":"\nThe attempted operation violated 1 enforced policy, described below:\n\nPolicy: Boo Test policy\n- Description:\n    ↳ \n- Rationale:\n    ↳ \n- Remediation:\n    ↳ \n- Violations:\n    - Container 'nginx' has image with tag 'latest'\n\n\nIn case of emergency, add the annotation {\"admission.stackrox.io/break-glass\": \"ticket-1234\"} to your deployment with an updated ticket number\n\n","reason":"Failed currently enforced policies from StackRox"}}}
admission-control/alerts: 2025/02/17 21:35:20.734667 sender.go:113: Debug: Sending 1 alert results to Sensor
  1. Use oc scale to scale deployment replicas on the same deployment
ksanchet@ksanchet-mac:~/go/src/github.com/stackrox/stackrox/deploy/openshift$ oc -n default scale deploy/nginx --replicas=5
Error from server (Failed currently enforced policies from StackRox): admission webhook "policyeval.stackrox.io" denied the request: 
The attempted operation violated 1 enforced policy, described below:

Policy: Boo Test policy
- Description:
   ↳ 
- Rationale:
   ↳ 
- Remediation:
   ↳ 
- Violations:
   - Container 'nginx' has image with tag 'latest'


In case of emergency, add the annotation {"admission.stackrox.io/break-glass": "ticket-1234"} to your deployment with an updated ticket number
  1. Use deployment edit to update the replicas of the same deployment.
ksanchet@ksanchet-mac:~/go/src/github.com/stackrox/stackrox/deploy/openshift$ kubectl -n default edit deploy/nginx
error: deployments.apps "nginx" could not be patched: admission webhook "policyeval.stackrox.io" denied the request: 
The attempted operation violated 1 enforced policy, described below:

Policy: Boo Test policy
- Description:
    ↳ 
- Rationale:
    ↳ 
- Remediation:
    ↳ 
- Violations:
    - Container 'nginx' has image with tag 'latest'


In case of emergency, add the annotation {"admission.stackrox.io/break-glass": "ticket-1234"} to your deployment with an updated ticket number


You can run `kubectl replace -f /var/folders/f2/mx8qpbtd0fs_23ln92y_d7m40000gn/T/kubectl-edit-7uary.yaml` to try this update again.

@rhacs-bot
Copy link
Contributor

rhacs-bot commented Feb 15, 2025

Images are ready for the commit at 6f4b7e6.

To use with deploy scripts, first export MAIN_IMAGE_TAG=4.8.x-24-g6f4b7e61dc.

@codecov
Copy link

codecov bot commented Feb 15, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 49.04%. Comparing base (2e2c217) to head (6f4b7e6).
Report is 388 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #14296      +/-   ##
==========================================
- Coverage   49.06%   49.04%   -0.02%     
==========================================
  Files        2514     2515       +1     
  Lines      182847   182869      +22     
==========================================
- Hits        89707    89684      -23     
- Misses      86020    86063      +43     
- Partials     7120     7122       +2     
Flag Coverage Δ
go-unit-tests 49.04% <ø> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@clickboo clickboo force-pushed the boo-adm-cntrl-scale-subresource branch 4 times, most recently from 89aa6d0 to 8d562b1 Compare February 17, 2025 22:23
@clickboo clickboo changed the title WIP ROX-13493: Add admission controller support for Scale subresource Feb 17, 2025
@clickboo clickboo force-pushed the boo-adm-cntrl-scale-subresource branch from f90f303 to 2c5448c Compare February 18, 2025 01:34
@clickboo clickboo force-pushed the boo-adm-cntrl-scale-subresource branch from 161cf91 to 212b19b Compare February 18, 2025 01:48
@clickboo clickboo force-pushed the boo-adm-cntrl-scale-subresource branch from 212b19b to 0b93d14 Compare February 18, 2025 01:49
@clickboo
Copy link
Contributor Author

clickboo commented Feb 18, 2025

/test ocp-4-12-nongroovy-e2e-tests

@openshift-ci
Copy link

openshift-ci bot commented Feb 18, 2025

@clickboo: The /retest command does not accept any targets.
The following commands are available to trigger required jobs:

/test gke-nongroovy-e2e-tests
/test shell-unit-tests

The following commands are available to trigger optional jobs:

/test aks-qa-e2e-tests
/test aro-qa-e2e-tests
/test eks-qa-e2e-tests
/test gke-nongroovy-compatibility-tests
/test gke-operator-e2e-tests
/test gke-perf-scale-tests
/test gke-qa-e2e-tests
/test gke-race-condition-qa-e2e-tests
/test gke-scale-tests
/test gke-scanner-v4-install-tests
/test gke-sensor-integration-tests
/test gke-ui-e2e-tests
/test gke-upgrade-tests
/test gke-version-compatibility-tests
/test ibmcloudz-4-14-qa-e2e-tests
/test ibmcloudz-4-15-qa-e2e-tests
/test ibmcloudz-4-16-qa-e2e-tests
/test ibmcloudz-4-17-qa-e2e-tests
/test ocp-4-12-compliance-e2e-tests
/test ocp-4-12-ebpf-qa-e2e-tests
/test ocp-4-12-nongroovy-e2e-tests
/test ocp-4-12-operator-e2e-tests
/test ocp-4-12-qa-e2e-tests
/test ocp-4-12-scanner-v4-install-tests
/test ocp-4-12-sensor-integration-tests
/test ocp-4-12-ui-e2e-tests
/test ocp-4-17-compliance-e2e-tests
/test ocp-4-17-crun-qa-e2e-tests
/test ocp-4-17-ebpf-qa-e2e-tests
/test ocp-4-17-fips-qa-e2e-tests
/test ocp-4-17-nongroovy-e2e-tests
/test ocp-4-17-operator-e2e-tests
/test ocp-4-17-qa-e2e-tests
/test ocp-4-17-scanner-v4-install-tests
/test ocp-4-17-sensor-integration-tests
/test ocp-4-17-ui-e2e-tests
/test ocp-dev-preview-compliance-e2e-tests
/test ocp-dev-preview-ebpf-qa-e2e-tests
/test ocp-dev-preview-fips-qa-e2e-tests
/test ocp-dev-preview-nongroovy-e2e-tests
/test ocp-dev-preview-operator-e2e-tests
/test ocp-dev-preview-qa-e2e-tests
/test ocp-dev-preview-scanner-v4-install-tests
/test ocp-dev-preview-sensor-integration-tests
/test ocp-dev-preview-ui-e2e-tests
/test ocp-next-candidate-compliance-e2e-tests
/test ocp-next-candidate-ebpf-qa-e2e-tests
/test ocp-next-candidate-fips-qa-e2e-tests
/test ocp-next-candidate-nongroovy-e2e-tests
/test ocp-next-candidate-operator-e2e-tests
/test ocp-next-candidate-qa-e2e-tests
/test ocp-next-candidate-scanner-v4-install-tests
/test ocp-next-candidate-sensor-integration-tests
/test ocp-next-candidate-ui-e2e-tests
/test ocp-stable-scanner-v4-install-compliance-e2e-tests
/test ocp-stable-scanner-v4-install-ebpf-qa-e2e-tests
/test ocp-stable-scanner-v4-install-nongroovy-e2e-tests
/test ocp-stable-scanner-v4-install-operator-e2e-tests
/test ocp-stable-scanner-v4-install-perf-scale-tests
/test ocp-stable-scanner-v4-install-qa-e2e-tests
/test ocp-stable-scanner-v4-install-scanner-v4-install-tests
/test ocp-stable-scanner-v4-install-sensor-integration-tests
/test ocp-stable-scanner-v4-install-ui-e2e-tests
/test osd-aws-qa-e2e-tests
/test osd-gcp-qa-e2e-tests
/test powervs-4-14-qa-corebpf-e2e-tests
/test powervs-4-15-qa-corebpf-e2e-tests
/test powervs-4-16-qa-corebpf-e2e-tests
/test powervs-4-17-qa-corebpf-e2e-tests
/test rosa-hcp-qa-e2e-tests
/test rosa-qa-e2e-tests
/test ui-component-tests

Use /test all to run the following jobs that were automatically triggered:

pull-ci-stackrox-stackrox-master-gke-nongroovy-e2e-tests
pull-ci-stackrox-stackrox-master-gke-qa-e2e-tests
pull-ci-stackrox-stackrox-master-gke-sensor-integration-tests
pull-ci-stackrox-stackrox-master-gke-upgrade-tests
pull-ci-stackrox-stackrox-master-ocp-4-12-nongroovy-e2e-tests
pull-ci-stackrox-stackrox-master-ocp-4-17-nongroovy-e2e-tests
Details

In response to this:

/retest ocp-4-12-nongroovy-e2e-tests

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Copy link
Contributor

@dashrews78 dashrews78 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks OK based on my knowledge

@clickboo clickboo added this to the 4.7.0-rc.1 milestone Feb 18, 2025
@johannes94 johannes94 modified the milestones: 4.7.0-rc.1, 4.7.0-rc.2 Feb 19, 2025
@clickboo clickboo modified the milestones: 4.7.0-rc.2, 4.7.1 Feb 19, 2025
@clickboo clickboo merged commit 9f548f9 into master Feb 19, 2025
97 checks passed
@clickboo clickboo deleted the boo-adm-cntrl-scale-subresource branch February 19, 2025 17:39
@clickboo clickboo added the backport release-4.6 Create a PR to backport this PR to release-4.6 label Feb 19, 2025
@rhacs-bot
Copy link
Contributor

The backport to release-4.6 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-release-4.6 release-4.6
# Navigate to the new working tree
cd .worktrees/backport-release-4.6
# Create a new branch
git switch --create backport-14296-to-release-4.6
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 9f548f9ad152c3b742f412a0c3d1df781189be48
# Push it to GitHub
git push --set-upstream origin backport-14296-to-release-4.6
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-release-4.6

Then, create a pull request where the base branch is release-4.6 and the compare/head branch is backport-14296-to-release-4.6.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants