Skip to content

perf: reduce busybox init-time memory allocation#20024

Draft
davdhacs wants to merge 8 commits intomasterfrom
davdhacs/pr1e-busybox-init-v2
Draft

perf: reduce busybox init-time memory allocation#20024
davdhacs wants to merge 8 commits intomasterfrom
davdhacs/pr1e-busybox-init-v2

Conversation

@davdhacs
Copy link
Copy Markdown
Contributor

@davdhacs davdhacs commented Apr 15, 2026

Description

Reduce init-time heap allocations in the busybox binary through lazy initialization patterns. The busybox binary is the shared entrypoint for sensor, admission-control, config-controller, and other components — savings here apply to all.

Changes

  • Wrap pkg/postgres/schema registration with sync.OnceValue (~200 generated files)
  • Make schema registration idempotent for compatibility
  • Lazy regexp compilation in pkg/booleanpolicy
  • Move pkg/printers/table to sub-package (breaks transitive import chain)
  • Move test code from sensor/common/centralproxy/testutils.go to _test.go
  • Build-tag sensor/kubernetes/fake to exclude from production binary
  • Inline CVE ID utilities to avoid heavyweight pkg/cve import chain

Measurements

  • Busybox init heap: 16.1 MB → 12.9 MB (-20%)
  • Malloc count: 245K → 173K (-29%)
  • Admission-control standalone: 9.1 MB → 5.6 MB (-38%)

Risk

Low. Changes are lazy wrappers around existing code. sync.OnceValue is thread-safe. Schema registration is idempotent. Build tags verified with CI.

Part of the memory baseline optimization series.

User-facing documentation

Testing and quality

  • the change is production ready
  • CI results are inspected

Automated testing

  • modified existing tests

How I validated my change

  • Deployed on live GKE test cluster, all components stable
  • Verified all schema registrations work with sync.OnceValue lazy loading
  • Verified build tags exclude sensor/kubernetes/fake from production binary
  • Deployed on live GKE test cluster, all components stable

AI-assisted.

Reduce init-time memory for the busybox binary by eliminating unnecessary
imports, deferring allocations with sync.OnceValue, and breaking heavy
transitive dependency chains.

Results (Linux amd64):
- Busybox: 16.1 MB -> 12.9 MB heap (-20%), 245K -> 173K mallocs (-29%)
- AC standalone: 9.1 MB -> 7.2 MB heap (-21%), 87K -> 51K mallocs (-41%)
- Binary size: 205 MB -> 194 MB (-5%)

Generated with assistance from AI

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@openshift-ci
Copy link
Copy Markdown

openshift-ci bot commented Apr 15, 2026

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, we are unable to review this pull request

The GitHub API does not allow us to fetch diffs exceeding 300 files, and this pull request has 315

Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, we are unable to review this pull request

The GitHub API does not allow us to fetch diffs exceeding 300 files, and this pull request has 315

@davdhacs davdhacs force-pushed the davdhacs/pr1e-busybox-init-v2 branch from f70d9c4 to 88838a7 Compare April 15, 2026 17:47
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 15, 2026

🚀 Build Images Ready

Images are ready for commit 9e5c763. To use with deploy scripts:

export MAIN_IMAGE_TAG=4.11.x-665-g9e5c7631a3

Test files still accessed schema variables and regex variables directly
instead of calling the lazy wrapper functions:
- schema.XxxSchema.OptionsMap → schema.XxxSchema().OptionsMap (16 files)
- deploymentBaseSchema = schema.DeploymentsSchema → schema.DeploymentsSchema()
- comparatorDecimalValueRegex → comparatorDecimalValueRegex() (validate_test.go)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@davdhacs davdhacs force-pushed the davdhacs/pr1e-busybox-init-v2 branch from 88838a7 to 2b0ea0e Compare April 15, 2026 18:18
The schema.go.tpl template was updated to use sync.OnceValue but the
test schemas in migrator/migrations/postgreshelper/schema/ and
pkg/postgres/schema/test_*.go were not regenerated. CI's
check-generated-files step regenerates them, producing a mismatch.

Run: make go-generated-srcs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@davdhacs davdhacs force-pushed the davdhacs/pr1e-busybox-init-v2 branch 2 times, most recently from 8070e24 to 89ee853 Compare April 15, 2026 20:22
With sync.OnceValue schemas, registration happens on first access.
Tests that explicitly call RegisterCategoryToTable or RegisterTable
after accessing a schema would cause fatal duplicate registration.
Make both functions idempotent — silently ignore re-registration
of the same table.

Also fix select_field_test.go which incorrectly added () to
TestStructsSchema (a test schema not converted to sync.OnceValue).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@davdhacs davdhacs force-pushed the davdhacs/pr1e-busybox-init-v2 branch 2 times, most recently from da02250 to 3afcaae Compare April 15, 2026 21:50
StackRox lint rules require "github.com/stackrox/rox/pkg/sync" instead
of stdlib "sync". Add OnceValue wrapper to pkg/sync/common_aliases.go
and update the schema template to use it. Regenerated all schema files.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@davdhacs davdhacs force-pushed the davdhacs/pr1e-busybox-init-v2 branch from 3afcaae to e521989 Compare April 15, 2026 22:21
With sync.OnceValue, schema construction is lazy — but ApplyAllSchemas
needs table create statements in registeredTables at startup to create
database tables. Split registration into two phases:

1. init(): RegisterTableStmt registers table name + create statement
   (cheap, no walker.Walk)
2. Lazy: RegisterTable updates with the full walker.Schema on first
   schema function access

Also fixes:
- goimports formatting on pkg/branding, pkg/cloudproviders/aws
- Remove unused //nolint:wrapcheck directive
- Regenerate all schema files with init() registration

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@davdhacs davdhacs force-pushed the davdhacs/pr1e-busybox-init-v2 branch from c39e023 to 59839c0 Compare April 15, 2026 23:37
@codecov
Copy link
Copy Markdown

codecov bot commented Apr 16, 2026

Codecov Report

❌ Patch coverage is 92.09660% with 72 lines in your changes missing coverage. Please review.
✅ Project coverage is 49.70%. Comparing base (4d2d701) to head (9e5c763).
⚠️ Report is 11 commits behind head on master.

Files with missing lines Patch % Lines
pkg/helm/util/render.go 0.00% 13 Missing ⚠️
central/imagev2/datastore/store/postgres/store.go 55.55% 4 Missing ⚠️
central/compliance/aggregation/aggregation.go 0.00% 3 Missing ⚠️
...or/migrations/postgreshelper/schema/test_child2.go 0.00% 3 Missing ⚠️
...ions/postgreshelper/schema/test_g2_grand_child1.go 0.00% 3 Missing ⚠️
...rations/postgreshelper/schema/test_grand_child1.go 0.00% 3 Missing ⚠️
...r/migrations/postgreshelper/schema/test_parent1.go 0.00% 3 Missing ⚠️
...tions/postgreshelper/schema/test_short_circuits.go 0.00% 3 Missing ⚠️
migrator/version/version.go 0.00% 3 Missing ⚠️
pkg/postgres/schema/all.go 91.17% 2 Missing and 1 partial ⚠️
... and 19 more
Additional details and impacted files
@@            Coverage Diff             @@
##           master   #20024      +/-   ##
==========================================
+ Coverage   49.61%   49.70%   +0.09%     
==========================================
  Files        2765     2765              
  Lines      208628   209432     +804     
==========================================
+ Hits       103509   104107     +598     
- Misses      97462    97663     +201     
- Partials     7657     7662       +5     
Flag Coverage Δ
go-unit-tests 49.70% <92.09%> (+0.09%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@davdhacs
Copy link
Copy Markdown
Contributor Author

/test gke-qa-e2e-tests

@davdhacs
Copy link
Copy Markdown
Contributor Author

/test gke-qa-e2e-tests
/test gke-nongroovy-e2e-tests
/test gke-ui-e2e-tests
/test gke-upgrade-tests
/test ocp-4-12-nongroovy-e2e-tests
/test ocp-4-21-nongroovy-e2e-tests

- Remove tools/measure-busybox-mem/ (unrelated to this PR, measures
  import chain costs not sync.OnceValue improvement)
- Fix schema.go.tpl import ordering: move pkg/sync to third-party
  group (was incorrectly mixed with stdlib imports)
- Regenerate all schema files with corrected import ordering

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@davdhacs
Copy link
Copy Markdown
Contributor Author

/test gke-qa-e2e-tests
/test gke-nongroovy-e2e-tests

davdhacs added a commit that referenced this pull request Apr 16, 2026
Reduce init-time allocations by deferring computation to first use:
- pkg/booleanpolicy: lazy regexp compilation via sync.OnceValue,
  lazy evaluator.Factory via lazyFactory struct
- pkg/branding: lazy branded logo loading
- pkg/cloudproviders/aws: lazy root CA pool construction
- pkg/probeupload: lazy probe name regex
- pkg/printers/table: moved to sub-package to break transitive
  import chain (roxctl → pkg/printers → tablewriter)
- pkg/postgres/id: inline CVE ID utilities (avoids heavyweight
  pkg/cve import chain)
- sensor/kubernetes/fake: //go:build fakeworkloads tag excludes
  fake workload generator from production binary
- sensor/common/centralproxy: move test code to _test.go

Depends on: #20024 (for pkg/sync.OnceValue wrapper)

AI-assisted.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@davdhacs davdhacs force-pushed the davdhacs/pr1e-busybox-init-v2 branch from d165ee7 to e321a4b Compare April 16, 2026 12:42
Keep only schema lazy loading (sync.OnceValue) changes in this PR:
- Template changes (schema.go.tpl, singleton.go.tpl, store.go.tpl)
- Schema infrastructure (all.go, mapping.go, common_aliases.go)
- Generated schema files
- Mechanical caller updates (Schema.Field → Schema().Field)

Moved to separate PR:
- pkg/booleanpolicy lazy regexp/factory
- pkg/branding lazy logo
- pkg/cloudproviders/aws lazy certs
- pkg/probeupload lazy regex
- pkg/printers/table sub-package move
- pkg/postgres/id inline utilities
- sensor/kubernetes/fake build tags
- sensor/common/centralproxy testutils rename

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@davdhacs
Copy link
Copy Markdown
Contributor Author

/test gke-qa-e2e-tests
/test gke-nongroovy-e2e-tests

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant