perf(ci): disable GOMODCACHE by default in cache-go-dependencies by davdhacs · Pull Request #19688 · stackrox/stackrox

davdhacs · 2026-03-30T15:02:22Z

Description

Remove GOMODCACHE caching from CI. Go lazy-downloads only the modules each job needs from the global proxy.

Timing — all affected jobs

Unit Tests workflow (master vs PR, single run):

Job	Master	PR	Cache (before → after)
go (GOTAGS="")	36.3m	35.6m	111s → 42s
go (GOTAGS=release)	31.9m	31.4m	111s → 40s
go-postgres (GOTAGS="")	31.9m	28.3m	76s → 17s
go-postgres (release)	27.3m	31.0m	88s → 21s
sensor-integration	22.3m	21.4m	71s → 10s
go-bench	20.5m	17.3m	90s → 14s
local-roxctl	8.9m	8.1m	80s → 24s

Style workflow (master vs PR averages, 5 PR runs):

Job	Master	PR avg	Cache (before → after)
style-check	19.8m	17.2m	111s → 35s
check-generated-files	11.6m	11.6m	71s → 13s

Build workflow (master vs PR, 3 PR runs):

Job	Master	PR avg	Cache (before → after)
pre-build-cli	6.4m	4.9m	122s → 49s
pre-build-go-binaries (amd64)	5.0m	3.4m	90s → 16s
pre-build-docs	5.2m	1.7m	67s → 2s

Cache step 60-100s faster across all jobs. Total time unchanged or faster.

Changes

Remove GOMODCACHE save/restore and make deps from cache action
Add go mod download github.com/stackrox/scanner to style and build workflows for scanner proto generation
Remove go mod tidy "Resolve mods for protos" steps from build.yaml
protogen.mk and copy_scanner_protos.sh unchanged

Testing and quality

CI results inspected

How I validated my change

Multiple CI runs across all three workflows. All jobs pass. Cache step 60-100s faster. Total time unchanged or faster.

Partially generated by AI.

openshift-ci · 2026-03-30T15:02:27Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

sourcery-ai

Hey - I've reviewed your changes and they look great!

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

coderabbitai · 2026-03-30T15:11:58Z

📝 Walkthrough

Walkthrough

Removed Go module cache handling from the composite action (GOMODCACHE restore/save and its outputs) and the explicit "Download Go modules" step; workflows now explicitly run go mod download github.com/stackrox/scanner at several jobs and the composite action only retains GOCACHE caching.

Changes

Cohort / File(s)	Summary
Composite Action: cache-go-dependencies `.github/actions/cache-go-dependencies/action.yaml`	Removed `GOMODCACHE` path output and all `GOMODCACHE` restore/save steps; deleted the unconditional "Download Go modules" step. Left `GOCACHE` caching logic unchanged.
Workflow: build `.github/workflows/build.yaml`	Replaced `go mod tidy` steps with targeted `go mod download github.com/stackrox/scanner` in `pre-build-docs` and `build-and-push-operator` jobs; shifts dependency handling from tidy to explicit download.
Workflows: style checks `.github/workflows/style.yaml`	Added `go mod download github.com/stackrox/scanner` in `check-generated-files` and `style-check` jobs after Go dependency caching and before subsequent steps.

Sequence Diagram(s)

sequenceDiagram
    participant WF as Workflow Job
    participant CA as Composite Action
    participant CACHE as actions/cache
    participant GO as Go toolchain / proxy
    participant MODULE as github.com/stackrox/scanner

    WF->>CA: invoke cache-go-dependencies
    CA->>CACHE: restore GOCACHE
    note right of CA: GOMODCACHE restore/save removed
    CA->>GO: run build/test (Go may fetch modules lazily)
    WF->>GO: run `go mod download github.com/stackrox/scanner` (explicit download in workflows)
    GO->>MODULE: download module via proxy
    CA->>CACHE: save GOCACHE (as before)

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately describes the main change: removing GOMODCACHE caching from the CI cache action for performance optimization.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Description check	✅ Passed	The PR description provides clear context including a detailed comparison of timing metrics, specific changes made, and validation through CI runs, though some optional template sections are not completed.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch davdhacs/skip-gomodcache

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

rhacs-bot · 2026-03-30T15:18:21Z

Images are ready for the commit at 7544f30.

To use with deploy scripts, first export MAIN_IMAGE_TAG=4.11.x-508-g7544f3013f.

codecov · 2026-03-30T15:43:28Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 49.65%. Comparing base (44d74f3) to head (7544f30).
⚠️ Report is 5 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #19688      +/-   ##
==========================================
- Coverage   49.67%   49.65%   -0.02%     
==========================================
  Files        2747     2747              
  Lines      207296   207296              
==========================================
- Hits       102964   102937      -27     
- Misses      96683    96702      +19     
- Partials     7649     7657       +8

Flag	Coverage Δ
go-unit-tests	`49.65% <ø> (-0.02%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

coderabbitai · 2026-03-30T15:50:43Z

CodeRabbit chat interactions are restricted to organization members for this repository. Ask an organization member to interact with CodeRabbit, or set chat.allow_non_org_members: true in your configuration.

Replace skip-mod-cache with cache-mod (default false) from #19688. GOMODCACHE disabled by default — Go lazy-downloads from proxy. Remove make deps from action, go mod tidy from build.yaml. Scanner proto fixes: targeted go mod download. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Replace cache-mod approach with clean removal from #19688: - Remove GOMODCACHE save/restore and cache-mod input entirely - Remove make deps from action - Remove go mod tidy from build.yaml - Scanner proto: ensure-scanner-protos lazy download in protogen.mk - copy_scanner_protos.sh: targeted go mod download Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Add cache-mod input (default: false) to cache-go-dependencies action. When false, GOMODCACHE restore/save and make deps are skipped. Go lazy-downloads only needed modules when GOCACHE doesn't have them. Test jobs with warm GOCACHE don't need modules — the 2.9GB GOMODCACHE restore was adding ~60s per job for no benefit. Build and style jobs that compile code should set cache-mod: true. No workflow changes in this PR — callers opt-in by setting cache-mod. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

make deps runs go mod tidy on ALL go.mod files, downloading 4.8GB of modules (1665 dirs, 3 minutes). With lazy loading, Go downloads only what each job needs from the module proxy: - go test ./pkg/set: 13 dirs, 8s - go build ./roxctl: 549 dirs, 36s - go vet ./pkg/set: 13 dirs, 3s Tested locally with empty GOMODCACHE + warm GOCACHE. All operations succeed — Go lazy-downloads from the proxy as needed. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

go list -m returns empty Dir when module source isn't in GOMODCACHE. Add explicit go mod download for the single scanner module (~few MB) instead of relying on make deps downloading all 4.8GB of modules. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Same issue as copy_scanner_protos.sh — go list -m returns empty Dir when module source isn't in GOMODCACHE. Add go mod download for the single scanner module before go list. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Remove GOMODCACHE save/restore and make deps from cache-go-dependencies. Go lazy-downloads only needed modules from the global proxy. Scanner proto resolution (go list -m needs module source downloaded) is handled by explicit go mod download steps in the GHA workflows that need it, not in the Makefile or scripts: - style.yaml: check-generated-files, style-check - build.yaml: pre-build-docs (swagger), build-and-push-operator protogen.mk and copy_scanner_protos.sh are unchanged — they still error if scanner module is not available, which is the right behavior for local development. Cache step: 92s → 33s. Total compute unchanged. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Replace skip-mod-cache with cache-mod (default false) from #19688. GOMODCACHE disabled by default — Go lazy-downloads from proxy. Remove make deps from action, go mod tidy from build.yaml. Scanner proto fixes: targeted go mod download. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Replace cache-mod approach with clean removal from #19688: - Remove GOMODCACHE save/restore and cache-mod input entirely - Remove make deps from action - Remove go mod tidy from build.yaml - Scanner proto: ensure-scanner-protos lazy download in protogen.mk - copy_scanner_protos.sh: targeted go mod download Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

sourcery-ai

Hey - I've left some high level feedback:

The go mod download github.com/stackrox/scanner step is now duplicated across multiple workflows; consider extracting this into a small reusable composite action or a shared script to keep the behavior consistent and easier to update.

Prompt for AI Agents

Please address the comments from this code review:

## Overall Comments
- The `go mod download github.com/stackrox/scanner` step is now duplicated across multiple workflows; consider extracting this into a small reusable composite action or a shared script to keep the behavior consistent and easier to update.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

coderabbitai · 2026-03-31T03:45:05Z

CodeRabbit chat interactions are restricted to organization members for this repository. Ask an organization member to interact with CodeRabbit, or set chat.allow_non_org_members: true in your configuration.

sourcery-ai

Hey - I've left some high level feedback:

You now have the go mod download github.com/stackrox/scanner step duplicated across multiple workflows; consider extracting this into a small reusable composite action or script to keep the proto-generation prerequisites DRY and easier to adjust later.
With go mod tidy removed from build.yaml, if there are any workflows or scripts that relied on tidy to normalize go.mod/go.sum before proto generation, it may be safer to explicitly run go mod download (or similar) for any additional modules those steps depend on, rather than only the scanner module.

Prompt for AI Agents

Please address the comments from this code review:

## Overall Comments
- You now have the `go mod download github.com/stackrox/scanner` step duplicated across multiple workflows; consider extracting this into a small reusable composite action or script to keep the proto-generation prerequisites DRY and easier to adjust later.
- With `go mod tidy` removed from `build.yaml`, if there are any workflows or scripts that relied on `tidy` to normalize `go.mod/go.sum` before proto generation, it may be safer to explicitly run `go mod download` (or similar) for any additional modules those steps depend on, rather than only the scanner module.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

davdhacs · 2026-03-31T15:54:01Z

You now have the go mod download github.com/stackrox/scanner step duplicated across multiple workflows; consider extracting this into a small reusable composite action or script to keep the proto-generation prerequisites DRY and easier to adjust later.

Because it is a single line command, I think it is better as-is to show exactly what it is doing.

With go mod tidy removed from build.yaml, if there are any workflows or scripts that relied on tidy to normalize go.mod/go.sum before proto generation, it may be safer to explicitly run go mod download (or similar) for any additional modules those steps depend on, rather than only the scanner module.

our go.mod/go.sum are tracked in the repo. So this is not a concern.

janisz

LGTM I thought that GHA cache is faster but it's not much faster than the proxy

davdhacs · 2026-03-31T16:38:48Z

LGTM I thought that GHA cache is faster but it's not much faster than the proxy

Yes!

speeds: The GHA cache is usually a little faster than the global proxy. But the GHA cache download speed wildly varies (average 184MB/s but ranges 10MB/s to 400MB/s (maybe noisy-neighbor)). And the global proxy from github runners appears to stay around 180MB/s.

GHA cache:

  360 MB/s avg, 8s elapsed  https://github.com/stackrox/stackrox/actions/runs/23756986758
  338 MB/s avg, 8s elapsed  https://github.com/stackrox/stackrox/actions/runs/23654038249
  245 MB/s avg, 11s elapsed https://github.com/stackrox/stackrox/actions/runs/23650520985
  192 MB/s avg, 15s elapsed https://github.com/stackrox/stackrox/actions/runs/23754835181
  131 MB/s avg, 22s elapsed https://github.com/stackrox/stackrox/actions/runs/23800615444
  125 MB/s avg, 23s elapsed https://github.com/stackrox/stackrox/actions/runs/23774174062

And usually jobs don't download the full set of modules. So then the 180MB/s is for downloading fewer packages and the time spent is less.

…-mtime Resolve conflict: keep mtime stabilization step, drop removed 'Download Go modules' step (removed by #19688 GOMODCACHE PR). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

davdhacs · 2026-03-31T19:39:47Z

Post-merge validation: confirmed no regression on master. Two consecutive runs show consistent improvement:

Job	Pre-merge (run 23809907701)	GOMODCACHE merge (run 23811469828)	ROX-33550 post (run 23812858733)
go ("")	38m03s	35m09s	37m00s
go (release)	35m57s	32m44s	31m55s
go-postgres ("", 15)	31m58s	29m38s	26m08s
go-postgres (release, 15)	32m38s	33m25s	29m20s
sensor-integration	24m14s	21m30s	21m20s
local-roxctl	10m16s	7m55s	8m55s
go-bench	19m35s	15m49s	17m40s

Most Go jobs 2-4 min faster from skipping GOMODCACHE restore + make deps.

davdhacs added the ai-assisted label Mar 30, 2026

openshift-ci bot added the do-not-merge/work-in-progress label Mar 30, 2026

github-actions bot added area/ci ai-review coderabbit-review labels Mar 30, 2026

sourcery-ai bot reviewed Mar 30, 2026

View reviewed changes

davdhacs force-pushed the davdhacs/skip-gomodcache branch 6 times, most recently from bf34ec5 to c5d2336 Compare March 30, 2026 20:36

davdhacs force-pushed the davdhacs/skip-gomodcache branch from c5d2336 to ad43dcd Compare March 30, 2026 20:46

davdhacs removed ai-assisted coderabbit-review ai-review labels Mar 31, 2026

davdhacs and others added 6 commits March 30, 2026 21:00

empty: retry CI (container init was failing)

0114b7d

davdhacs added 2 commits March 30, 2026 21:00

empty: timing data point 2

bd2b70d

empty: timing data point 3

d941366

davdhacs force-pushed the davdhacs/skip-gomodcache branch from 732ed20 to d941366 Compare March 31, 2026 03:00

github-actions bot added ai-review coderabbit-review labels Mar 31, 2026

sourcery-ai bot reviewed Mar 31, 2026

View reviewed changes

empty: timing data point 6

20b1014

davdhacs mentioned this pull request Mar 31, 2026

perf(ci): enable full Go test caching (fixed ldflags + stable mtimes) #19618

Closed

Merge origin/master into davdhacs/skip-gomodcache

7544f30

davdhacs marked this pull request as ready for review March 31, 2026 15:33

davdhacs requested a review from a team as a code owner March 31, 2026 15:33

openshift-ci bot removed the do-not-merge/work-in-progress label Mar 31, 2026

davdhacs requested a review from janisz March 31, 2026 15:33

sourcery-ai bot reviewed Mar 31, 2026

View reviewed changes

janisz approved these changes Mar 31, 2026

View reviewed changes

davdhacs merged commit 92a755a into master Mar 31, 2026
107 of 108 checks passed

davdhacs deleted the davdhacs/skip-gomodcache branch March 31, 2026 17:45

Conversation

davdhacs commented Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Timing — all affected jobs

Changes

Testing and quality

How I validated my change

Uh oh!

openshift-ci bot commented Mar 30, 2026

Uh oh!

sourcery-ai bot left a comment

Choose a reason for hiding this comment

Uh oh!

coderabbitai bot commented Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Sequence Diagram(s)

Estimated code review effort

Uh oh!

rhacs-bot commented Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

coderabbitai bot commented Mar 30, 2026

Uh oh!

sourcery-ai bot left a comment

Choose a reason for hiding this comment

Uh oh!

coderabbitai bot commented Mar 31, 2026

Uh oh!

sourcery-ai bot left a comment

Choose a reason for hiding this comment

Uh oh!

davdhacs commented Mar 31, 2026

Uh oh!

janisz left a comment

Choose a reason for hiding this comment

Uh oh!

davdhacs commented Mar 31, 2026

Uh oh!

Uh oh!

davdhacs commented Mar 31, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

davdhacs commented Mar 30, 2026 •

edited

Loading

coderabbitai bot commented Mar 30, 2026 •

edited

Loading

rhacs-bot commented Mar 30, 2026 •

edited

Loading

codecov bot commented Mar 30, 2026 •

edited

Loading