perf(ci): shallow clone for CI jobs by davdhacs · Pull Request #19294 · stackrox/stackrox

davdhacs · 2026-03-05T02:07:15Z

Summary

Remove fetch-depth: 0 from all checkout steps. Shallow clones (depth=1, the default) skip git history download, saving ~10s and ~220 MB per job (~405 MB → ~184 MB checkout).

How it works

The job-preamble action resolves version info for shallow clones in ~3s:

git ls-remote --tags finds the nearest X.Y.x version tag (~0.4s, no download)
git fetch --shallow-exclude=<tag> deepens history to that tag (~2s, 39 MB)
Creates a local tag at the boundary so git describe works
Sets BUILD_TAG env var as fallback for make/env.mk

Tries tags in reverse version order to handle both master (4.11.x) and release branches (4.10.x). Commit count is off by 1 (--shallow-exclude stops above the tag commit) — acceptable for version strings.

Benchmarked against 14 alternative strategies (test results). This approach is the fastest that produces correct version strings (8s vs 18s full clone).

Changes

job-preamble (+28 lines): shallow clone detection, tag lookup via ls-remote, history deepening via --shallow-exclude, local tag creation, BUILD_TAG env var
14 workflow files (-69 lines): removed fetch-depth: 0

No changes to make/env.mk, Makefiles, scripts, or the release process.

Exceptions

Kept fetch-depth: 0 for:

misc-checks: needs full history for git diff --check
push-scanner-manifests: no job-preamble to resolve version tag
update_*_periodic workflows: need all branches/tags for version bumps

Test plan

Unit Tests pass (go, go-postgres, go-bench, sensor, local-roxctl, shell, openshift-ci)
Build passes (pre-build-cli, pre-build-go-binaries, pre-build-docs, operators, main images)
Style passes (golangci-lint, check-generated-files, style-check, actionlint, shellcheck)
Scanner builds pass
Version strings correct (4.11.x-241-g<sha>, off by 1 from full clone's 242)
Prow e2e tests pass

🤖 Generated with Claude Code

openshift-ci · 2026-03-05T02:07:19Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

sourcery-ai

Hey - I've found 1 issue, and left some high level feedback:

In the make/env.mk fallback branch, git tag --list '[0-9]*.[0-9]*.x' can return nothing in a shallow clone, producing a version like -0-g<sha>; consider explicitly handling the “no tag found” case (e.g., defaulting to a clearer placeholder or skipping the tag- prefix when empty).
The job-preamble step relies on git ls-remote --tags origin | grep -oP 'refs/tags/\K[0-9]+\.[0-9]+\.x$' which assumes tags with a strict <major>.<minor>.x format and the availability of grep -P; consider making this more robust to different tag naming schemes (e.g., v1.2.x) and environments where grep -P may not be present.

Prompt for AI Agents

Please address the comments from this code review:

## Overall Comments
- In the `make/env.mk` fallback branch, `git tag --list '[0-9]*.[0-9]*.x'` can return nothing in a shallow clone, producing a version like `-0-g<sha>`; consider explicitly handling the “no tag found” case (e.g., defaulting to a clearer placeholder or skipping the `tag-` prefix when empty).
- The `job-preamble` step relies on `git ls-remote --tags origin | grep -oP 'refs/tags/\K[0-9]+\.[0-9]+\.x$'` which assumes tags with a strict `<major>.<minor>.x` format and the availability of `grep -P`; consider making this more robust to different tag naming schemes (e.g., `v1.2.x`) and environments where `grep -P` may not be present.

## Individual Comments

### Comment 1
<location path=".github/actions/job-preamble/action.yaml" line_range="156" />
<code_context>
+          echo "::warning::No version tag found via ls-remote"
+          exit 0
+        fi
+        git fetch --shallow-exclude="$tag" origin HEAD 2>/dev/null || true
+      shell: bash
+
</code_context>
<issue_to_address>
**question (bug_risk):** Clarify/verify that `--shallow-exclude` with `HEAD` actually yields enough history for `git describe` in all cases.

This approach assumes `HEAD` is on `origin/HEAD` and that the tag is on the same branch, so that `--shallow-exclude="$tag" origin HEAD` makes the tag an ancestor of `HEAD`.

If tags are created on other branches, this may either fetch much more history than intended or still leave the tag unreachable from `HEAD`. Consider explicitly fetching the expected branch (e.g. `origin/main`/`origin/master`) or resolving the tracked remote ref for `HEAD` and fetching that, to avoid surprises when default branches or tag placement change.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

.github/actions/job-preamble/action.yaml

rhacs-bot · 2026-03-05T02:50:53Z

Images are ready for the commit at ce30ad3.

To use with deploy scripts, first export MAIN_IMAGE_TAG=4.11.x-242-gce30ad3966.

codecov · 2026-03-05T03:17:02Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 49.59%. Comparing base (04dcf46) to head (9c50d30).
⚠️ Report is 8 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #19294      +/-   ##
==========================================
- Coverage   49.62%   49.59%   -0.03%     
==========================================
  Files        2680     2680              
  Lines      202214   202231      +17     
==========================================
- Hits       100346   100306      -40     
- Misses      94393    94432      +39     
- Partials     7475     7493      +18

Flag	Coverage Δ
go-unit-tests	`49.59% <ø> (-0.03%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Remove fetch-depth: 0 from all checkout steps. Shallow clones (depth=1, the default) skip git history download, saving ~10s and ~220 MB per job. The job-preamble action resolves version info for shallow clones in ~3s: 1. git ls-remote finds the nearest X.Y.x version tag (~0.4s) 2. git fetch --shallow-exclude deepens history to the tag (~2s) 3. A local tag at the boundary enables git describe and make tag 4. BUILD_TAG env var is set as a fallback for make/env.mk Tries tags in reverse version order to handle both master and release branches. Commit count is off by 1 (--shallow-exclude stops above the tag commit) which is acceptable for version strings. Kept fetch-depth: 0 for misc-checks (git diff --check needs full history), push-scanner-manifests (no preamble), and update_*_periodic workflows (need all branches/tags). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

davdhacs · 2026-03-05T13:47:45Z

/test all

Remove fetch-depth: 0 from all checkout steps. Shallow clones (depth=1, the default) skip git history download, saving ~10s and ~220 MB per job. The job-preamble action resolves version info for shallow clones in ~3s: 1. git ls-remote finds the nearest version tag (~0.4s) 2. git fetch --shallow-exclude deepens history to the tag (~2s) 3. A local tag at the boundary enables git describe and make tag 4. BUILD_TAG env var is set as a fallback for make/env.mk On master: finds the latest X.Y.x tag. On release branches: detects version from branch name and finds the nearest X.Y.Z or RC tag (e.g. release-4.10 finds 4.10.0, not 4.11.x). Kept fetch-depth: 0 for misc-checks (git diff --check needs full history), push-scanner-manifests (no preamble), and update_*_periodic workflows (need all branches/tags). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

davdhacs added the ai-assisted label Mar 5, 2026

openshift-ci bot added the do-not-merge/work-in-progress label Mar 5, 2026

github-actions bot added area/ci ai-review labels Mar 5, 2026

sourcery-ai bot reviewed Mar 5, 2026

View reviewed changes

.github/actions/job-preamble/action.yaml Outdated Show resolved Hide resolved

davdhacs force-pushed the davdhacs/gha-shallow-clone branch 3 times, most recently from 2227924 to 5bc30cc Compare March 5, 2026 02:36

davdhacs changed the title ~~perf(ci): shallow clone for all CI jobs~~ perf(ci): shallow clone for CI jobs Mar 5, 2026

davdhacs force-pushed the davdhacs/gha-shallow-clone branch 12 times, most recently from a419396 to 1f6b3ad Compare March 5, 2026 05:40

davdhacs force-pushed the davdhacs/gha-shallow-clone branch from 1f6b3ad to 5f5f03e Compare March 5, 2026 13:46

davdhacs force-pushed the davdhacs/gha-shallow-clone branch 3 times, most recently from ce30ad3 to f1051af Compare March 5, 2026 16:19

davdhacs force-pushed the davdhacs/gha-shallow-clone branch 3 times, most recently from e2c3843 to ef1c51f Compare March 5, 2026 17:08

davdhacs force-pushed the davdhacs/gha-shallow-clone branch from ef1c51f to f9ac8d2 Compare March 5, 2026 17:15

davdhacs closed this Mar 5, 2026

davdhacs reopened this Mar 5, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

perf(ci): shallow clone for CI jobs#19294

perf(ci): shallow clone for CI jobs#19294
davdhacs wants to merge 2 commits intomasterfrom
davdhacs/gha-shallow-clone

davdhacs commented Mar 5, 2026 •

edited

Loading

Uh oh!

openshift-ci bot commented Mar 5, 2026

Uh oh!

sourcery-ai bot left a comment

Uh oh!

Uh oh!

rhacs-bot commented Mar 5, 2026 •

edited

Loading

Uh oh!

codecov bot commented Mar 5, 2026 •

edited

Loading

Uh oh!

davdhacs commented Mar 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

davdhacs commented Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

How it works

Changes

Exceptions

Test plan

Uh oh!

openshift-ci bot commented Mar 5, 2026

Uh oh!

sourcery-ai bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

rhacs-bot commented Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

davdhacs commented Mar 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

davdhacs commented Mar 5, 2026 •

edited

Loading

rhacs-bot commented Mar 5, 2026 •

edited

Loading

codecov bot commented Mar 5, 2026 •

edited

Loading