Skip to content

Comments

chore: docker workflow refactor#2820

Merged
auricom merged 34 commits intomainfrom
claude/docker_rework
Nov 10, 2025
Merged

chore: docker workflow refactor#2820
auricom merged 34 commits intomainfrom
claude/docker_rework

Conversation

@auricom
Copy link
Contributor

@auricom auricom commented Nov 6, 2025

🎯 Summary

This PR refactors the Docker build and release workflows to improve maintainability, security, and separation of concerns. The changes reorganize Dockerfiles into a consistent apps/ directory structure and split monolithic workflows into focused, reusable components.


📋 Changes

Workflow Refactoring

  • Separated workflows for better modularity:

    • ci.yml - Main CI orchestrator with image tag determination
    • docker.yml - Reusable Docker build workflow
    • docker-tests.yml - Docker image testing workflow
    • release.yml - Tag-based release workflow for app publishing
  • Removed ci_release.yml (deprecated/consolidated into new structure)

  • Enhanced test.yml - Simplified by removing Docker build logic (134 lines removed)

Docker Organization

  • Moved Dockerfiles to standardized locations:

    • Dockerfileapps/testapp/Dockerfile
    • da/cmd/local-da/Dockerfileapps/local-da/Dockerfile
    • Updated apps/grpc/single/Dockerfile and docker-compose.yml
  • Removed Dockerfile.da (consolidated)

Documentation

  • Added .github/RELEASE_QUICK_START.md (231 lines)

    • Quick reference guide for Docker and Go module releases

🔄 Workflow Architecture

┌─────────────┐
   ci.yml      (Main orchestrator)
└──────┬──────┘
       
       ├──► lint.yml
       ├──► docker.yml ──► Build images
       ├──► test.yml
       ├──► docker-tests.yml ──► Test images
       └──► proto.yml

┌─────────────────┐
  release.yml      (Tag-based releases)
└─────────────────┘
       
       └──► Build & push versioned images to GHCR

📝 Release Process

Docker App Releases (Automated)

git tag evm/single/v0.2.0
git push origin evm/single/v0.2.0
# Workflow automatically builds and publishes to GHCR

Tag Format

{app-path}/v{major}.{minor}.{patch}

Examples:

  • evm/single/v0.2.0ghcr.io/evstack/ev-node-evm-single:v0.2.0
  • testapp/v1.0.0ghcr.io/evstack/ev-node-testapp:v1.0.0

See RELEASE_QUICK_START.md for complete guide.

@github-actions
Copy link
Contributor

github-actions bot commented Nov 6, 2025

The latest Buf updates on your PR. Results from workflow CI and Release / buf-check (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed⏩ skipped✅ passed✅ passedNov 6, 2025, 10:10 AM

@claude
Copy link
Contributor

claude bot commented Nov 6, 2025

Claude finished @auricom's task —— View job


Comprehensive Code Review: Docker Workflow Refactor

I've completed a thorough review of this Docker workflow refactoring PR. The changes represent a significant improvement in modularity and organization. Below are my findings organized by focus area:


🎯 Overall Assessment

Strengths:

  • ✅ Excellent separation of concerns - splitting monolithic workflows into focused, reusable components
  • ✅ Clear architectural improvement with proper workflow orchestration
  • ✅ Comprehensive documentation in RELEASE.md covering both Docker and Go module releases
  • ✅ Standardized Dockerfile organization under apps/ directory
  • ✅ Multi-platform build support (amd64, arm64)
  • ✅ Proper tag validation and error handling in release workflows

Areas for Improvement: See detailed findings below


🔒 1. Security Findings

Critical: Unpinned Docker Actions (Lines flagged by GitHub Security)

Risk Level: Medium - Using version tags instead of commit SHAs creates supply chain risk

Affected Files:

  • .github/workflows/docker-build-push.yml:34, 37, 44
  • .github/workflows/release.yml:69, 72, 79
  • .github/workflows/lint.yml:51, 65, 80, 40

Current:

- uses: docker/setup-buildx-action@v3
- uses: docker/login-action@v3
- uses: docker/build-push-action@v6

Recommended:

- uses: docker/setup-buildx-action@988b5a0280414f521da01fcc63a27aeeb4b104db # v3.10.0
- uses: docker/login-action@9780b0c442fbb1117ed29e0efdff1e18412f7567 # v3.3.0
- uses: docker/build-push-action@4f58ea79222b3b9dc2c8bbdd6debcef730109a75 # v6.9.0

Rationale: Commit SHA pinning prevents malicious code injection if action maintainers' accounts are compromised. This is a GitHub security best practice for production workflows.

Missing Permissions in Workflows

Issue: .github/workflows/ci.yml sets permissions: {} at the top level, which is good, but the test job doesn't need the permissions it declares.

Current (ci.yml:61-64):

test:
  permissions:
    actions: read
    contents: read
  uses: ./.github/workflows/test.yml

Issue: As reviewer @chatton noted, test.yml no longer requires image-tag input or actions: read permission since Docker logic was removed.

Recommendation:

test:
  permissions:
    contents: read
  uses: ./.github/workflows/test.yml
  secrets: inherit

Docker Image Security

Finding: apps/grpc/single/Dockerfile properly implements security best practices:

  • ✅ Non-root user (lines 34-35, 48)
  • ✅ Minimal runtime image (alpine)
  • ✅ Proper file permissions

Recommendation: Consider applying the same non-root user pattern to other Dockerfiles:

  • apps/testapp/Dockerfile - runs as root
  • apps/evm/single/Dockerfile - runs as root
  • apps/local-da/Dockerfile - runs as root

🏗️ 2. Code Quality & Architecture

Excellent: Workflow Orchestration Design

The new workflow architecture is well-designed:

ci.yml (orchestrator)
  ├─> lint.yml
  ├─> docker-build-push.yml (reusable, matrix-based)
  ├─> test.yml
  ├─> docker-tests.yml (depends on docker build)
  └─> proto.yml

release.yml (tag-triggered, standalone)
  └─> parse-tag → build-and-push

Strengths:

  • Clear separation between CI and release workflows
  • Reusable workflow pattern enables DRY principle
  • Matrix strategy allows parallel app builds
  • Proper job dependencies (needs: clauses)

Issue: Redundant Workflow Dependency

File: .github/workflows/ci.yml:69

docker-tests:
  needs: [determine-image-tag, docker]  # ← docker already depends on determine-image-tag

Issue: docker-tests lists both determine-image-tag and docker as dependencies, but since docker already depends on determine-image-tag, the explicit dependency is redundant.

Recommendation:

docker-tests:
  needs: docker  # Sufficient - docker already waits for determine-image-tag

Issue: Workflow Naming Clarity

File: .github/workflows/docker-build-push.yml

As reviewer @chatton suggested, the filename should be more explicit about its purpose.

Current: docker-build-push.yml
Recommended: docker-build.yml or build-docker.yml

Rationale: The workflow is a reusable component that builds (and sometimes pushes) images. The simpler name better reflects its role as a building block.

Good: Image Tag Determination Logic

File: .github/workflows/ci.yml:12-39

The tag determination logic is well-implemented:

  • ✅ Handles PR numbers: pr-123
  • ✅ Handles branch names with sanitization (replaces / with -)
  • ✅ Validates tag format with regex
  • ✅ Provides notice messages for debugging

Minor Enhancement: Consider adding validation for maximum tag length (Docker tags max at 128 chars).


🐳 3. Dockerfile Review

apps/testapp/Dockerfile

Issues:

  1. Base image inconsistency: Uses golang:1.24 as final stage, unnecessarily large
  2. Multi-stage inefficiency: Copies binary but inherits full Go toolchain

Current (line 28):

FROM base  # Still contains Go toolchain, build tools

Recommended:

FROM alpine:3.22.2  # Minimal runtime, ~5MB vs ~800MB
RUN apk --no-cache add ca-certificates
  1. Build step issue (line 24): Running both go mod download AND make install may download dependencies twice

Recommended:

COPY . .
RUN make install  # Make target should handle dependencies

apps/evm/single/Dockerfile

Issue: Questionable build pattern (line 13)

RUN go mod tidy && CGO_ENABLED=0 GOOS=linux go build -o evm-single .

Problem: Running go mod tidy during Docker build can cause:

  • Non-reproducible builds (different go.mod/go.sum than source)
  • Build failures if go.mod is already tidy
  • CI/CD antipattern (go.mod changes should be committed)

Recommended:

RUN CGO_ENABLED=0 GOOS=linux go build -o evm-single .

Run go mod tidy before building the image, not during.

apps/local-da/Dockerfile

Good practices observed:

  • ✅ Multi-stage build
  • ✅ Minimal runtime image
  • ✅ Uses Makefile for consistent builds

Minor issue: Inconsistent WORKDIR convention (uses /root vs /home/ev-node)

apps/grpc/single/Dockerfile

Excellent implementation:

  • ✅ Non-root user with proper UID/GID (1000)
  • ✅ Proper permissions and ownership
  • ✅ Documented port exposures
  • ✅ Security best practices throughout

This should be the reference implementation for other Dockerfiles.


📚 4. Documentation Review

RELEASE.md - Excellent Comprehensive Guide

Strengths:

  • ✅ Clear separation between Docker and Go module releases
  • ✅ Detailed dependency graph visualization
  • ✅ Step-by-step instructions with wait times for Go proxy
  • ✅ Multiple scenarios covered (single app, multiple apps, hotfixes)
  • ✅ Troubleshooting section
  • ✅ Best practices and gotchas documented

Addresses Reviewer Concern: @tac0turtle suggested merging with an existing release doc. However, the current RELEASE.md appears to BE that consolidated document - it covers both Docker releases (new) and Go module releases. No additional merge appears necessary unless there's another release doc I'm not seeing.

Minor suggestions:

  1. Add a table of contents for easier navigation
  2. Consider adding example workflow run links
  3. Add estimated time for full multi-module release (e.g., "Allow 2-3 hours for full release cycle")

⚡ 5. Performance & Efficiency

Build Optimization Opportunities

Current: Each Dockerfile follows its own pattern for dependency downloading.

apps/evm/single/Dockerfile does this well (lines 5-8):

COPY core core
COPY go.mod go.sum ./
RUN go mod download
COPY . .

Rationale: Leverages Docker layer caching - dependencies only re-download if go.mod/go.sum changes.

apps/testapp/Dockerfile could benefit from the same pattern (currently copies everything before downloading).

Matrix Build Efficiency

File: .github/workflows/docker-build-push.yml:26-28

strategy:
  matrix:
    app: ${{ fromJson(inputs.apps) }}

Good: Parallel builds for multiple apps
Enhancement opportunity: Add fail-fast: false to ensure all builds complete even if one fails during testing.


🧪 6. Testing & Reliability

Test Workflow Cleanup

File: .github/workflows/test.yml

Improvement: PR description notes "134 lines removed" - excellent cleanup! The test workflow is now focused solely on Go tests without Docker build concerns.

Issue: Workflow still has image-tag in inputs (based on @chatton's comment) but doesn't use it. This is a remnant from the old structure.

Docker Test Dependencies

File: .github/workflows/docker-tests.yml:19-34

Good:

  • ✅ Properly depends on docker build completion
  • ✅ Uses environment variables for image configuration
  • ✅ Separate job for upgrade tests

Enhancement: Consider adding timeout to prevent stuck jobs:

docker-tests:
  timeout-minutes: 30
  # ...

Merge Group Optimization

File: .github/workflows/docker-build-push.yml:21

if: github.event_name != 'merge_group'

Excellent: Skips redundant builds for merge queue since images are already built on PRs. Smart optimization!


🔧 7. Error Handling & Resilience

Release Tag Validation

File: .github/workflows/release.yml:23-55

Excellent implementation:

  • ✅ Validates tag format
  • ✅ Checks app directory exists
  • ✅ Verifies Dockerfile presence
  • ✅ Clear error messages with ::error::
  • ✅ Uses notice messages for transparency

This is exemplary error handling for workflows.

Image Tag Validation

File: .github/workflows/ci.yml:34-37

if [[ ! "$TAG" =~ ^[a-zA-Z0-9._-]+$ ]]; then
  echo "::error::Invalid image tag format: $TAG"
  exit 1
fi

Good: Prevents invalid Docker tags
Enhancement: Add check for tag length (Docker limit is 128 chars)

Missing: Retry Logic for Docker Push

Files: All workflows using docker/build-push-action

Issue: Docker registry pushes can fail due to network issues, especially for multi-platform builds.

Recommendation: Add retry logic or at least document expected behavior on transient failures.


📋 Addressing Reviewer Comments

@tac0turtle's Comments:

  1. "love the removal of this" (lint.yml:35) - Agreed, cleanup is excellent

  2. "could we merge this with RELEASE.md" - Current RELEASE.md appears to already be the consolidated version. If there's a separate doc, I don't see it in the changed files. The current RELEASE.md covers both Docker and Go modules comprehensively.

  3. "remove?" comment on apps/grpc/single/Dockerfile - Context unclear, but if referring to blank lines, the file is well-structured

  4. "are you worried about possible errors that arise between minor releases" - Likely referring to Dockerfile Go version pinning. This is actually good practice for reproducibility.

@chatton's Comments:

  1. Remove image-tag from test.yml - Correct, no longer needed after Docker logic removal

  2. Rename docker-build-push.yml - Good suggestion for clarity

  3. Remove unused dependency in ci.yml - The determine-image-tag dependency in docker-tests is redundant

  4. Move permissions to workflow level in docker-tests.yml - Good consolidation suggestion


🎯 Action Items Summary

High Priority (Security)

  1. Pin Docker actions to commit SHAs across all workflows
  2. Remove redundant permissions from test workflow in ci.yml
  3. Consider adding non-root users to remaining Dockerfiles

Medium Priority (Correctness)

  1. Remove go mod tidy from apps/evm/single/Dockerfile (line 13)
  2. Fix apps/testapp/Dockerfile to use minimal runtime image
  3. Remove redundant determine-image-tag dependency from docker-tests job
  4. Rename workflow file from docker-build-push.yml to docker-build.yml

Low Priority (Enhancement)

  1. Add fail-fast: false to matrix builds for better visibility
  2. Add timeout-minutes to long-running test jobs
  3. Optimize dependency caching in apps/testapp/Dockerfile
  4. Add table of contents to RELEASE.md

✅ Conclusion

This is a high-quality refactor that significantly improves the codebase's CI/CD architecture. The separation of concerns, reusable workflow pattern, and comprehensive documentation are excellent.

Primary concerns:

  • Security: Unpinned actions need commit SHA pinning
  • Dockerfiles: Some optimization and security hardening opportunities
  • Minor workflow cleanup: Remove unused dependencies and parameters

Recommendation: Address the security concerns (action pinning, permissions), then this PR is ready to merge. The workflow architecture is solid and the documentation is comprehensive.


@codecov
Copy link

codecov bot commented Nov 6, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 64.64%. Comparing base (8384123) to head (811e200).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2820   +/-   ##
=======================================
  Coverage   64.64%   64.64%           
=======================================
  Files          81       81           
  Lines        7224     7224           
=======================================
  Hits         4670     4670           
  Misses       2014     2014           
  Partials      540      540           
Flag Coverage Δ
combined 64.64% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@auricom auricom marked this pull request as ready for review November 7, 2025 14:46
password: ${{ secrets.GITHUB_TOKEN }}

- name: Build and push Docker image
uses: docker/build-push-action@v6

Check warning

Code scanning / CodeQL

Unpinned tag for a non-immutable Action in workflow Medium

Unpinned 3rd party Action 'Release' step
Uses Step
uses 'docker/build-push-action' with ref 'v6', not a pinned commit hash
@github-actions
Copy link
Contributor

github-actions bot commented Nov 7, 2025

PR Preview Action v1.6.2

🚀 View preview at
https://evstack.github.io/docs-preview/pr-2820/

Built to branch main at 2025-11-07 15:52 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

@auricom auricom force-pushed the claude/docker_rework branch from e2b4760 to 52a0c80 Compare November 7, 2025 15:52
tac0turtle
tac0turtle previously approved these changes Nov 7, 2025
Copy link
Contributor

@tac0turtle tac0turtle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@auricom auricom enabled auto-merge November 7, 2025 16:55
@auricom auricom disabled auto-merge November 10, 2025 08:17
@tac0turtle tac0turtle removed this pull request from the merge queue due to a manual request Nov 10, 2025
@auricom auricom marked this pull request as draft November 10, 2025 10:49
chatton
chatton previously approved these changes Nov 10, 2025
Copy link
Contributor

@chatton chatton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like a great refactor thanks for the cleanup, I just had a few comments about a few redundant dependencies and some renaming suggestions but nothing major.

@auricom auricom marked this pull request as ready for review November 10, 2025 13:52
tac0turtle
tac0turtle previously approved these changes Nov 10, 2025
@auricom auricom marked this pull request as draft November 10, 2025 15:07
@auricom auricom marked this pull request as ready for review November 10, 2025 15:53
@auricom auricom enabled auto-merge November 10, 2025 15:55
@auricom auricom added this pull request to the merge queue Nov 10, 2025
Merged via the queue into main with commit 0c98bb3 Nov 10, 2025
29 of 30 checks passed
@auricom auricom deleted the claude/docker_rework branch November 10, 2025 16:05
@github-project-automation github-project-automation bot moved this to Done in Evolve Nov 10, 2025
alpe pushed a commit that referenced this pull request Nov 11, 2025
## 🎯 Summary

This PR refactors the Docker build and release workflows to improve
maintainability, security, and separation of concerns. The changes
reorganize Dockerfiles into a consistent `apps/` directory structure and
split monolithic workflows into focused, reusable components.

---

## 📋 Changes

### Workflow Refactoring

- __Separated workflows__ for better modularity:

  - `ci.yml` - Main CI orchestrator with image tag determination
  - `docker.yml` - Reusable Docker build workflow
  - `docker-tests.yml` - Docker image testing workflow
  - `release.yml` - Tag-based release workflow for app publishing

- __Removed__ `ci_release.yml` (deprecated/consolidated into new
structure)

- __Enhanced__ `test.yml` - Simplified by removing Docker build logic
(134 lines removed)

### Docker Organization

- __Moved Dockerfiles__ to standardized locations:

  - `Dockerfile` → `apps/testapp/Dockerfile`
  - `da/cmd/local-da/Dockerfile` → `apps/local-da/Dockerfile`
  - Updated `apps/grpc/single/Dockerfile` and `docker-compose.yml`

- __Removed__ `Dockerfile.da` (consolidated)

### Documentation

- __Added__ `.github/RELEASE_QUICK_START.md` (231 lines)

  - Quick reference guide for Docker and Go module releases

## 🔄 Workflow Architecture

```javascript
┌─────────────┐
│   ci.yml    │  (Main orchestrator)
└──────┬──────┘
       │
       ├──► lint.yml
       ├──► docker.yml ──► Build images
       ├──► test.yml
       ├──► docker-tests.yml ──► Test images
       └──► proto.yml

┌─────────────────┐
│  release.yml    │  (Tag-based releases)
└─────────────────┘
       │
       └──► Build & push versioned images to GHCR
```

---

## 📝 Release Process

### Docker App Releases (Automated)

```bash
git tag evm/single/v0.2.0
git push origin evm/single/v0.2.0
# Workflow automatically builds and publishes to GHCR
```

### Tag Format

`{app-path}/v{major}.{minor}.{patch}`

Examples:

- `evm/single/v0.2.0` → `ghcr.io/evstack/ev-node-evm-single:v0.2.0`
- `testapp/v1.0.0` → `ghcr.io/evstack/ev-node-testapp:v1.0.0`

See [RELEASE_QUICK_START.md](.github/RELEASE_QUICK_START.md) for
complete guide.

---------

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
alpe added a commit that referenced this pull request Nov 11, 2025
* main:
  build(deps): Bump golangci/golangci-lint-action from 8.0.0 to 9.0.0 (#2826)
  build(deps): Bump actions/github-script from 7 to 8 (#2827)
  refactor: replace interface{} with any for clarity and modernization (#2829)
  add note on full node for evm (#2825)
  chore: docker workflow refactor (#2820)
  docs: add version notice warning about pre-v1 releases (#2824)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

3 participants