Skip to content

refactor(chore): Decouple v1 API from storage.Cluster protobuf#19824

Draft
vjwilson wants to merge 3 commits intomasterfrom
vjwilson/refactor-cluster-storage-proto
Draft

refactor(chore): Decouple v1 API from storage.Cluster protobuf#19824
vjwilson wants to merge 3 commits intomasterfrom
vjwilson/refactor-cluster-storage-proto

Conversation

@vjwilson
Copy link
Copy Markdown
Contributor

@vjwilson vjwilson commented Apr 3, 2026

Introduce a dedicated v1.ClusterConfig message in cluster_service.proto that is field-number-identical to storage.Cluster, ensuring full gRPC wire and REST/JSON backward compatibility across version-skewed Central/roxctl/Sensor deployments.

  • Define v1.ClusterConfig with all 32 fields matching storage.Cluster field numbers and types; shared sub-messages and enums remain in storage package to limit blast radius
  • Add conversion layer in central/cluster/service/convert.go between storage.Cluster and v1.ClusterConfig; server-managed fields (status, health_status, etc.) are accepted in requests but ignored
  • Update service_impl.go RPCs to use v1.ClusterConfig at the API boundary while keeping storage.Cluster internally
  • Update all consumers: roxctl (sensor generate, cluster delete), config-controller client and mocks
  • Add roxctl/sensor/generate/convert.go helper for validation calls that still require storage.Cluster

Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com

User-facing documentation

  • CHANGELOG.md update is not needed
  • documentation PR is not needed

Testing and quality

  • the change is production ready
  • CI results are inspected

Automated testing

  • added unit tests
  • modified existing tests

How I validated my change

  • go build ./... passes with zero errors
  • All unit tests pass for changed packages: central/cluster/service, roxctl/sensor/generate, roxctl/cluster/delete, config-controller/pkg/client
  • v1.ClusterConfig is field-number-identical to storage.Cluster, verified by inspection of the proto definition, ensuring wire compatibility

@vjwilson vjwilson requested a review from a team April 3, 2026 18:19
@vjwilson vjwilson added the do-not-merge A change which is not meant to be merged label Apr 3, 2026
@openshift-ci
Copy link
Copy Markdown

openshift-ci bot commented Apr 3, 2026

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 1 issue, and left some high level feedback:

  • There are now three separate mapping implementations between storage.Cluster and v1.ClusterConfig (central service, sensor generate, plus the implicit proto identity), which makes it easy for fields to drift; consider centralizing these conversions in a shared helper/package so new/changed fields only need to be wired in one place.
  • The notion of "server‑managed" fields (status, health_status, helm_config, etc.) is currently enforced only implicitly in convertAPIClusterToStorage; it may be safer to capture this set in a single, clearly named helper or configuration so future contributors don’t accidentally start persisting client-supplied values for those fields.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- There are now three separate mapping implementations between `storage.Cluster` and `v1.ClusterConfig` (central service, sensor generate, plus the implicit proto identity), which makes it easy for fields to drift; consider centralizing these conversions in a shared helper/package so new/changed fields only need to be wired in one place.
- The notion of "server‑managed" fields (status, health_status, helm_config, etc.) is currently enforced only implicitly in `convertAPIClusterToStorage`; it may be safer to capture this set in a single, clearly named helper or configuration so future contributors don’t accidentally start persisting client-supplied values for those fields.

## Individual Comments

### Comment 1
<location path="central/cluster/service/convert.go" line_range="11-20" />
<code_context>
+func convertStorageClusterToAPI(cluster *storage.Cluster) *v1.ClusterConfig {
</code_context>
<issue_to_address>
**suggestion:** Manual 1:1 field mapping in both conversion functions is fragile and may drift as proto definitions evolve.

Both `convertStorageClusterToAPI` and `convertAPIClusterToStorage` manually copy every field, which is easy to miss when new fields are added, contradicting the comment that all fields are copied. Please consider a more robust pattern (shared helper, codegen, or tests/compile-time checks such as round-trip `proto.Equal`) so new fields can’t be accidentally omitted.

Suggested implementation:

```golang
import (
	v1 "github.com/stackrox/rox/generated/api/v1"
	"github.com/stackrox/rox/generated/storage"
	"google.golang.org/protobuf/encoding/protojson"
)

```

```golang
 // convertStorageClusterToAPI converts a storage.Cluster to the API representation.
 // All fields are copied 1:1 via protojson. The API type is structurally identical to the storage
 // type today; they are separated to allow independent evolution in the future. Using protojson
 // reduces the risk of fields drifting as proto definitions evolve.
func convertStorageClusterToAPI(cluster *storage.Cluster) *v1.ClusterConfig {
	if cluster == nil {
		return nil
	}

	out := &v1.ClusterConfig{}

	// Use protojson to perform a structural copy so newly-added fields are automatically handled
	// as long as the underlying proto definitions remain structurally compatible.
	bytes, err := protojson.Marshal(cluster)
	if err != nil {
		// In the unlikely event of an error, return an empty config rather than partially populated data.
		// Callers should treat a nil cluster input as the primary "no config" signal.
		return out
	}

	if err := protojson.Unmarshal(bytes, out); err != nil {
		return out
	}

	return out

```

To fully implement the suggestion for both directions, the same pattern should be applied to `convertAPIClusterToStorage`, replacing its manual 1:1 field assignments with a protojson-based structural copy (marshal the `*v1.ClusterConfig` to JSON, then unmarshal into `*storage.Cluster`). This will ensure that both conversion functions automatically handle newly added fields as long as the proto messages remain structurally compatible.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment on lines +11 to +20
func convertStorageClusterToAPI(cluster *storage.Cluster) *v1.ClusterConfig {
if cluster == nil {
return nil
}
return &v1.ClusterConfig{
Id: cluster.GetId(),
Name: cluster.GetName(),
Type: cluster.GetType(),
Labels: cluster.GetLabels(),
MainImage: cluster.GetMainImage(),
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: Manual 1:1 field mapping in both conversion functions is fragile and may drift as proto definitions evolve.

Both convertStorageClusterToAPI and convertAPIClusterToStorage manually copy every field, which is easy to miss when new fields are added, contradicting the comment that all fields are copied. Please consider a more robust pattern (shared helper, codegen, or tests/compile-time checks such as round-trip proto.Equal) so new fields can’t be accidentally omitted.

Suggested implementation:

import (
	v1 "github.com/stackrox/rox/generated/api/v1"
	"github.com/stackrox/rox/generated/storage"
	"google.golang.org/protobuf/encoding/protojson"
)
 // convertStorageClusterToAPI converts a storage.Cluster to the API representation.
 // All fields are copied 1:1 via protojson. The API type is structurally identical to the storage
 // type today; they are separated to allow independent evolution in the future. Using protojson
 // reduces the risk of fields drifting as proto definitions evolve.
func convertStorageClusterToAPI(cluster *storage.Cluster) *v1.ClusterConfig {
	if cluster == nil {
		return nil
	}

	out := &v1.ClusterConfig{}

	// Use protojson to perform a structural copy so newly-added fields are automatically handled
	// as long as the underlying proto definitions remain structurally compatible.
	bytes, err := protojson.Marshal(cluster)
	if err != nil {
		// In the unlikely event of an error, return an empty config rather than partially populated data.
		// Callers should treat a nil cluster input as the primary "no config" signal.
		return out
	}

	if err := protojson.Unmarshal(bytes, out); err != nil {
		return out
	}

	return out

To fully implement the suggestion for both directions, the same pattern should be applied to convertAPIClusterToStorage, replacing its manual 1:1 field assignments with a protojson-based structural copy (marshal the *v1.ClusterConfig to JSON, then unmarshal into *storage.Cluster). This will ensure that both conversion functions automatically handle newly added fields as long as the proto messages remain structurally compatible.

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Apr 3, 2026

📝 Walkthrough

Walkthrough

Introduces a new API type v1.ClusterConfig and conversion helpers between storage Cluster and API ClusterConfig. Service handlers, client interfaces, and CLI/roxctl sensor commands are updated to use the API type; API→storage conversion omits server-managed fields.

Changes

Cohort / File(s) Summary
Proto API Definition
proto/api/v1/cluster_service.proto
Added ClusterConfig message; updated PostCluster/PutCluster request types and ClusterResponse.cluster / ClustersList.clusters to use ClusterConfig instead of storage.Cluster.
Service Layer Conversions
central/cluster/service/convert.go, central/cluster/service/convert_test.go
Added conversion helpers: convertStorageClusterToAPI, convertStorageClustersToAPI, convertAPIClusterToStorage; tests validating nil handling, field mapping, and discarding of server-managed fields.
Cluster Service Implementation
central/cluster/service/service_impl.go
Changed PostCluster/PutCluster signatures to accept *v1.ClusterConfig; handlers convert API config → storage model before persistence and convert stored clusters → API models for responses.
Config Controller Client
config-controller/pkg/client/client.go, config-controller/pkg/client/mocks/client.go, config-controller/pkg/client/client_test.go
CentralClient.ListClusters (and mocks/tests) now return []*v1.ClusterConfig instead of []*storage.Cluster; adjusted implementations and tests accordingly.
roxctl Cluster Delete
roxctl/cluster/delete/delete.go, roxctl/cluster/delete/delete_test.go
CLI delete command and tests switched to operate on *v1.ClusterConfig (removed unused storage import and adapted mocks/helpers).
roxctl Sensor Generate
roxctl/sensor/generate/convert.go, roxctl/sensor/generate/generate.go, roxctl/sensor/generate/generate_test.go, roxctl/sensor/generate/k8s.go, roxctl/sensor/generate/openshift.go, roxctl/sensor/generate/convert_test.go
Added clusterConfigToStorage conversion; sensor generation command internals now use *v1.ClusterConfig; validation calls updated to validate converted storage representation; tests updated/added.

Sequence Diagram

sequenceDiagram
    participant Client
    participant Service as Cluster Service
    participant Convert as Conversion Layer
    participant Storage as Datastore

    Client->>Service: PostCluster(v1.ClusterConfig)
    Service->>Convert: convertAPIClusterToStorage(config)
    Convert-->>Service: storage.Cluster
    Service->>Storage: AddCluster(storage.Cluster)
    Storage-->>Service: id
    Service->>Storage: GetCluster(id)
    Storage-->>Service: storage.Cluster
    Service->>Convert: convertStorageClusterToAPI(storage.Cluster)
    Convert-->>Service: v1.ClusterConfig
    Service-->>Client: ClusterResponse(v1.ClusterConfig)
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

🚥 Pre-merge checks | ✅ 1 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 17.65% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Description check ⚠️ Warning The PR description provides clear objectives and validation details but omits critical sections from the template. Add the 'How I validated my change' section with detailed explanation of validation approach, and fill in the generic 'change me!' placeholder in the Description section with substantive details about the refactoring.
✅ Passed checks (1 passed)
Check name Status Explanation
Title check ✅ Passed The title 'refactor(chore): Decouple v1 API from storage.Cluster protobuf' clearly and concisely summarizes the main architectural change: decoupling the v1 API from storage.Cluster by introducing a dedicated v1.ClusterConfig message.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch vjwilson/refactor-cluster-storage-proto

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (2)
roxctl/sensor/generate/k8s.go (1)

33-35: Keep validation and submission in lockstep.

Roxctl now validates a synthesized storage.Cluster, but later submits the original v1.ClusterConfig. That adds a second API↔storage mapping alongside the server-side converter, so any missed field in clusterConfigToStorage will make local validation disagree with what Central actually receives. Please either share the mapping or add exhaustive parity tests around the converter.

As per coding guidelines, focus on major issues impacting performance, readability, maintainability and security.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@roxctl/sensor/generate/k8s.go` around lines 33 - 35, Validation uses
clusterConfigToStorage(k8sCommand.cluster) but the code later submits the
original v1.ClusterConfig, causing possible mismatches; convert once and keep in
lockstep by validating and submitting the same storage.Cluster instance returned
by clusterConfigToStorage(k8sCommand.cluster) (or else validate the v1 type
using the exact server-side converter), and add exhaustive parity tests for
clusterConfigToStorage to ensure every field is mapped identically to the server
converter; refer to clusterValidation.ValidatePartial, clusterConfigToStorage,
and k8sCommand.cluster when making this change.
proto/api/v1/cluster_service.proto (1)

26-94: Add a descriptor-parity test to guard against future drift.

ClusterConfig is currently field-number-identical to storage.Cluster, but there is no automated check preventing this contract from diverging. A future refactoring could alter field numbers, types, or reserved ranges in one message without catching the mismatch, breaking version-skewed Central/roxctl/Sensor interop at runtime. Adding a presubmit or CI check that validates structural parity would catch such regressions early.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@proto/api/v1/cluster_service.proto` around lines 26 - 94, Add a
descriptor-parity presubmit/CI test that verifies ClusterConfig is
field-number-identical to storage.Cluster: implement a test (e.g.,
TestClusterConfigDescriptorParity) that loads the protobuf descriptors for
message names "ClusterConfig" and "storage.Cluster" and asserts equality of each
field's number, name, type, label (singular/repeated), and reserved tag ranges;
also assert parity for map fields (labels, audit_log_state) and repeated
sensor_capabilities and that reserved tags (e.g., reserved 6,8,9-12,14) match.
Hook this test into the existing proto test suite so any future changes to
ClusterConfig or storage.Cluster fail CI until explicitly reconciled.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@proto/api/v1/cluster_service.proto`:
- Around line 26-94: Add a descriptor-parity presubmit/CI test that verifies
ClusterConfig is field-number-identical to storage.Cluster: implement a test
(e.g., TestClusterConfigDescriptorParity) that loads the protobuf descriptors
for message names "ClusterConfig" and "storage.Cluster" and asserts equality of
each field's number, name, type, label (singular/repeated), and reserved tag
ranges; also assert parity for map fields (labels, audit_log_state) and repeated
sensor_capabilities and that reserved tags (e.g., reserved 6,8,9-12,14) match.
Hook this test into the existing proto test suite so any future changes to
ClusterConfig or storage.Cluster fail CI until explicitly reconciled.

In `@roxctl/sensor/generate/k8s.go`:
- Around line 33-35: Validation uses clusterConfigToStorage(k8sCommand.cluster)
but the code later submits the original v1.ClusterConfig, causing possible
mismatches; convert once and keep in lockstep by validating and submitting the
same storage.Cluster instance returned by
clusterConfigToStorage(k8sCommand.cluster) (or else validate the v1 type using
the exact server-side converter), and add exhaustive parity tests for
clusterConfigToStorage to ensure every field is mapped identically to the server
converter; refer to clusterValidation.ValidatePartial, clusterConfigToStorage,
and k8sCommand.cluster when making this change.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Central YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Pro

Run ID: 0cfeadff-3685-4ba7-9dce-7e5c7151ca09

📥 Commits

Reviewing files that changed from the base of the PR and between 2e6eff0 and 9f9a3e5.

⛔ Files ignored due to path filters (6)
  • generated/api/v1/cluster_service.pb.go is excluded by !**/*.pb.go, !**/generated/**
  • generated/api/v1/cluster_service.pb.gw.go is excluded by !**/*.pb.gw.go, !**/generated/**
  • generated/api/v1/cluster_service.swagger.json is excluded by !**/generated/**
  • generated/api/v1/cluster_service_grpc.pb.go is excluded by !**/*.pb.go, !**/generated/**
  • generated/api/v1/cluster_service_vtproto.pb.go is excluded by !**/*.pb.go, !**/generated/**
  • generated/internalapi/central/v1/token_service.pb.go is excluded by !**/*.pb.go, !**/generated/**
📒 Files selected for processing (13)
  • central/cluster/service/convert.go
  • central/cluster/service/service_impl.go
  • config-controller/pkg/client/client.go
  • config-controller/pkg/client/client_test.go
  • config-controller/pkg/client/mocks/client.go
  • proto/api/v1/cluster_service.proto
  • roxctl/cluster/delete/delete.go
  • roxctl/cluster/delete/delete_test.go
  • roxctl/sensor/generate/convert.go
  • roxctl/sensor/generate/generate.go
  • roxctl/sensor/generate/generate_test.go
  • roxctl/sensor/generate/k8s.go
  • roxctl/sensor/generate/openshift.go

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 3, 2026

🚀 Build Images Ready

Images are ready for commit 38e59a0. To use with deploy scripts:

export MAIN_IMAGE_TAG=4.11.x-560-g38e59a0dc9

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@central/cluster/service/convert_test.go`:
- Around line 65-67: The test sets AuditLogState in the input fixture but never
verifies it in the "AllFields" test; update the TestAllFields (or the test
function that builds the input struct in convert_test.go) to assert that the
converted output contains the same AuditLogState map (keys and
storage.AuditLogFileState values). Use the same comparison style used elsewhere
in the test (e.g. require.Equal/reflect.DeepEqual/cmp) to compare the expected
AuditLogState against the actual result from the conversion function,
referencing the AuditLogState field and storage.AuditLogFileState type so future
regressions are caught.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Central YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Pro

Run ID: 35e087e5-4f43-4841-b55c-394ed16ccd10

📥 Commits

Reviewing files that changed from the base of the PR and between 9f9a3e5 and 460e71f.

📒 Files selected for processing (2)
  • central/cluster/service/convert_test.go
  • roxctl/sensor/generate/convert_test.go

Comment on lines +65 to +67
AuditLogState: map[string]*storage.AuditLogFileState{
"node1": {CollectLogsSince: nil},
},
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

AuditLogState is populated but never asserted in the “AllFields” test.

Line 65 sets AuditLogState, but the verification block does not check it. That leaves this field unprotected against conversion regressions.

Suggested test fix
@@
 	protoassert.Equal(t, cluster.GetMostRecentSensorId(), result.GetMostRecentSensorId())
+	assert.Equal(t, cluster.GetAuditLogState(), result.GetAuditLogState())
 	assert.Equal(t, cluster.GetInitBundleId(), result.GetInitBundleId())
 	assert.Equal(t, cluster.GetSensorCapabilities(), result.GetSensorCapabilities())

Also applies to: 74-97

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@central/cluster/service/convert_test.go` around lines 65 - 67, The test sets
AuditLogState in the input fixture but never verifies it in the "AllFields"
test; update the TestAllFields (or the test function that builds the input
struct in convert_test.go) to assert that the converted output contains the same
AuditLogState map (keys and storage.AuditLogFileState values). Use the same
comparison style used elsewhere in the test (e.g.
require.Equal/reflect.DeepEqual/cmp) to compare the expected AuditLogState
against the actual result from the conversion function, referencing the
AuditLogState field and storage.AuditLogFileState type so future regressions are
caught.

@codecov
Copy link
Copy Markdown

codecov bot commented Apr 3, 2026

Codecov Report

❌ Patch coverage is 88.39286% with 13 lines in your changes missing coverage. Please review.
✅ Project coverage is 49.62%. Comparing base (b61d01e) to head (38e59a0).
⚠️ Report is 3 commits behind head on master.

Files with missing lines Patch % Lines
central/cluster/service/service_impl.go 22.22% 7 Missing ⚠️
config-controller/pkg/client/client.go 0.00% 2 Missing ⚠️
roxctl/sensor/generate/generate.go 0.00% 2 Missing ⚠️
roxctl/sensor/generate/k8s.go 0.00% 1 Missing ⚠️
roxctl/sensor/generate/openshift.go 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master   #19824      +/-   ##
==========================================
+ Coverage   49.60%   49.62%   +0.02%     
==========================================
  Files        2763     2765       +2     
  Lines      208271   208435     +164     
==========================================
+ Hits       103312   103436     +124     
- Misses      97292    97333      +41     
+ Partials     7667     7666       -1     
Flag Coverage Δ
go-unit-tests 49.62% <88.39%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant