ROX-33511: Fix cluster selection by name + new cluster ID selection#19351
ROX-33511: Fix cluster selection by name + new cluster ID selection#19351rhybrillou wants to merge 7 commits intomasterfrom
Conversation
|
Skipping CI for Draft Pull Request. |
There was a problem hiding this comment.
Hey - I've found 3 issues, and left some high level feedback:
- The new
selectors/convertRulesToSelectorspath largely duplicates the oldconvertRulesToLabelSelectorslogic; consider consolidating or removing the older helper to avoid divergence and reduce maintenance overhead. - In
convertRulesToSelectors, you eagerly allocatenamespacesByClusterIDandnamespacesByClusterNameeven when there are no included namespaces; you could lazily initialize these maps inaddToNamespaceMapto keep the zero-value ofselectorslighter and simplify checks likematchNamespaceByClusterKey.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- The new `selectors`/`convertRulesToSelectors` path largely duplicates the old `convertRulesToLabelSelectors` logic; consider consolidating or removing the older helper to avoid divergence and reduce maintenance overhead.
- In `convertRulesToSelectors`, you eagerly allocate `namespacesByClusterID` and `namespacesByClusterName` even when there are no included namespaces; you could lazily initialize these maps in `addToNamespaceMap` to keep the zero-value of `selectors` lighter and simplify checks like `matchNamespaceByClusterKey`.
## Individual Comments
### Comment 1
<location path="pkg/sac/effectiveaccessscope/effective_access_scope.go" line_range="275-284" />
<code_context>
+ // Set the cluster state to the pre-existing state
</code_context>
<issue_to_address>
**suggestion (bug_risk):** Cluster state aggregation keeps max state but always overwrites attributes with the latest cluster
With duplicate cluster names, `clusterState` is computed as the max of the prior state and this cluster’s state, but attributes always come from the most recently seen cluster. This can produce a mismatch where the state reflects one cluster instance while the attributes come from another. If attributes are meant to describe the cluster that determined the final (max) state, consider only updating attributes when this cluster changes `clusterState`, or explicitly define that “last cluster wins” for both state and attributes and avoid reusing the prior state.
Suggested implementation:
```golang
clusterName := cluster.GetName()
clusterID := cluster.GetId()
clusterState := Excluded
previousClusterState := Excluded
// There is no need to check if root is Included as we start with Excluded root.
// If root is eventually Included then we can include the cluster and short-circuit:
// no need to match if parent is included.
// Capture the pre-existing state, if any, so we can decide later whether to
// update attributes based on whether this cluster changes the aggregated state.
if clusterSubTree := root.Clusters[clusterName]; clusterSubTree != nil {
previousClusterState = clusterSubTree.State
clusterState = clusterSubTree.State
}
// Augment cluster labels with cluster's name.
augmentedClusterLabels := augmentLabels(cluster.GetLabels(), clusterNameLabel, clusterName)
```
You’ll need to adjust the code later in this function where the aggregated `clusterState` and attributes are written back to `root.Clusters[clusterName]` (or the corresponding subtree struct), along these lines:
1. After you compute the new aggregated state for this cluster (e.g., something like `clusterState = max(clusterState, thisClusterState)`), compare it to `previousClusterState`.
2. Only overwrite attributes for this cluster when `clusterState` is strictly greater than `previousClusterState`. For example (pseudo-code, adjust to the actual types/fields):
```go
// After computing final clusterState for this cluster:
subTree, ok := root.Clusters[clusterName]
if !ok {
subTree = &ScopeTreeNode{}
root.Clusters[clusterName] = subTree
}
subTree.State = clusterState
// Only update attributes if this cluster actually raised the aggregate state.
if clusterState > previousClusterState {
subTree.Attributes = computedAttributesForThisCluster
// or whatever fields/labels you currently set from this cluster
}
```
3. If your state type is not an ordered enum, replace the `>` comparison with your existing “max”/ordering logic, but still keep the semantics: update attributes only when this cluster is the one determining the final (max) state.
This keeps `clusterState` as the max over all clusters with the same name while ensuring attributes correspond to the cluster instance that actually determined that max state.
</issue_to_address>
### Comment 2
<location path="pkg/sac/effectiveaccessscope/conversion.go" line_range="132-140" />
<code_context>
+ includedNamespaces := scopeRules.GetIncludedNamespaces()
+ output.namespacesByClusterID = make(map[string]map[string]bool)
+ output.namespacesByClusterName = make(map[string]map[string]bool)
+ for _, namespace := range includedNamespaces {
+ clusterID := namespace.GetClusterId()
+ clusterName := namespace.GetClusterName()
+ namespaceName := namespace.GetNamespaceName()
+ if clusterID != "" {
+ addToNamespaceMap(output.namespacesByClusterID, clusterID, namespaceName)
+ continue
+ }
+ addToNamespaceMap(output.namespacesByClusterName, clusterName, namespaceName)
+ }
+
</code_context>
<issue_to_address>
**issue:** Included namespaces with neither cluster ID nor name end up under an empty cluster key
With an empty `clusterId`, the code falls back to `clusterName`, which may also be empty, creating an entry under the `""` key in `namespacesByClusterName`. If namespaces without any cluster identifier are not meaningful, consider skipping them or treating them as invalid instead of aggregating them under an empty string key.
</issue_to_address>
### Comment 3
<location path="pkg/sac/effectiveaccessscope/effective_access_scope.go" line_range="262" />
<code_context>
-// name exist.
-func (root *ScopeTree) populateStateForCluster(cluster Cluster, clusterSelectors []labels.Selector, detail v1.ComputeEffectiveAccessScopeRequest_Detail) {
+// The highest selection level across observed clusters with the same name should be kept.
+func (root *ScopeTree) populateStateForCluster(
+ cluster Cluster,
+ ruleSelectors *selectors,
</code_context>
<issue_to_address>
**issue (complexity):** Consider moving cluster and namespace matching logic into methods on the new `selectors` type and using a shared `maxScopeState` helper so that `populateStateForCluster`/`populateStateForNamespace` focus only on updating the tree.
You can keep all the new functionality while reducing duplication and branching by pushing the matching logic into `selectors` and centralizing state merging.
### 1. Encapsulate cluster matching in `selectors`
Move the ID/name/label logic out of `populateStateForCluster`:
```go
func (s *selectors) matchCluster(c Cluster) scopeState {
clusterName := c.GetName()
clusterID := c.GetId()
// Augment cluster labels with cluster's name.
augmented := augmentLabels(c.GetLabels(), clusterNameLabel, clusterName)
state := Excluded
if s.clustersByID[clusterID] {
state = Included
}
if s.clustersByName[clusterName] {
state = Included
}
state = maxScopeState(state, matchLabels(s.clustersByLabel, augmented))
return state
}
func maxScopeState(states ...scopeState) scopeState {
max := Excluded
for _, s := range states {
if s > max {
max = s
}
}
return max
}
```
Then `populateStateForCluster` becomes mostly tree-update logic:
```go
func (root *ScopeTree) populateStateForCluster(
cluster Cluster,
ruleSelectors *selectors,
detail v1.ComputeEffectiveAccessScopeRequest_Detail,
) {
clusterName := cluster.GetName()
// Start from previous state if present
state := Excluded
if subTree := root.Clusters[clusterName]; subTree != nil {
state = subTree.State
}
// Merge with selector result
state = maxScopeState(state, ruleSelectors.matchCluster(cluster))
root.Clusters[clusterName] = newClusterScopeSubTree(state, nodeAttributesForCluster(cluster, detail))
root.clusterIDToName[cluster.GetId()] = clusterName
}
```
### 2. Encapsulate namespace matching in `selectors`
Similarly, hide the namespace ID/name/label composition in `selectors`. You can move `matchNamespaceByClusterKey` into the same type:
```go
func (s *selectors) matchNamespace(ns Namespace) scopeState {
clusterID := ns.GetClusterId()
clusterName := ns.GetClusterName()
nsName := ns.GetName()
nsFQSN := getNamespaceFQSN(clusterName, nsName)
labels := augmentLabels(ns.GetLabels(), namespaceNameLabel, nsFQSN)
state := Excluded
if s.matchNamespaceByClusterKey(s.namespacesByClusterID, clusterID, nsName) {
state = Included
}
if s.matchNamespaceByClusterKey(s.namespacesByClusterName, clusterName, nsName) {
state = Included
}
state = maxScopeState(state, matchLabels(s.namespacesByLabel, labels))
return state
}
func (s *selectors) matchNamespaceByClusterKey(
targetMap map[string]map[string]bool,
clusterKey, namespaceKey string,
) bool {
if targetMap == nil {
return false
}
m, ok := targetMap[clusterKey]
if !ok {
return false
}
return m[namespaceKey]
}
```
Then `populateStateForNamespace` reduces to:
```go
func (cluster *clustersScopeSubTree) populateStateForNamespace(
namespace Namespace,
ruleSelectors *selectors,
detail v1.ComputeEffectiveAccessScopeRequest_Detail,
) {
nsName := namespace.GetName()
// Parent Included still short-circuits
if cluster.State == Included {
cluster.Namespaces[nsName] = newNamespacesScopeSubTree(Included, nodeAttributesForNamespace(namespace, detail))
return
}
nsState := ruleSelectors.matchNamespace(namespace)
cluster.Namespaces[nsName] = newNamespacesScopeSubTree(nsState, nodeAttributesForNamespace(namespace, detail))
}
```
This keeps all new capabilities (ID-based and explicit namespace rules) but:
- removes duplicated precedence logic from the callers,
- keeps `selectors` as the single place that knows about its internal maps/slices,
- makes `populateStateForCluster` and `populateStateForNamespace` primarily about tree updates rather than rule evaluation.
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
|
Images are ready for the commit at 2fcd37a. To use with deploy scripts, first |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #19351 +/- ##
==========================================
- Coverage 49.71% 49.28% -0.43%
==========================================
Files 2701 2726 +25
Lines 203453 205696 +2243
==========================================
+ Hits 101140 101376 +236
- Misses 94786 96785 +1999
- Partials 7527 7535 +8
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Hey - I've left some high level feedback:
- In
ScopeTree.populateStateForCluster, when a cluster subtree already exists you return early after possibly updating its state but never updateclusterIDToName, so additional clusters with the same name but different IDs won't be reachable viaGetClusterByID; consider always addingclusterIDToName[clusterID] = clusterNameeven when the subtree already exists. - In
TestSelectorsMatchNamespace, the test case description "namespace NOT matched by label is included" contradicts the expectedExcludedstate; updating the description to match the actual expectation would make the test intent clearer.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- In `ScopeTree.populateStateForCluster`, when a cluster subtree already exists you return early after possibly updating its state but never update `clusterIDToName`, so additional clusters with the same name but different IDs won't be reachable via `GetClusterByID`; consider always adding `clusterIDToName[clusterID] = clusterName` even when the subtree already exists.
- In `TestSelectorsMatchNamespace`, the test case description "namespace NOT matched by label is included" contradicts the expected `Excluded` state; updating the description to match the actual expectation would make the test intent clearer.Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
| // The namespace field and at least one cluster field must be set. | ||
| // The cluster ID takes precedence over the cluster name when a cluster with that ID exists. |
There was a problem hiding this comment.
pkg/accessscope/validate.go needs an update then.
proto/storage/role.proto
Outdated
| repeated string included_clusters = 1; | ||
| repeated string included_cluster_ids = 5; |
There was a problem hiding this comment.
We need to explain, how these will be handled:
- if there is anything in ids, named clusters are ignored?
- union of two sets?
There was a problem hiding this comment.
The current implementation is union of two sets.
There was a problem hiding this comment.
It is still consistent with the statement a few lines above (included_cluster_ids being one more repeated field in the rules):
// Each element of any repeated field is an individual rule. Rules are
// joined by logical OR: if there exists a rule allowing resource `x`,
// `x` is in the access scope.
|
/retest |
| namespacesByClusterID map[string]map[string]bool | ||
| namespacesByClusterName map[string]map[string]bool |
There was a problem hiding this comment.
| namespacesByClusterID map[string]map[string]bool | |
| namespacesByClusterName map[string]map[string]bool | |
| namespacesByClusterID map[string]map[string]struct{} | |
| namespacesByClusterName map[string]map[string]struct{} |
There was a problem hiding this comment.
| namespacesByClusterID map[string]map[string]bool | |
| namespacesByClusterName map[string]map[string]bool | |
| namespacesByClusterID map[string]set.StringSet | |
| namespacesByClusterName map[string]set.StringSet |
| clustersByID map[string]bool | ||
| clustersByName map[string]bool |
There was a problem hiding this comment.
| clustersByID map[string]bool | |
| clustersByName map[string]bool | |
| clustersByID set.StringSet | |
| clustersByName set.StringSet |
| namespace: nsAtreides, | ||
| expected: Included, | ||
| }, | ||
| "namespace NOT matched by label is included": { |
There was a problem hiding this comment.
| "namespace NOT matched by label is included": { | |
| "namespace NOT matched by label is excluded": { |
1cc8450 to
3c9d5b1
Compare
Description
Cluster selection by name for clusters whose name does not comply with the Kubernetes syntax constraints break.
This PR aims at fixing this by changing the effective access scope computation and how cluster selection rules are applied.
The change also introduces the ability to perform selection by cluster ID rather than by cluster name. This new selection method allows for increased safety for token requested by sensor on a given cluster against cluster replacement by another with the same name on Central side.
User-facing documentation
is updated ORupdate is not neededis created and is linked above ORis not neededTesting and quality
Automated testing
How I validated my change
Manual testing
my=cluster).Log in with the configured AuthProvider in order to be granted the impacted role.
Open the ACS main dashboard.
With the previous behaviour, the dashboard displays no data as the lookup queries return errors.
With the fixed behaviour, the dashboard displays the expected data.
