ROX-32679: Preset process filter levels#18556
Conversation
|
Skipping CI for Draft Pull Request. |
There was a problem hiding this comment.
Hey - I've found 2 issues, and left some high level feedback:
- In
GetProcessFilterModeConfig, the comment says it returns nil if the mode is invalid or not set, but the implementation returns a default config for invalid/empty values and only returns nil when the env var is unset; consider aligning the comment and behavior (and clarifying interaction with theWithDefault("default")setting). - The mode presets (
defaultConfig,aggressiveConfig,minimalConfig) are constructed on every call toGetProcessFilterModeConfig; consider defining them as package-level vars or a map keyed by mode to avoid duplication and make it easier to add or adjust modes in the future. - The handling of
warnStrdiffers between central and sensor singletons (log.Warnvslog.Info), which may be confusing for operators; consider standardizing the log level or adding a short comment if the difference is intentional.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- In `GetProcessFilterModeConfig`, the comment says it returns nil if the mode is invalid or not set, but the implementation returns a default config for invalid/empty values and only returns nil when the env var is unset; consider aligning the comment and behavior (and clarifying interaction with the `WithDefault("default")` setting).
- The mode presets (`defaultConfig`, `aggressiveConfig`, `minimalConfig`) are constructed on every call to `GetProcessFilterModeConfig`; consider defining them as package-level vars or a map keyed by mode to avoid duplication and make it easier to add or adjust modes in the future.
- The handling of `warnStr` differs between central and sensor singletons (`log.Warn` vs `log.Info`), which may be confusing for operators; consider standardizing the log level or adding a short comment if the difference is intentional.
## Individual Comments
### Comment 1
<location> `central/processindicator/filter/singleton.go:28-29` </location>
<code_context>
}
- singletonFilter = filter.NewFilter(maxExactPathMatches, maxUniqueProcesses, fanOutLevels)
+
+ log.Infof("Process filter configuration: mode=%s, maxExactPathMatches=%d, fanOutLevels=%v, maxProcessPaths=%d",
+ env.ProcessFilterMode.Setting(), config.MaxExactPathMatches, config.FanOutLevels, config.MaxProcessPaths)
+
+ singletonFilter = filter.NewFilter(config.MaxExactPathMatches, config.MaxProcessPaths, config.FanOutLevels)
</code_context>
<issue_to_address>
**issue:** Logging `ProcessFilterMode.Setting()` may misrepresent the active mode when the env var is unset
`GetProcessFilterModeConfig` only applies a mode when `ROX_PROCESS_FILTER_MODE` is *set*, but this log always prints `env.ProcessFilterMode.Setting()`, which will usually be the default value even when no preset is applied. This can suggest a mode is active when behavior is actually driven by individual settings. Please either log the raw env value / whether a preset was applied, or adjust `GetProcessFilterModeConfig` so the default mode is treated as active when the env var is absent, to keep logging consistent with behavior.
</issue_to_address>
### Comment 2
<location> `pkg/env/process_filter_mode.go:29` </location>
<code_context>
+ MaxProcessPaths int
+}
+
+// GetProcessFilterModeConfig returns the configuration for the current filter mode.
+// Returns nil if the mode is invalid or not set.
+func GetProcessFilterModeConfig() (*ProcessFilterModeConfig, string) {
</code_context>
<issue_to_address>
**issue (complexity):** Consider refactoring the mode configuration and warning handling into reusable helpers and switch-based logic to reduce branching and clarify behavior.
You can simplify the mode handling and warning merging without changing behavior.
### 1. Use static presets + a `switch` for mode selection
Define presets once at package level and simplify `GetProcessFilterModeConfig` with a `switch`. This keeps behavior identical (including the “env set but empty string” case) but flattens branching and avoids per-call allocations:
```go
var (
defaultProcessFilterModeConfig = ProcessFilterModeConfig{
MaxExactPathMatches: 5,
FanOutLevels: []int{8, 6, 4, 2},
MaxProcessPaths: 5000,
}
aggressiveProcessFilterModeConfig = ProcessFilterModeConfig{
MaxExactPathMatches: 1,
FanOutLevels: []int{},
MaxProcessPaths: 1000,
}
minimalProcessFilterModeConfig = ProcessFilterModeConfig{
MaxExactPathMatches: 100,
FanOutLevels: []int{20, 15, 10, 5},
MaxProcessPaths: 20000,
}
)
func GetProcessFilterModeConfig() (*ProcessFilterModeConfig, string) {
modeRaw, ok := os.LookupEnv(ProcessFilterMode.EnvVar())
if !ok {
// Mode not set at all
return nil, ""
}
mode := strings.ToLower(modeRaw)
switch mode {
case "aggressive":
return &aggressiveProcessFilterModeConfig, ""
case "default":
return &defaultProcessFilterModeConfig, ""
case "minimal":
return &minimalProcessFilterModeConfig, ""
case "":
// Env set but empty string
return &defaultProcessFilterModeConfig,
fmt.Sprintf("%s set to empty string. Will use the default.", ProcessFilterMode.EnvVar())
default:
// Env set to unrecognised value
return &defaultProcessFilterModeConfig,
fmt.Sprintf("Unrecognized mode for environment variable %s: %s. Will use the default.",
ProcessFilterMode.EnvVar(), mode)
}
}
```
This removes the multi-branch `if/else` chain and the confusing `else if mode == ""` + trailing `return nil, ""`.
### 2. Simplify warning accumulation in `GetEffectiveProcessFilterConfig`
You can avoid the branchy string concatenation by using a tiny helper that appends non-empty warnings:
```go
func joinWarnings(warns ...string) string {
var nonEmpty []string
for _, w := range warns {
if w != "" {
nonEmpty = append(nonEmpty, w)
}
}
return strings.Join(nonEmpty, "\n")
}
func GetEffectiveProcessFilterConfig() (ProcessFilterModeConfig, string) {
config := ProcessFilterModeConfig{
MaxExactPathMatches: ProcessFilterMaxExactPathMatches.IntegerSetting(),
MaxProcessPaths: ProcessFilterMaxProcessPaths.IntegerSetting(),
}
var fanOutWarnStr string
config.FanOutLevels, fanOutWarnStr = ProcessFilterFanOutLevels.IntegerArraySetting()
modeConfig, modeWarnStr := GetProcessFilterModeConfig()
if modeConfig == nil {
// No valid mode set, return current settings
return config, joinWarnings(fanOutWarnStr, modeWarnStr)
}
// Apply mode preset, but only for values that aren't explicitly overridden
if _, ok := os.LookupEnv(ProcessFilterMaxExactPathMatches.EnvVar()); !ok {
config.MaxExactPathMatches = modeConfig.MaxExactPathMatches
}
if _, ok := os.LookupEnv(ProcessFilterFanOutLevels.EnvVar()); !ok {
config.FanOutLevels = modeConfig.FanOutLevels
}
if _, ok := os.LookupEnv(ProcessFilterMaxProcessPaths.EnvVar()); !ok {
config.MaxProcessPaths = modeConfig.MaxProcessPaths
}
return config, joinWarnings(fanOutWarnStr, modeWarnStr)
}
```
This keeps all existing behavior (order and presence of warnings) but reduces branching and makes the intent clearer.
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
|
Images are ready for the commit at d03c269. To use with deploy scripts, first |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #18556 +/- ##
==========================================
+ Coverage 49.62% 49.63% +0.01%
==========================================
Files 2680 2681 +1
Lines 202214 202251 +37
==========================================
+ Hits 100346 100390 +44
+ Misses 94393 94385 -8
- Partials 7475 7476 +1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Hey - I've found 5 issues, and left some high level feedback:
- The mode presets hardcode default values (e.g., 5 / [8 6 4 2] / 5000) that are also defined in the individual env settings; consider deriving the preset defaults from the existing
ProcessFilter*settings to avoid divergence if someone changes the base defaults later. - In the singleton initializers you call
GetEffectiveProcessFilterConfigand then callGetProcessFilterModeConfigagain just to decide what to log; consider returning the mode (or an indicator) fromGetEffectiveProcessFilterConfigto avoid recomputing the mode configuration.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- The mode presets hardcode default values (e.g., 5 / [8 6 4 2] / 5000) that are also defined in the individual env settings; consider deriving the preset defaults from the existing `ProcessFilter*` settings to avoid divergence if someone changes the base defaults later.
- In the singleton initializers you call `GetEffectiveProcessFilterConfig` and then call `GetProcessFilterModeConfig` again just to decide what to log; consider returning the mode (or an indicator) from `GetEffectiveProcessFilterConfig` to avoid recomputing the mode configuration.
## Individual Comments
### Comment 1
<location> `pkg/env/process_filter_mode.go:37-40` </location>
<code_context>
+ return nil, ""
+ }
+
+ defaultConfig := &ProcessFilterModeConfig{
+ MaxExactPathMatches: 5,
+ FanOutLevels: []int{8, 6, 4, 2},
+ MaxProcessPaths: 5000,
+ }
+
</code_context>
<issue_to_address>
**suggestion:** Avoid duplicating default values already defined in the individual env settings to prevent future drift.
These values are also defined as defaults in `ProcessFilterMaxExactPathMatches`, `ProcessFilterFanOutLevels`, and `ProcessFilterMaxProcessPaths`. If they change in one place but not the other, the mode presets (especially `default`) will no longer match the documented base settings. Please derive these values from the existing env settings (e.g., via their default accessors or shared constants) so there’s a single source of truth.
Suggested implementation:
```golang
_, ok := os.LookupEnv(ProcessFilterMode.EnvVar())
if !ok {
return nil, ""
}
defaultConfig := &ProcessFilterModeConfig{
MaxExactPathMatches: ProcessFilterMaxExactPathMatches.Default(),
FanOutLevels: ProcessFilterFanOutLevels.Default(),
MaxProcessPaths: ProcessFilterMaxProcessPaths.Default(),
}
```
This change assumes that:
1. `ProcessFilterMaxExactPathMatches`, `ProcessFilterFanOutLevels`, and `ProcessFilterMaxProcessPaths` each expose a `Default()` method returning the appropriate Go type (`int`, `[]int`, and `int` respectively).
2. If the actual API differs (e.g., the defaults are exposed via a `DefaultValue()` method, exported constants, or fields), adjust the calls accordingly, e.g.:
- `ProcessFilterMaxExactPathMatches.DefaultValue()`
- `ProcessFilterFanOutLevelsDefault`
- `DefaultProcessFilterMaxProcessPaths`
Make sure the types align with the `ProcessFilterModeConfig` fields (add casts if necessary).
</issue_to_address>
### Comment 2
<location> `central/processindicator/filter/singleton.go:30-34` </location>
<code_context>
- if fanOutLevelsWarning != "" {
- log.Warn(fanOutLevelsWarning)
+ modeStr := ""
+ if modeConfig, _ := env.GetProcessFilterModeConfig(); modeConfig != nil {
+ modeStr = fmt.Sprintf("mode=%s, ", env.ProcessFilterMode.Setting())
}
- singletonFilter = filter.NewFilter(maxExactPathMatches, maxUniqueProcesses, fanOutLevels)
+ log.Infof("Process filter configuration: %smaxExactPathMatches=%d, fanOutLevels=%v, maxProcessPaths=%d",
+ modeStr, config.MaxExactPathMatches, config.FanOutLevels, config.MaxProcessPaths)
+
</code_context>
<issue_to_address>
**issue:** The logged `mode` string can misrepresent the effective configuration when the mode value is invalid or empty.
When `ROX_PROCESS_FILTER_MODE` is invalid or empty, `GetProcessFilterModeConfig` falls back to `defaultConfig`, but `modeStr` is derived directly from `ProcessFilterMode.Setting()`. As a result, the log may show `mode=<invalid>` even though the default preset is applied, which can mislead operators.
Please either log the resolved/normalized mode from `GetProcessFilterModeConfig` or clearly state in the log that the provided mode is invalid and defaults are being used.
</issue_to_address>
### Comment 3
<location> `pkg/env/process_filter_mode_test.go:16` </location>
<code_context>
+ name string
</code_context>
<issue_to_address>
**suggestion (testing):** `expectNil` is never exercised; missing test case for "no mode set" and warning behavior
The `expectNil` flag is never set in the table, so that branch is effectively untested and the "env var not set" case is unverified. Please add a test where `ROX_PROCESS_FILTER_MODE` is unset and assert `config` is `nil` with an empty warning, and also assert on the warning strings for the `"invalid"` and `""` modes. If you decide not to cover the nil case, consider removing `expectNil` from the table to avoid dead fields.
</issue_to_address>
### Comment 4
<location> `pkg/env/process_filter_mode_test.go:206-215` </location>
<code_context>
+ }
+}
+
+func TestIsEnvVarExplicitlySet(t *testing.T) {
+ testVar := "TEST_ENV_VAR_FOR_TESTING"
+
+ // Clean up before and after
+ _ = os.Unsetenv(testVar)
+ defer func() {
+ _ = os.Unsetenv(testVar)
+ }()
+
+ // Test when not set
+ _, ok := os.LookupEnv(testVar)
+ assert.False(t, ok)
+
+ // Test when set to non-empty value
+ err := os.Setenv(testVar, "value")
+ require.NoError(t, err)
+ _, ok = os.LookupEnv(testVar)
+ assert.True(t, ok)
+
+ // Test when set to empty value (should still be considered "set")
</code_context>
<issue_to_address>
**issue (testing):** `TestIsEnvVarExplicitlySet` is misleading and doesn't exercise any project code
This test only asserts `os.LookupEnv` behavior, which is already covered by the standard library and not by your `env` package. The name also suggests a helper `IsEnvVarExplicitlySet`, which doesn't exist here, making the intent unclear.
Consider either removing this test or replacing it with one that exercises `GetEffectiveProcessFilterConfig`’s handling of env vars (e.g., verifying that an empty string still overrides defaults) and asserting on the resulting `ProcessFilterModeConfig`, rather than on `os.LookupEnv` directly.
</issue_to_address>
### Comment 5
<location> `pkg/env/process_filter_mode_test.go:186` </location>
<code_context>
+ // Clean up any existing env vars
</code_context>
<issue_to_address>
**suggestion (testing):** Consider using `t.Setenv` to avoid manual env cleanup and reduce flakiness risk
The tests here manually manage env vars with `os.Setenv`/`os.Unsetenv`, which makes it easy to miss a cleanup and leak state between (sub)tests as more cases are added. Using `t.Setenv` from the `testing` package will automatically restore env vars per test, simplify setup/teardown, and reduce cross-test interference. For example, replace `os.Setenv("ROX_PROCESS_FILTER_MODE", tc.mode)` plus `Unsetenv` with `t.Setenv("ROX_PROCESS_FILTER_MODE", tc.mode)` inside each subtest.
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
|
/test gke-qa-e2e-tests |
vikin91
left a comment
There was a problem hiding this comment.
I reviewed the implementation details and the idea with its complexity.
The idea and the implementation is good - it brings many benefits to the users who would change that setting. The API of accessing that from Central/Sensor is separated and relatively easy to use (I left a comment how we could potentially decrease the complexity).
The things that could be improved:
- Warning logging may be inconsistent due to early returns and many potential states (a value: set, unset, invalid, empty). Merging of multiple warnings could be improved by using - for example - the
multierrorpkg (or simply returning a slice). - Double resolution by requiring a call to two
GetEffective...methods. - Some tests could be tightened (see detailed comments).
I am reviewing it ❌ for now, but it is a solid PR with only few bits to correct.
I haven't checked fully what sourcery-ai suggested, but maybe it would be worth giving it a closer look as two of my comments were already mentioned in the AI review.
vikin91
left a comment
There was a problem hiding this comment.
Thank you for improving! I went over the code again and posted few issues - they are minor but it would be good to address them. Let's get the CI green and I am happy to accept the PR.
Co-authored-by: Piotr Rygielski <114479+vikin91@users.noreply.github.com>
10a7294 to
1dfbc1c
Compare
|
/test ocp-4-20-nongroovy-e2e-tests |
vikin91
left a comment
There was a problem hiding this comment.
Thank you for improving - looks good to me!
Co-authored-by: Piotr Rygielski <114479+vikin91@users.noreply.github.com>
Description
Recently the parameters for process indicator filtering was made configurable. However, that filtering is complicated, and we should not make customers take the time and effort to understand it so that they change it for their purpose. We should make it easier for customers to set the process indicator filtering for their own purposes. To do that a new environment variable
ROX_PROCESS_FILTER_MODEis introduced. That environment variable is essentially an enum with valid values default, aggressive, and minimal. Setting that environment variable sets all three parameters involving process filtering (ROX_PROCESS_FILTER_MAX_EXACT_PATH_MATCHES, ROX_PROCESS_FILTER_FAN_OUT_LEVELS, and ROX_PROCESS_FILTER_MAX_PROCESS_PATHS).User-facing documentation
Testing and quality
Automated testing
How I validated my change
Deployed ACS and saw the following
Edited the sensor deployment yaml configuration
Test overriding one of the parameters
Tested the other levels