Skip to content

refactor: optimize custom metric aggregator memory#18319

Merged
parametalol merged 2 commits intomasterfrom
michael/refactor-aggregator-memory
Jan 13, 2026
Merged

refactor: optimize custom metric aggregator memory#18319
parametalol merged 2 commits intomasterfrom
michael/refactor-aggregator-memory

Conversation

@parametalol
Copy link
Contributor

@parametalol parametalol commented Dec 22, 2025

Description

An aggregator allocates a map of maps for counting the findings and populating the metrics output.

We don't need to do it actually on every gathering, so this PR optimizes aggregator initialization to create it once
per gatherer.

This will also help for future trackers of type counter, where it would be even more expensive to create an aggregator
on each increment event.

User-facing documentation

Testing and quality

  • the change is production ready: the change is GA, or otherwise the functionality is gated by a feature flag
  • CI results are inspected

Automated testing

  • added unit tests
  • added e2e tests
  • added regression tests
  • added compatibility tests
  • modified existing tests

How I validated my change

Benchmark Results

The benchmarks (not committed) compare the old approach (creating new aggregator per cycle) vs the new approach (reusing with reset):

BenchmarkAggregator_CreateNew-8        	  18452	    63012 ns/op	  21808 B/op	    229 allocs/op
BenchmarkAggregator_ReuseWithReset-8   	  29622	    48039 ns/op	  18504 B/op	    203 allocs/op
Metric CreateNew (old) ReuseWithReset (new) Improvement
Time 63,012 ns/op 48,039 ns/op 24% faster
Memory 21,808 B/op 18,504 B/op 15% less memory
Allocations 229 allocs/op 203 allocs/op 11% fewer allocs

The optimization in commit e92df27 saves approximately:

  • 3.3 KB of allocations per gather cycle
  • 26 allocations per gather cycle
  • Reduces GC pressure over time since the outer map and inner maps are reused

@rhacs-bot
Copy link
Contributor

rhacs-bot commented Dec 22, 2025

Images are ready for the commit at 680c2be.

To use with deploy scripts, first export MAIN_IMAGE_TAG=4.10.x-765-g680c2bebdf.

@codecov
Copy link

codecov bot commented Dec 22, 2025

Codecov Report

❌ Patch coverage is 97.14286% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 48.93%. Comparing base (8973035) to head (680c2be).
⚠️ Report is 8 commits behind head on master.

Files with missing lines Patch % Lines
central/metrics/custom/tracker/tracker_base.go 96.42% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##           master   #18319   +/-   ##
=======================================
  Coverage   48.92%   48.93%           
=======================================
  Files        2629     2629           
  Lines      197814   197826   +12     
=======================================
+ Hits        96787    96797   +10     
- Misses      93642    93643    +1     
- Partials     7385     7386    +1     
Flag Coverage Δ
go-unit-tests 48.93% <97.14%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 2 issues

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location> `central/metrics/custom/tracker/tracker_base.go:311-312` </location>
<code_context>
 		if !gr.trySetRunning() {
 			return nil
 		}
+		// Recreate aggregator if config has changed since last run.
+		if gr.config != cfg {
+			gr.aggregator = makeAggregator(cfg.metrics, cfg.filters, tracker.getters)
+			gr.config = cfg
</code_context>

<issue_to_address>
**suggestion (bug_risk):** Pointer-based config comparison may miss changes if the configuration is mutated in place

This relies on pointer inequality (`gr.config != cfg`) rather than config content or a version, so it only works if callers always replace the `*Configuration` instead of mutating or reusing it. If a `Configuration` is updated in place, the pointers remain equal, the aggregator won’t be rebuilt, and metrics can become stale. Either enforce/clarify immutability of `Configuration` or introduce a version/epoch or other change indicator instead of comparing pointers.

Suggested implementation:

```golang
		// Recreate aggregator if config has changed since last run.
		// Use a configuration version/epoch instead of pointer identity so that
		// in-place configuration mutations are also detected.
		if gr.configVersion != cfg.Version {
			gr.aggregator = makeAggregator(cfg.metrics, cfg.filters, tracker.getters)
			gr.configVersion = cfg.Version
			gr.config = cfg
		}

```

To fully implement this correctly, you will also need to:

1. **Add a version/epoch field to the configuration type** (whatever `cfg` is):
   - For example:
     ```go
     type Configuration struct {
         Version uint64
         // existing fields: metrics, filters, ...
     }
     ```
   - Ensure that any code that mutates the configuration either:
     - Treats it as immutable and creates a new `Configuration` with an incremented `Version`, or
     - Explicitly increments `Version` whenever the configuration is changed in place.

2. **Extend the `gatherer` struct to store the last-seen config version**:
   - In the `gatherer[F]` type definition, add:
     ```go
     configVersion uint64
     config        *Configuration // or the actual config type, if not already present
     ```
     (If `config` is already present in the struct, just add `configVersion`.)

3. **Initialize `configVersion` when a new gatherer is created**:
   - Wherever a new `gatherer` is constructed (likely in the same file), set:
     ```go
     gr.configVersion = cfg.Version
     gr.config = cfg
     ```
   - This ensures the first run starts with the correct baseline version.

4. **Update all existing references to config comparison**:
   - If there are other places that compare `gr.config` and `cfg` by pointer (`gr.config != cfg` or `==`), update them to compare `configVersion` vs `cfg.Version` instead, or remove redundant checks if they become unnecessary.

These additional changes ensure the aggregator is safely rebuilt whenever the configuration changes, regardless of whether the change is via replacement or in-place mutation.
</issue_to_address>

### Comment 2
<location> `central/metrics/custom/tracker/tracker_base_test.go:360-369` </location>
<code_context>
+func TestTrackerBase_Gather_afterReconfiguration(t *testing.T) {
</code_context>

<issue_to_address>
**suggestion (testing):** Also assert that old metrics are no longer gathered after reconfiguration

To fully validate the unregister path and aggregator re-creation, please also assert that metrics from the initial configuration (e.g., keys from `md1`) are absent from `newMetrics`. This will confirm we’re not emitting stale metrics after a config change and that old aggregations are correctly cleaned up.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@parametalol parametalol force-pushed the michael/refactor-aggregator-memory branch from e92df27 to 680c2be Compare January 13, 2026 14:00
@parametalol parametalol merged commit 70e1ba4 into master Jan 13, 2026
91 checks passed
@parametalol parametalol deleted the michael/refactor-aggregator-memory branch January 13, 2026 16:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants