Skip to content

perf(proto): Use pool for serialisation#17373

Closed
janisz wants to merge 4 commits intomasterfrom
use_pool_for_serialisation
Closed

perf(proto): Use pool for serialisation#17373
janisz wants to merge 4 commits intomasterfrom
use_pool_for_serialisation

Conversation

@janisz
Copy link
Contributor

@janisz janisz commented Oct 20, 2025

Description

This PR brings a pooling optimisation for store upsert. Instead of marshaling into a new slice of bytes we can use a pool just like grpc does (and using the same layered pool implementation). This should reduce the pressure on GC as less objects will be to collect.

Refs:

master.txtnew.txt                │
                          │     B/opB/op      vs baseMany/upsert_1_alerts-8       14.04Ki ± 0%   11.80Ki ± 0%  -15.97% (p=0.000 n=10)
Many/upsert_2_alerts-8       33.43Ki ± 0%   28.95Ki ± 0%  -13.41% (p=0.000 n=10)
Many/upsert_4_alerts-8       71.98Ki ± 0%   63.02Ki ± 0%  -12.46% (p=0.000 n=10)
Many/upsert_8_alerts-8       171.8Ki ± 0%   153.9Ki ± 0%  -10.44% (p=0.000 n=10)
Many/upsert_16_alerts-8      337.0Ki ± 0%   301.1Ki ± 0%  -10.64% (p=0.000 n=10)
Many/upsert_32_alerts-8      643.3Ki ± 0%   571.6Ki ± 0%  -11.15% (p=0.000 n=10)
Many/upsert_64_alerts-8      1.398Mi ± 0%   1.258Mi ± 0%  -10.01% (p=0.000 n=10)
Many/upsert_128_alerts-8     988.9Ki ± 0%   701.6Ki ± 0%  -29.05% (p=0.000 n=10)
Many/upsert_256_alerts-8     1.668Mi ± 0%   1.108Mi ± 0%  -33.60% (p=0.000 n=10)
Many/upsert_512_alerts-8     3.060Mi ± 0%   1.940Mi ± 0%  -36.60% (p=0.000 n=10)
Many/upsert_1024_alerts-8    5.860Mi ± 0%   3.603Mi ± 0%  -38.51% (p=0.000 n=10)
Many/upsert_2048_alerts-8   11.777Mi ± 0%   7.315Mi ± 0%  -37.88% (p=0.000 n=10)
Many/upsert_4096_alerts-8    23.10Mi ± 0%   14.19Mi ± 0%  -38.57% (p=0.000 n=10)
Many/upsert_8192_alerts-8    46.21Mi ± 0%   28.46Mi ± 0%  -38.41% (p=0.000 n=10)
geomean                      981.8Ki        735.7Ki       -25.07%master.txtnew.txt                │
                          │  allocs/opallocs/op   vs baseMany/upsert_1_alerts-8       134.0 ± 0%    134.0 ± 0%       ~ (p=1.000 n=10) ¹
Many/upsert_2_alerts-8       245.0 ± 0%    244.0 ± 0%  -0.41% (p=0.000 n=10)
Many/upsert_4_alerts-8       464.0 ± 0%    461.0 ± 0%  -0.65% (p=0.000 n=10)
Many/upsert_8_alerts-8       900.0 ± 0%    893.0 ± 0%  -0.78% (p=0.000 n=10)
Many/upsert_16_alerts-8     1.767k ± 0%   1.752k ± 0%  -0.85% (p=0.000 n=10)
Many/upsert_32_alerts-8     3.498k ± 0%   3.467k ± 0%  -0.89% (p=0.000 n=10)
Many/upsert_64_alerts-8     6.958k ± 0%   6.895k ± 0%  -0.91% (p=0.000 n=10)
Many/upsert_128_alerts-8    13.48k ± 0%   13.36k ± 0%  -0.94% (p=0.000 n=10)
Many/upsert_256_alerts-8    26.80k ± 0%   26.55k ± 0%  -0.95% (p=0.000 n=10)
Many/upsert_512_alerts-8    53.43k ± 0%   52.92k ± 0%  -0.95% (p=0.000 n=10)
Many/upsert_1024_alerts-8   106.7k ± 0%   105.7k ± 0%  -0.96% (p=0.000 n=10)
Many/upsert_2048_alerts-8   213.2k ± 0%   211.2k ± 0%  -0.96% (p=0.000 n=10)
Many/upsert_4096_alerts-8   426.2k ± 0%   422.1k ± 0%  -0.96% (p=0.000 n=10)
Many/upsert_8192_alerts-8   852.2k ± 0%   844.0k ± 0%  -0.96% (p=0.000 n=10)
geomean                     9.956k        9.877k       -0.80%

User-facing documentation

Testing and quality

  • the change is production ready: the change is GA, or otherwise the functionality is gated by a feature flag
  • CI results are inspected

Automated testing

  • added unit tests
  • added e2e tests
  • added regression tests
  • added compatibility tests
  • modified existing tests

How I validated my change

change me!

Signed-off-by: Tomasz Janiszewski <tomek@redhat.com>
Signed-off-by: Tomasz Janiszewski <tomek@redhat.com>
Signed-off-by: Tomasz Janiszewski <tomek@redhat.com>
@janisz janisz requested review from a team as code owners October 20, 2025 10:43
@janisz janisz changed the title Use pool for serialisation perf(proto): Use pool for serialisation Oct 20, 2025
@janisz janisz changed the title perf(proto): Use pool for serialisation WIP: perf(proto): Use pool for serialisation Oct 20, 2025
Signed-off-by: Tomasz Janiszewski <tomek@redhat.com>
@openshift-ci
Copy link

openshift-ci bot commented Oct 20, 2025

@janisz: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/gke-upgrade-tests a493dd3 link false /test gke-upgrade-tests
ci/prow/gke-ui-e2e-tests a493dd3 link true /test gke-ui-e2e-tests
ci/prow/gke-nongroovy-e2e-tests a493dd3 link true /test gke-nongroovy-e2e-tests
ci/prow/gke-operator-e2e-tests a493dd3 link false /test gke-operator-e2e-tests
ci/prow/gke-qa-e2e-tests a493dd3 link false /test gke-qa-e2e-tests
ci/prow/ocp-4-12-nongroovy-e2e-tests a493dd3 link false /test ocp-4-12-nongroovy-e2e-tests
ci/prow/ocp-4-12-qa-e2e-tests a493dd3 link false /test ocp-4-12-qa-e2e-tests
ci/prow/ocp-4-12-operator-e2e-tests a493dd3 link false /test ocp-4-12-operator-e2e-tests
ci/prow/ocp-4-12-compliance-e2e-tests a493dd3 link false /test ocp-4-12-compliance-e2e-tests
ci/prow/ocp-4-19-operator-e2e-tests a493dd3 link false /test ocp-4-19-operator-e2e-tests
ci/prow/ocp-4-19-nongroovy-e2e-tests a493dd3 link false /test ocp-4-19-nongroovy-e2e-tests
ci/prow/ocp-4-19-compliance-e2e-tests a493dd3 link false /test ocp-4-19-compliance-e2e-tests
ci/prow/ocp-4-18-nongroovy-e2e-tests a493dd3 link false /test ocp-4-18-nongroovy-e2e-tests
ci/prow/ocp-4-19-qa-e2e-tests a493dd3 link false /test ocp-4-19-qa-e2e-tests

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@janisz janisz changed the title WIP: perf(proto): Use pool for serialisation perf(proto): Use pool for serialisation Oct 20, 2025
Subsystem: metrics.CentralSubsystem.String(),
Name: "store_serialized_bytes_upsert_histogram",
Help: "Size of serialized column",
//TODO: adjust to the real usage after tests
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO blocking comment

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a JIRA ticket reference (add a ticket for this follow-up fine-tuning if needed to get a ticket number)

@@ -0,0 +1,82 @@
# Process Filter Optimization: String Keys → uint64 Hashes
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels unrelated and very much worthy of its own dedicated PR. Please move to separate PR.

func insertIntoActiveComponents(batch *pgx.Batch, pool pgSearch.BufferPool, obj *storage.ActiveComponent) (*[]byte, error) {

serialized, marshalErr := obj.MarshalVT()
buf := pool.Get(obj.SizeVT())
Copy link
Contributor

@parametalol parametalol Oct 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting would be to see if the new Green Tea GC achieves the same performance.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I was wondering how we can measure the difference. For example change to swiss map went unnoticed at least by me

Copy link
Contributor

@rhybrillou rhybrillou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change looks good overall.
I'd be protective in the generated code that calls MarshalToSizedBufferVT and default to MarshalVT if the buffer is too small (this might require some default return values for the returned buffer in the insert function as well as specific processing for the returned buffers linked to the edge cases.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this change belong to this PR ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this change is already on master.

Subsystem: metrics.CentralSubsystem.String(),
Name: "store_serialized_bytes_upsert_histogram",
Help: "Size of serialized column",
//TODO: adjust to the real usage after tests
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a JIRA ticket reference (add a ticket for this follow-up fine-tuning if needed to get a ticket number)

{{if not $schema.Parent }}
serialized, marshalErr := obj.MarshalVT()
buf := pool.Get(obj.SizeVT())
n, marshalErr := obj.MarshalToSizedBufferVT(*buf)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make sense to compare the buffer capacity with obj.SizeVT() ?
It could create problems if we try to serialize to a too small buffer.

Comment on lines +293 to +294
buf := pool.Get(obj.SizeVT())
pooledBuffers = append(pooledBuffers, buf)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ensure the buffer is large enough to write the serialized object into.

@janisz
Copy link
Contributor Author

janisz commented Dec 18, 2025

@janisz janisz closed this Dec 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants