Add config keys for controlling public/private template secondary storage replica counts by Damans227 · Pull Request #12877 · apache/cloudstack

Damans227 · 2026-03-23T16:12:29Z

Description

Adds two new operator-level configuration keys to control the number of secondary storage copies made for public and private templates, decoupling replica count from template visibility.

public.template.secstorage.copy (default: 0 = all stores, preserving existing behavior)
private.template.secstorage.copy (default: 1, preserving existing behavior)

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

How Has This Been Tested?

…rage replica counts

codecov · 2026-03-23T16:22:13Z

Codecov Report

❌ Patch coverage is 40.90909% with 13 lines in your changes missing coverage. Please review.
✅ Project coverage is 18.02%. Comparing base (b744824) to head (0708236).

Files with missing lines	Patch %	Lines
.../com/cloud/template/HypervisorTemplateAdapter.java	20.00%	7 Missing and 1 partial ⚠️
...n/java/com/cloud/template/TemplateAdapterBase.java	50.00%	5 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff            @@
##               main   #12877   +/-   ##
=========================================
  Coverage     18.02%   18.02%           
- Complexity    16450    16452    +2     
=========================================
  Files          5968     5968           
  Lines        537086   537093    +7     
  Branches      65961    65964    +3     
=========================================
+ Hits          96819    96824    +5     
- Misses       429347   429348    +1     
- Partials      10920    10921    +1

Flag	Coverage Δ
uitests	`3.53% <ø> (ø)`
unittests	`19.19% <40.90%> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

…vate templates

Damans227 · 2026-03-23T16:48:59Z

@blueorangutan package

blueorangutan · 2026-03-23T16:50:05Z

@Damans227 a [SL] Jenkins job has been kicked to build packages. It will be bundled with no SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2026-03-23T17:33:35Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 17224

Copilot

Pull request overview

This PR introduces new template-related configuration keys intended to control how many secondary storage replicas are created for public vs private templates, decoupling replication behavior from template visibility.

Changes:

Adds two new Zone-scoped ConfigKey<Integer> settings: public.template.secstorage.copy and private.template.secstorage.copy.
Reworks secondary storage allocation logic to enforce a per-zone replica limit using a zone→count map.
Updates unit tests to match the new method signatures/replica-limit behavior.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
`engine/components-api/src/main/java/com/cloud/template/TemplateManager.java`	Adds new Zone-scoped config keys for public/private template replica limits.
`server/src/main/java/com/cloud/template/TemplateManagerImpl.java`	Registers the new config keys via `getConfigKeys()`.
`server/src/main/java/com/cloud/template/TemplateAdapterBase.java`	Replaces zone-set visibility logic with zone copy counting + replica limit enforcement.
`server/src/main/java/com/cloud/template/HypervisorTemplateAdapter.java`	Threads replica-limit and zone copy count through template creation/allocation paths.
`server/src/test/java/com/cloud/template/HypervisorTemplateAdapterTest.java`	Updates tests for the new signatures and adds coverage for replica-limit behavior.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-03-25T16:17:19Z

server/src/main/java/com/cloud/template/TemplateAdapterBase.java

+        int replicaLimit = isPrivateTemplate(template)
+                ? TemplateManager.PrivateTemplateSecStorageCopy.value()
+                : TemplateManager.PublicTemplateSecStorageCopy.value();
+        Map<Long, Integer> zoneCopyCount = new HashMap<>();


PublicTemplateSecStorageCopy/PrivateTemplateSecStorageCopy are declared with ConfigKey.Scope.Zone, but here value() is used, which always reads the global value (see ConfigKey#value()), ignoring per-zone configuration. Compute the limit per image store/zone using valueIn(zoneId) (or valueInScope(Scope.Zone, zoneId)), and avoid reading it once before iterating image stores across multiple zones.

Copilot · 2026-03-25T16:17:19Z

server/src/main/java/com/cloud/template/TemplateAdapterBase.java

    /**
     * If the template/ISO is marked as private, then it is allocated to a random secondary storage; otherwise, allocates to every storage pool in every zone given by the
     * {@link TemplateProfile#getZoneIdList()}.
     */
    protected void postUploadAllocation(List<DataStore> imageStores, VMTemplateVO template, List<TemplateOrVolumePostUploadCommand> payloads) {


This method JavaDoc no longer matches the behavior: allocation is now governed by replica-limit config keys (public/private can both be limited), not strictly by template visibility (private=random vs public=all). Please update the comment to reflect the new replica-limit semantics (and that the limits are zone-scoped).

Copilot · 2026-03-25T16:17:20Z

server/src/main/java/com/cloud/template/HypervisorTemplateAdapter.java

+                        ? TemplateManager.PrivateTemplateSecStorageCopy.value()
+                        : TemplateManager.PublicTemplateSecStorageCopy.value();


PublicTemplateSecStorageCopy/PrivateTemplateSecStorageCopy are Scope.Zone config keys, but value() always reads the global value, so zone-level overrides will never apply. In this loop the zone ID is already available (zoneId), so use valueIn(zoneId) when deriving replicaLimit.

Suggested change

? TemplateManager.PrivateTemplateSecStorageCopy.value()

: TemplateManager.PublicTemplateSecStorageCopy.value();

? TemplateManager.PrivateTemplateSecStorageCopy.valueIn(zoneId)

: TemplateManager.PublicTemplateSecStorageCopy.valueIn(zoneId);

Copilot · 2026-03-25T16:17:20Z

server/src/main/java/com/cloud/template/HypervisorTemplateAdapter.java

    protected void standardImageStoreAllocation(List<DataStore> imageStores, VMTemplateVO template) {
-        Set<Long> zoneSet = new HashSet<Long>();
+        int replicaLimit = isPrivateTemplate(template)
+                ? TemplateManager.PrivateTemplateSecStorageCopy.value()
+                : TemplateManager.PublicTemplateSecStorageCopy.value();
        Collections.shuffle(imageStores);
-        validateSecondaryStorageAndCreateTemplate(imageStores, template, zoneSet);
+        validateSecondaryStorageAndCreateTemplate(imageStores, template, new HashMap<>(), replicaLimit);


standardImageStoreAllocation derives replicaLimit using ConfigKey#value(), which only reads the global value; these keys are Scope.Zone, so per-zone settings won’t take effect. Since imageStores is zone-specific (from getImageStoresByZoneIds(zoneId)), derive the zone ID from the list (e.g., first store) and use valueIn(zoneId) (or compute the limit per-store inside the loop).

Copilot · 2026-03-25T16:17:20Z

engine/components-api/src/main/java/com/cloud/template/TemplateManager.java

+    ConfigKey<Integer> PublicTemplateSecStorageCopy = new ConfigKey<Integer>("Advanced", Integer.class,
+            PublicTemplateSecStorageCopyCK, "0",
+            "Maximum number of secondary storage pools to which a public template is copied. " +
+            "0 means copy to all secondary storage pools (default behavior).",
+            true, ConfigKey.Scope.Zone);
+
+    ConfigKey<Integer> PrivateTemplateSecStorageCopy = new ConfigKey<Integer>("Advanced", Integer.class,
+            PrivateTemplateSecStorageCopyCK, "1",
+            "Maximum number of secondary storage pools to which a private template is copied. " +
+            "Default is 1 to preserve existing behavior.",
+            true, ConfigKey.Scope.Zone);


The config key descriptions don’t mention that the limits are evaluated per-zone (and the keys themselves are zone-scoped). Consider clarifying the wording (e.g., “per zone”) so operators understand how replica limits are applied across multiple zones/image stores.

Add config keys for controlling public/private template secondary sto…

40b9e4d

…rage replica counts

Daman Arora added 2 commits March 23, 2026 12:36

Allow configurable secondary storage replica count for public and pri…

f997444

…vate templates

Add unit tests

0708236

DaanHoogland requested a review from Copilot March 25, 2026 16:12

Copilot started reviewing on behalf of DaanHoogland March 25, 2026 16:13 View session

Copilot AI reviewed Mar 25, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add config keys for controlling public/private template secondary storage replica counts#12877

Add config keys for controlling public/private template secondary storage replica counts#12877
Damans227 wants to merge 3 commits intoapache:mainfrom
Damans227:feature/FR284-template-secstorage-copy-control

Damans227 commented Mar 23, 2026

Uh oh!

codecov bot commented Mar 23, 2026 •

edited

Loading

Uh oh!

Damans227 commented Mar 23, 2026

Uh oh!

blueorangutan commented Mar 23, 2026

Uh oh!

blueorangutan commented Mar 23, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Mar 25, 2026

Uh oh!

Copilot AI Mar 25, 2026

Uh oh!

Copilot AI Mar 25, 2026

Uh oh!

Copilot AI Mar 25, 2026

Uh oh!

Copilot AI Mar 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		? TemplateManager.PrivateTemplateSecStorageCopy.value()
		: TemplateManager.PublicTemplateSecStorageCopy.value();

Conversation

Damans227 commented Mar 23, 2026

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

How Has This Been Tested?

Uh oh!

codecov bot commented Mar 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Damans227 commented Mar 23, 2026

Uh oh!

blueorangutan commented Mar 23, 2026

Uh oh!

blueorangutan commented Mar 23, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codecov bot commented Mar 23, 2026 •

edited

Loading