Skip to content

ROX-31229: Add compliance E2E tests for tailored profiles#19880

Open
guzalv wants to merge 17 commits intomasterfrom
gualvare/ROX-31229-e2e-tailored-profiles
Open

ROX-31229: Add compliance E2E tests for tailored profiles#19880
guzalv wants to merge 17 commits intomasterfrom
gualvare/ROX-31229-e2e-tailored-profiles

Conversation

@guzalv
Copy link
Copy Markdown
Contributor

@guzalv guzalv commented Apr 7, 2026

Description

Every existing compliance e2e test now creates its own TP dynamically (instead of relying on a globally-installed one), and uses mixed profiles (Profile + TailoredProfile) wherever scan configs are created. This validates that ACS stores operator_kind correctly and that ScanSettingBinding resources carry the right k8s Kind per profile entry.

A new test (TestComplianceV2TailoredProfileVariants) has been added to ensure that different TP flavors are handled correctly:

  • extends-base with both enabled and disabled rules
  • from-scratch with custom rules (CEL CustomRule)
  • from-scratch with regular rules

User-facing documentation

Testing and quality

  • the change is production ready: the change is GA, or otherwise the functionality is gated by a feature flag
  • CI results are inspected

Automated testing

  • added e2e tests
  • modified existing tests

How I validated my change

All 10 compliance v2 e2e tests pass on a real OCP cluster with Compliance Operator v1.8.2:

--- PASS: TestComplianceV2CentralSendsScanConfiguration (65.35s)
--- PASS: TestComplianceV2Integration (0.69s)
--- PASS: TestComplianceV2ProfileGetSummaries (2.97s)
--- PASS: TestComplianceV2DeleteComplianceScanConfigurations (4.09s)
--- PASS: TestComplianceV2ProfileGet (4.65s)
--- PASS: TestComplianceV2ComplianceObjectMetadata (7.19s)
--- PASS: TestComplianceV2TailoredProfileVariants (9.14s)
    --- PASS: .../extends (...)
    --- PASS: .../custom-rules (...)
    --- PASS: .../from-scratch (...)
--- PASS: TestComplianceV2UpdateScanConfigurations (11.46s)
--- PASS: TestComplianceV2CreateGetScanConfigurations (104.68s)
--- PASS: TestComplianceV2ScheduleRescan (219.45s)
PASS
ok      github.com/stackrox/rox/tests   286.655s

@openshift-ci
Copy link
Copy Markdown

openshift-ci bot commented Apr 7, 2026

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@codecov
Copy link
Copy Markdown

codecov bot commented Apr 7, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 49.56%. Comparing base (0f13ca6) to head (bcf6d30).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master   #19880   +/-   ##
=======================================
  Coverage   49.56%   49.56%           
=======================================
  Files        2764     2764           
  Lines      208357   208357           
=======================================
  Hits       103269   103269           
  Misses      97436    97436           
  Partials     7652     7652           
Flag Coverage Δ
go-unit-tests 49.56% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 7, 2026

🚀 Build Images Ready

Images are ready for commit bcf6d30. To use with deploy scripts:

export MAIN_IMAGE_TAG=4.11.x-639-gbcf6d30ae8

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 7, 2026

🚀 Build Images Ready

Images are ready for commit 5dd8504. To use with deploy scripts:

export MAIN_IMAGE_TAG=4.11.x-570-g5dd85047c9

@guzalv
Copy link
Copy Markdown
Contributor Author

guzalv commented Apr 7, 2026

/test ocp-4-21-compliance-e2e-tests

guzalv added a commit that referenced this pull request Apr 8, 2026
Problem: The install_e2e_compliance_resources() function was creating a
TailoredProfile immediately after creating the CustomRule it references,
without waiting for the CustomRule to be processed by the compliance operator.
This caused a race condition where the TailoredProfile could fail to reach
READY state if the CustomRule wasn't validated yet.

Solution: Add a wait loop after creating the CustomRule to ensure it exists
and give the compliance operator time to process it (10 seconds) before
creating the TailoredProfile. Also improved error reporting to show the
TailoredProfile's error message and current state during the wait loop.

The e2e-tailored-profile.yaml references check-cm-marker CustomRule in its
enableRules, so the CustomRule must be ready before the TailoredProfile can
be validated successfully.

Fixes: #19880 CI failure in ocp-4-21-compliance-e2e-tests

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@guzalv
Copy link
Copy Markdown
Contributor Author

guzalv commented Apr 9, 2026

/test ocp-4-21-compliance-e2e-tests

Copy link
Copy Markdown
Contributor Author

@guzalv guzalv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Partial review

Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 1 issue, and left some high level feedback:

  • There’s quite a bit of inline TailoredProfile and CustomRule construction (e.g., in TestComplianceV2TailoredProfileVariants) that mirrors the helper-style logic in createTailoredProfile/createCustomRule; consider extracting common patterns into reusable helpers to keep the tests shorter and reduce duplication of ready-wait loops and cleanup logic.
  • Some calls to ListComplianceScanConfigurations now deliberately ignore errors (e.g., scanConfigs, _ := ...), which could hide real problems during debugging; it would be safer to assert or require on the error and only proceed when the call succeeds.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- There’s quite a bit of inline TailoredProfile and CustomRule construction (e.g., in TestComplianceV2TailoredProfileVariants) that mirrors the helper-style logic in createTailoredProfile/createCustomRule; consider extracting common patterns into reusable helpers to keep the tests shorter and reduce duplication of ready-wait loops and cleanup logic.
- Some calls to ListComplianceScanConfigurations now deliberately ignore errors (e.g., `scanConfigs, _ := ...`), which could hide real problems during debugging; it would be safer to assert or require on the error and only proceed when the call succeeds.

## Individual Comments

### Comment 1
<location path="tests/compliance_operator_v2_test.go" line_range="1206-1209" />
<code_context>
+	}
+
+	// --- Verify from-scratch TP with custom rule: custom rule present ---
+	fromScratchCRDetail, err := profileClient.GetComplianceProfile(ctx, &v2.ResourceByID{Id: fromScratchCRInACS.GetId()})
+	require.NoError(t, err)
+	foundCustomRule := false
+	for _, r := range fromScratchCRDetail.GetRules() {
+		if strings.Contains(r.GetName(), crName) {
+			foundCustomRule = true
</code_context>
<issue_to_address>
**issue (testing):** Make the custom-rule presence assertion more robust in TailoredProfileVariants

This assertion may be brittle: `strings.Contains(r.GetName(), crName)` assumes the ACS rule name matches the `CustomRule` resource name, but ACS may expose the XCCDF ID (e.g. `xccdf_org.example_rule_<name_with_underscores>`). Since `createCustomRule` builds the ID via `strings.ReplaceAll(name, "-", "_")`, `crName` with hyphens might never appear verbatim in `r.GetName()`. Instead, derive the expected rule ID using the same transformation as `createCustomRule` and compare for equality, or match on a prefix like `"xccdf_org.example_rule_"` plus the transformed name to avoid false negatives.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Apr 9, 2026

📝 Walkthrough

Walkthrough

Added shared test helpers for creating CustomRules and TailoredProfiles with ingestion polling. Introduced profileRef struct to track profile names and operator kinds. Reworked assertScanSettingBinding to validate profiles by name and Kind. Updated multiple existing tests and added a new TailoredProfileVariants test.

Changes

Cohort / File(s) Summary
Test Infrastructure & Helpers
tests/compliance_operator_v2_test.go
Added profileRef struct and helper methods (k8sKind(), profileNames()). Introduced new helpers: createCustomRule(), createTailoredProfile(), and waitForTPIngestion() for test setup. Reworked assertScanSettingBinding() signature to validate profiles by name and Kind.
Test Updates
tests/compliance_operator_v2_test.go
Modified 7 existing tests to create per-test tailored profiles and validate ACS ingestion. Updated profile assertions to use profileRef for mixed Profile + TailoredProfile validation. Extended duplicate-profile rejection checks to include TailoredProfile names.
New Test
tests/compliance_operator_v2_test.go
Added TestComplianceV2TailoredProfileVariants covering extends-base with disabled rules, from-scratch with custom rules, and from-scratch with regular rules variants, with rule-content assertions.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 31.58% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Title check ✅ Passed The title 'ROX-31229: Add compliance E2E tests for tailored profiles' accurately and clearly summarizes the main change: adding end-to-end test coverage for compliance tailored profiles.
Description check ✅ Passed The PR description provides a comprehensive overview of the changes, explaining the rationale for dynamic TP creation, the new test added, and validation results on a real OCP cluster with all tests passing.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch gualvare/ROX-31229-e2e-tailored-profiles

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (2)
tests/compliance_operator_v2_test.go (2)

296-304: Consider shorter poll intervals for faster feedback.

The 10-second poll interval for TailoredProfile readiness may delay test failure detection. A 5-second interval would provide faster feedback without significantly increasing load on the cluster.

Proposed change
-	}, 3*time.Minute, 10*time.Second)
+	}, 3*time.Minute, 5*time.Second)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@tests/compliance_operator_v2_test.go` around lines 296 - 304, The test is
using a 10-second poll interval in require.EventuallyWithT which slows feedback;
update the call in tests/compliance_operator_v2_test.go (the
require.EventuallyWithT block that fetches complianceoperatorv1.TailoredProfile
using client.Get with NamespacedName{Name: name, Namespace: coNamespaceV2}) to
use a 5-second poll interval instead of 10 seconds to get faster failure
detection while keeping the same overall timeout.

1205-1216: Consider documenting why strings.Contains is used for custom rule matching.

Using strings.Contains instead of exact matching suggests the Compliance Operator may transform rule names. A brief comment explaining this would improve maintainability.

Suggested comment
 	// --- Verify from-scratch TP with custom rule: custom rule present ---
 	fromScratchCRDetail, err := profileClient.GetComplianceProfile(ctx, &v2.ResourceByID{Id: fromScratchCRInACS.GetId()})
 	require.NoError(t, err)
 	foundCustomRule := false
 	for _, r := range fromScratchCRDetail.GetRules() {
+		// Use Contains because CO may prefix/transform rule names during processing.
 		if strings.Contains(r.GetName(), crName) {
 			foundCustomRule = true
 			break
 		}
 	}
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@tests/compliance_operator_v2_test.go` around lines 1205 - 1216, Tests use
strings.Contains when checking rule names on the fromScratchCRDetail.GetRules()
list because rule names can be transformed (prefixes/suffixes/IDs) by the
Compliance Operator; add a concise inline comment above the loop that references
fromScratchCRDetail, the loop over GetRules(), r.GetName(), and crName
explaining that non-exact matching is intentional due to operator-driven name
normalization and therefore we use substring matching (strings.Contains) rather
than exact equality.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@tests/compliance_operator_v2_test.go`:
- Around line 707-733: The test calls err.Error() after
CreateComplianceScanConfiguration for both duplicateProfileReq and
duplicateTPReq which can nil-panic if the call unexpectedly succeeds; update the
checks to first assert that an error was returned (e.g., use require.Error(t,
err) or assert.Error(t, err)) before asserting on the message with err.Error(),
referencing the CreateComplianceScanConfiguration calls, scanConfigService,
duplicateProfileReq and duplicateTPReq to locate the two failing assertions.

---

Nitpick comments:
In `@tests/compliance_operator_v2_test.go`:
- Around line 296-304: The test is using a 10-second poll interval in
require.EventuallyWithT which slows feedback; update the call in
tests/compliance_operator_v2_test.go (the require.EventuallyWithT block that
fetches complianceoperatorv1.TailoredProfile using client.Get with
NamespacedName{Name: name, Namespace: coNamespaceV2}) to use a 5-second poll
interval instead of 10 seconds to get faster failure detection while keeping the
same overall timeout.
- Around line 1205-1216: Tests use strings.Contains when checking rule names on
the fromScratchCRDetail.GetRules() list because rule names can be transformed
(prefixes/suffixes/IDs) by the Compliance Operator; add a concise inline comment
above the loop that references fromScratchCRDetail, the loop over GetRules(),
r.GetName(), and crName explaining that non-exact matching is intentional due to
operator-driven name normalization and therefore we use substring matching
(strings.Contains) rather than exact equality.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Central YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Pro

Run ID: a234cc97-b475-4304-af67-a3ddc55a7efb

📥 Commits

Reviewing files that changed from the base of the PR and between e037ba4 and 5834b14.

📒 Files selected for processing (1)
  • tests/compliance_operator_v2_test.go

guzalv added a commit that referenced this pull request Apr 10, 2026
Problem: The install_e2e_compliance_resources() function was creating a
TailoredProfile immediately after creating the CustomRule it references,
without waiting for the CustomRule to be processed by the compliance operator.
This caused a race condition where the TailoredProfile could fail to reach
READY state if the CustomRule wasn't validated yet.

Solution: Add a wait loop after creating the CustomRule to ensure it exists
and give the compliance operator time to process it (10 seconds) before
creating the TailoredProfile. Also improved error reporting to show the
TailoredProfile's error message and current state during the wait loop.

The e2e-tailored-profile.yaml references check-cm-marker CustomRule in its
enableRules, so the CustomRule must be ready before the TailoredProfile can
be validated successfully.

Fixes: #19880 CI failure in ocp-4-21-compliance-e2e-tests

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@guzalv guzalv changed the base branch from master to master-base/gualvare/fix-compliance-start-tp-informer-after-oob April 10, 2026 08:52
@guzalv
Copy link
Copy Markdown
Contributor Author

guzalv commented Apr 10, 2026

@guzalv guzalv force-pushed the gualvare/ROX-31229-e2e-tailored-profiles branch from 36ad77c to ef67153 Compare April 10, 2026 08:52
Copy link
Copy Markdown
Contributor Author

@guzalv guzalv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Partial review

Copy link
Copy Markdown
Contributor Author

@guzalv guzalv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Partial review

Copy link
Copy Markdown
Contributor Author

@guzalv guzalv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.

Base automatically changed from master-base/gualvare/fix-compliance-start-tp-informer-after-oob to master April 10, 2026 11:48
@guzalv guzalv changed the title ROX-31229: add e2e tests for tailored profile tracking ROX-31229: Add compliance E2e tests for tailored profile tracking Apr 10, 2026
@guzalv guzalv changed the title ROX-31229: Add compliance E2e tests for tailored profile tracking ROX-31229: Add compliance E2E tests for tailored profiles Apr 10, 2026
Install a CustomRule + from-scratch TailoredProfile (e2e-tailored-profile)
during compliance operator setup so they are available to all compliance e2e
tests.

Test changes:
- Add profileRef{name, operatorKind} helper type so assertScanSettingBinding
  can verify the Kind field on each SSB profile entry (Profile vs
  TailoredProfile), not just the name
- TestComplianceV2ProfileGet/GetSummaries: assert operator_kind on both
  the e2e TP and a regular profile
- TestComplianceV2CreateGetScanConfigurations: use mixed profiles (regular +
  TP); assert SSB kinds; test TP duplicate rejection
- TestComplianceV2UpdateScanConfigurations: update from single Profile to
  mixed Profile+TP; assert SSB kinds for both
- TestComplianceV2DeleteComplianceScanConfigurations: use mixed profiles
- TestComplianceV2CentralSendsScanConfiguration: validate startup sync path
  preserves profile_refs with correct kinds across Sensor reconnect
- TestComplianceV2ScheduleRescan: include TP alongside regular profile
- TestComplianceV2TailoredProfileVariants (new): creates an extends-base TP
  dynamically, verifies operator_kind == TAILORED_PROFILE for both from-scratch
  and extends-base variants, and verifies the disabled rule is excluded from
  the effective rules list

Note: CO does not allow mixing CustomRules and regular Rules in an
extends-based TP, so the variant test uses only DisableRules.

All tests verified on cluster (rox-26032-cluster-1).

AI-assisted implementation.
guzalv and others added 16 commits April 10, 2026 15:10
Problem: The install_e2e_compliance_resources() function was creating a
TailoredProfile immediately after creating the CustomRule it references,
without waiting for the CustomRule to be processed by the compliance operator.
This caused a race condition where the TailoredProfile could fail to reach
READY state if the CustomRule wasn't validated yet.

Solution: Add a wait loop after creating the CustomRule to ensure it exists
and give the compliance operator time to process it (10 seconds) before
creating the TailoredProfile. Also improved error reporting to show the
TailoredProfile's error message and current state during the wait loop.

The e2e-tailored-profile.yaml references check-cm-marker CustomRule in its
enableRules, so the CustomRule must be ready before the TailoredProfile can
be validated successfully.

Fixes: #19880 CI failure in ocp-4-21-compliance-e2e-tests

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Tests now create their own TP/CR resources per-test with unique names
and proper cleanup, so the shared install_e2e_compliance_resources()
and its yaml manifests are no longer needed.

Partially generated by AI.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
createTailoredProfile now creates a simple extends-based TP (ocp4-e8)
instead of requiring a custom rule. Tests that just need "any TP" use
this helper; variant-specific logic moved to the variants test.

TestComplianceV2TailoredProfileVariants now covers all TP types inline:
- extends-base with disabled rules
- from-scratch with custom rules
- from-scratch with regular rules only

Partially generated by AI.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Remove customRuleNamespace constant, reuse coNamespaceV2 for ConfigMap
- Simplify CustomRule ID to match name (per CO team recommendation)
- Truncate createTailoredProfile comment, include name in Title
- Remove -tp suffix from TP names (separate UUIDs where needed)
- Reduce timeouts from 3min/2min to 1min (compromise vs 10s)
- Fix "platform Profile" → "Profile" in comment
- Specify (Profile/TailoredProfile) in profileRef comment

Partially generated by AI.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Per review feedback — empirically these complete well within 10s.

Partially generated by AI.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Restore error checking on ListComplianceScanConfigurations calls that
  were silently ignoring errors (could hide real failures)
- Tighten custom rule assertion from strings.Contains to exact match
  (safe now that ID=name per CO team recommendation)
- Remove unused strings import

Partially generated by AI.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
If CreateComplianceScanConfiguration unexpectedly succeeds for a
duplicate profile, err is nil and err.Error() panics. Add require.Error
before accessing the error message.

Partially generated by AI.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The old name used "ingestion" which is an internal concept that isn't
obvious to readers. The new name makes the intent explicit: we're
waiting for the tailored profile to appear in Central's database.

Also adds a name query filter to ListComplianceProfiles so we don't
fetch all profiles on every poll iteration, and restores the deleted
"UUID-based name for test isolation" comment.

Generated with AI assistance.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Rename variable to match the naming convention used elsewhere in the
test (regularProfile). Also removes the duplicate operator_kind
assertion after GetComplianceProfile — it was already verified on the
ListComplianceProfiles result two lines above.

Generated with AI assistance.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Drop "inherited from base profile" from rules assertion message since
  TP rules are not necessarily inherited (from-scratch TPs have their
  own rules).
- Remove redundant operator_kind check on regular profile via
  GetComplianceProfile — already asserted on the List result above.

Generated with AI assistance.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
ScanSettingBinding profile kind assertions are already covered in
TestComplianceV2CentralSendsScanConfiguration. This test focuses on
the scan config CRUD API, not the SSB reconciliation.

Generated with AI assistance.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The same schedule (daily at 15:00, days 1-6) was inlined 7 times
across different test functions. Extract it into a package-level
variable to reduce duplication and noise.

Generated with AI assistance.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Use tpName := testID instead of generating separate UUIDs with "-tp"
  suffixes. The TP name just needs to be unique, and testID already is.
- Move TP creation in UpdateScanConfigurations right before the update
  step where it's actually needed, not at the top of the test.
- Remove "per-test" from comments — it adds no value.

Generated with AI assistance.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add EnableRules to extends-base TP variant to test enable+disable
  combination (different rules, since CO doesn't allow both on the
  same rule). Verify enabled rule is present in the stored profile.
- Rename variant 2 from fromScratchCR to customRulesTP since custom
  rules can only be from-scratch, making "fromScratch" redundant.
- Use separate fromScratchRule const in variant 3 instead of reusing
  disabledRule in an EnableRules context, which was confusing.
- Rename stored profile vars to storedExtendsTP, storedCustomRulesTP,
  storedFromScratchTP for clarity.

Generated with AI assistance.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Each variant's spec, expected rules, and rejected rules are now
co-located in a map entry. The shared creation/wait/assertion logic
runs once per subtest, making it easier to follow what each variant
tests and to add new variants.

Generated with AI assistance.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…comment

- Remove redundant expectRules/rejectRules from variants table; derive
  expected/rejected rules directly from the spec's EnableRules/DisableRules.
- Replace map[string]bool with []string + assert.Contains/NotContains.
- Add CO recommendation comment above CustomRule ID field.
- Increase waitUntilTPInCentralDB timeout to 1min (10s was flaky under load).
- Use a different enabled rule (alwaysadmit) to avoid overlap with the
  disabled rule's profile family.

Partially generated by AI.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@guzalv guzalv force-pushed the gualvare/ROX-31229-e2e-tailored-profiles branch from 1dd1603 to bcf6d30 Compare April 10, 2026 13:12
@guzalv guzalv marked this pull request as ready for review April 10, 2026 13:17
Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • The TailoredProfile creation/wait logic in TestComplianceV2TailoredProfileVariants duplicates the createTailoredProfile helper; consider reusing the helper there to keep the readiness checks and cleanup behavior consistent across tests.
  • In createCustomRule, the shared e2e-cr-config ConfigMap is created but never cleaned up; if you expect these tests to run repeatedly against longer-lived clusters, consider either cleaning it up in t.Cleanup or explicitly documenting that it is intentionally left behind as a shared fixture.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The TailoredProfile creation/wait logic in `TestComplianceV2TailoredProfileVariants` duplicates the `createTailoredProfile` helper; consider reusing the helper there to keep the readiness checks and cleanup behavior consistent across tests.
- In `createCustomRule`, the shared `e2e-cr-config` ConfigMap is created but never cleaned up; if you expect these tests to run repeatedly against longer-lived clusters, consider either cleaning it up in `t.Cleanup` or explicitly documenting that it is intentionally left behind as a shared fixture.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant