Description

Refactored tests/yamls/roxctl_verification.sh to decouple roxctl verification tests from YAML file contents, preventing brittle test failures when YAML files are modified for other purposes.

Problem:
The original test used a wildcard (*.yaml) to test all YAML files and blindly expected exactly 2 specific alerts from every file. This created tight coupling between the roxctl test and the YAML file contents:

• When YAML files were edited for other tests (e.g., TestPods, container_instances_test), the roxctl verification test would break unexpectedly
• There was no clear documentation of which files should trigger which policies
• Error messages were vague ("Did not find 2 alerts") without indicating what was expected

Solution:

• Replace wildcard file selection with an explicit associative array mapping each file to its expected policy violations
• Each file now has documented expectations that can be updated independently when the file changes
• Enhanced error reporting shows both expected and actual policy names, making it immediately clear what needs to be updated
• When someone modifies a YAML file for their test, they can easily see and update the corresponding expectation in the TEST_CASES array

Additional Improvements:

• Updated to use --output json instead of deprecated --json flag
• Fixed jq path for new roxctl JSON output format (.results[].violatedPolicies[].name)
• Added bash version check (requires 4.0+ for associative arrays)
• Changed shebang to #!/usr/bin/env bash for portability
• Added set -euo pipefail for safer script execution
• Added ROX_PASSWORD environment variable support for authentication
• Added --insecure-skip-tls-verify fallback when CA is not provided

Why This Matters:
This refactoring prevents the common scenario where:

1. Developer updates multi-container-pod.yaml for a pod lifecycle test
2. Roxctl verification test mysteriously fails in CI
3. Developer has to debug why changing a YAML file broke an unrelated test

Now, the test expectations are explicit and maintainable. When a YAML file changes, the developer can immediately see and update the corresponding entry in TEST_CASES.

AI Assistance:
The core refactoring logic and test structure were generated with AI assistance. Manual corrections included: fixing bash array iteration issues, updating to the correct jq path for the new roxctl output format, adding --insecure-skip-tls-verify support, and validating all test expectations against a live Central instance.

User-facing documentation

• CHANGELOG.md is updated OR update is not needed
• documentation PR is created and is linked above OR is not needed

Testing and quality

• the change is production ready: the change is GA, or otherwise the functionality is gated by a feature flag
• CI results are inspected

Automated testing

• added unit tests
• added e2e tests
• added regression tests
• added compatibility tests
• modified existing tests

How I validated my change

Local Validation:

1. Deployed Central locally on localhost:8443
2. Ran the refactored script: API_ENDPOINT="localhost:8443" ROX_PASSWORD="..." ./tests/yamls/roxctl_verification.sh
3. Verified all 15 YAML files are tested successfully with correct policy expectations
4. All files currently expect 2 alerts: "Latest tag" + "No CPU request or memory limit specified"
5. Confirmed no deprecation warnings from roxctl (using --output json)
6. Validated bash version check: fails gracefully with bash 3.2, succeeds with bash 4.0+
7. Ran make shell-style to verify no style violations
8. Tested that error messages clearly show both expected and found policies when expectations don't match
9. Verified shebang portability: works with both /bin/bash and Homebrew bash on macOS

Decoupling Validation:

• Modified a test YAML file's image tag as a simulation
• Observed clear error message indicating which policy expectations failed
• Updated the corresponding TEST_CASES entry
• Test passed immediately after update
• This demonstrates the improved maintainability vs. the old wildcard approach

CI Validation:

• Awaiting CI results to confirm compatibility with CI environment
• The refactored script maintains the same test coverage as the original