Skip to content

Comments

Filter CI test logs for sanitized builds#393

Merged
jbachorik merged 3 commits intomainfrom
jb/ci_reports
Feb 23, 2026
Merged

Filter CI test logs for sanitized builds#393
jbachorik merged 3 commits intomainfrom
jb/ci_reports

Conversation

@jbachorik
Copy link
Collaborator

What does this PR do?:
Adds a streaming log filter to CI test jobs that compresses verbose Gradle test output: passing tests emit a single summary line, failing tests emit full context (stdout/stderr including [TEST::INFO] lines, exception and stack trace), and crashed tests emit a warning with their buffered output. The unfiltered raw log is preserved via tee and collected as an artifact on failure.

Motivation:
Sanitized builds (asan/tsan) produce extremely noisy CI logs — per-test debug output, sanitizer stack traces, and profiler initialization messages make it very hard to spot actual failures. The filtered view makes failures immediately visible without having to scroll through thousands of irrelevant lines.

Additional Notes:

  • filter_gradle_log.py: streaming state-machine filter (OUTSIDE → BUFFERING → FAILING). [TEST::INFO] lines are suppressed for passing tests and in class-level init context (OUTSIDE state), but shown in full for failing tests.
  • ProfilerTestPlugin.kt: added "started" to testLogging.events — without it the filter never entered buffering mode.
  • ProfilerTestRunner.java: added GradleStyleTestListener that emits STARTED/PASSED/FAILED/SKIPPED markers to stdout, making the musl/Exec path produce the same format as the glibc/Test-task path.
  • ConfigurationPresets.kt: added log_path=/tmp/*san_%p.log to ASAN/UBSAN/TSAN options so sanitizer reports go to per-process files instead of polluting the build log; collected as artifacts.
  • prepare_reports.sh: collects test-raw.log and sanitizer log files alongside existing artifacts.
  • musl-amd64 Test step: added shell: bash — GHA defaults to busybox sh inside Alpine containers, which does not support PIPESTATUS.

How to test the change?:
Trigger a sanitized build (add test:asan or test:tsan label to a PR) and verify that the CI log shows only one line per passing test and full context for any failures. The raw unfiltered log is available as test-raw.log in the test-reports artifact.

For Datadog employees:

  • If this PR touches code that signs or publishes builds or packages, or handles
    credentials of any kind, I've requested a review from @DataDog/security-design-and-guidance.
  • This PR doesn't touch any of that.
  • JIRA: N/A

jbachorik and others added 3 commits February 23, 2026 09:38
- Add filter_gradle_log.py: streaming filter that emits one line per
  passing test and full context (STARTED + stdout + exception) for
  failing/crashed tests; [TEST::INFO] lines suppressed on pass
- Pipe all four test job types through tee + filter; raw log preserved
  in build/test-raw.log and collected as artifact on failure
- Add GradleStyleTestListener to ProfilerTestRunner so musl/Exec jobs
  emit STARTED/PASSED/FAILED markers matching Gradle's Test task format
- Redirect ASAN/UBSAN/TSAN output to /tmp/*san_%p.log via log_path so
  sanitizer reports no longer pollute the build log; collected as artifacts
- Collect sanitizer logs in prepare_reports.sh alongside hs_err files

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ProfilerTestPlugin was only emitting passed/skipped/failed events;
without STARTED the filter never entered buffering mode and passed
everything through unchanged.

Also suppress [TEST::INFO] lines in OUTSIDE state (class-level init
output that precedes any STARTED marker) since they are noise when no
test has failed; the raw log still contains them.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
GHA defaults to sh (busybox) inside Alpine containers; PIPESTATUS is
bash-only and caused a 'bad substitution' error. bash is already
installed in Setup OS, just needs to be declared for the Test step.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@jbachorik jbachorik added the test:asan Run CI tests with AddressSanitizer configuration label Feb 23, 2026
@dd-octo-sts
Copy link

dd-octo-sts bot commented Feb 23, 2026

CI Test Results

Run: #22305234112 | Commit: 5cb0b95 | Duration: 12m 31s (longest job)

1 of 70 test jobs failed

Status Overview

JDK glibc-aarch64/asan glibc-aarch64/debug glibc-amd64/asan glibc-amd64/debug musl-aarch64/debug musl-amd64/debug
8 - - - -
8-ibm - - - -
8-j9 - -
8-librca - - - -
8-orcl - - - -
8-zing - -
11 - - - -
11-j9 - -
11-librca - - - -
11-zing - -
17 - -
17-graal - -
17-j9 - -
17-librca - - - -
17-zing - -
21 - -
21-graal - -
21-librca - - - -
21-zing - -
25 - -
25-graal - -
25-librca - - - -

Legend: ✅ passed | ❌ failed | ⚪ skipped | 🚫 cancelled

Failed Tests

glibc-amd64/asan / 8-j9

Job: View logs

No detailed failure information available. Check the job logs.

Summary: Total: 70 | Passed: 69 | Failed: 1


Updated: 2026-02-23 12:18:02 UTC

@jbachorik jbachorik merged commit 57e6397 into main Feb 23, 2026
394 of 472 checks passed
@jbachorik jbachorik deleted the jb/ci_reports branch February 23, 2026 15:19
@github-actions github-actions bot added this to the 1.38.0 milestone Feb 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AI Continuous Integration no-release-notes test:asan Run CI tests with AddressSanitizer configuration trivial

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant