Add metadata when uploading benchmark data #6759
Open
+5
−1
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Generate a metadata file alongside criterion benchmark data containing the commit hash, ref name, and timestamp for tracking purposes. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Contributor
📝 WalkthroughWalkthroughThe workflow's benchmark publishing step now generates and commits a JSON metadata file alongside criterion benchmark results, containing generation timestamp, commit hash, and reference information. Changes
Estimated code review effort🎯 1 (Trivial) | ⏱️ ~5 minutes Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In @.github/workflows/cron-ci.yaml:
- Around line 183-187: The current printf call writing
./_data/criterion-metadata.json can produce invalid JSON if values like "${{
github.ref_name }}" contain quotes or special characters; replace the
printf-based template with a JSON encoder invocation (e.g., call out to Python
or jq) to construct and write the JSON so values are properly escaped and
serialized; update the step that writes criterion-metadata.json (replacing the
printf usage) to build a dict/object with keys generated_at, rustpython_commit,
rustpython_ref and output it via the JSON encoder, then keep the subsequent git
add ./assets/criterion ./_data/criterion-metadata.json as-is.
Comment on lines
+183
to
+187
| printf '{\n "generated_at": "%s",\n "rustpython_commit": "%s",\n "rustpython_ref": "%s"\n}\n' \ | ||
| "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \ | ||
| "${{ github.sha }}" \ | ||
| "${{ github.ref_name }}" > ./_data/criterion-metadata.json | ||
| git add ./assets/criterion ./_data/criterion-metadata.json |
Contributor
There was a problem hiding this comment.
Guard against invalid JSON by escaping metadata values.
printf writes raw strings; github.ref_name can legally contain quotes, which would corrupt the JSON file. Prefer a JSON encoder (Python or jq) to guarantee correct escaping.
✅ Suggested fix (Python JSON encoder)
- printf '{\n "generated_at": "%s",\n "rustpython_commit": "%s",\n "rustpython_ref": "%s"\n}\n' \
- "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
- "${{ github.sha }}" \
- "${{ github.ref_name }}" > ./_data/criterion-metadata.json
+ python - <<'PY' > ./_data/criterion-metadata.json
+ import json, os, datetime
+ data = {
+ "generated_at": datetime.datetime.utcnow().strftime("%Y-%m-%dT%H:%M:%SZ"),
+ "rustpython_commit": os.environ["GITHUB_SHA"],
+ "rustpython_ref": os.environ["GITHUB_REF_NAME"],
+ }
+ print(json.dumps(data, indent=2))
+ PY📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| printf '{\n "generated_at": "%s",\n "rustpython_commit": "%s",\n "rustpython_ref": "%s"\n}\n' \ | |
| "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \ | |
| "${{ github.sha }}" \ | |
| "${{ github.ref_name }}" > ./_data/criterion-metadata.json | |
| git add ./assets/criterion ./_data/criterion-metadata.json | |
| python - <<'PY' > ./_data/criterion-metadata.json | |
| import json, os, datetime | |
| data = { | |
| "generated_at": datetime.datetime.utcnow().strftime("%Y-%m-%dT%H:%M:%SZ"), | |
| "rustpython_commit": os.environ["GITHUB_SHA"], | |
| "rustpython_ref": os.environ["GITHUB_REF_NAME"], | |
| } | |
| print(json.dumps(data, indent=2)) | |
| PY | |
| git add ./assets/criterion ./_data/criterion-metadata.json |
🤖 Prompt for AI Agents
In @.github/workflows/cron-ci.yaml around lines 183 - 187, The current printf
call writing ./_data/criterion-metadata.json can produce invalid JSON if values
like "${{ github.ref_name }}" contain quotes or special characters; replace the
printf-based template with a JSON encoder invocation (e.g., call out to Python
or jq) to construct and write the JSON so values are properly escaped and
serialized; update the step that writes criterion-metadata.json (replacing the
printf usage) to build a dict/object with keys generated_at, rustpython_commit,
rustpython_ref and output it via the JSON encoder, then keep the subsequent git
add ./assets/criterion ./_data/criterion-metadata.json as-is.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a minor issue, but when looking at the data on https://rustpython.github.io/benchmarks, it is difficult to tell which commit the results were generated from. To address this, I updated the process to include metadata and push it to the RustPython/rustpython.github.io repository.
I will also open a PR to the RustPython/rustpython.github.io repository.See also RustPython/rustpython.github.io#83.
Summary by CodeRabbit
✏️ Tip: You can customize this high-level summary in your review settings.