feat(changelog): Support remote git repositories as template sources#1404
feat(changelog): Support remote git repositories as template sources#1404bilelomrani1 wants to merge 4 commits intopython-semantic-release:masterfrom
Conversation
da206de to
dd85343
Compare
|
While working on this feature, I noticed a potential limitation worth discussing for a future enhancement. When using user templates, With remote template sources, users may want to use a template from an external repository or storage that has its own Users can already control which part of a remote template to use by pointing template_dir = "git+https://github.com/org/templates@main/changelogs"But, there's no way to render templates to a location other than the project root. This matters when users want their changelog in a subdirectory like I'm proposing to add a [tool.semantic_release.changelog]
template_dir = "git+https://github.com/org/templates@main"
output_dir = "docs/"This would render the template rooted at I wanted to flag it as a potential follow-up. Happy to hear thoughts on whether this is a use case worth supporting and which approach would fit best with the existing design. |
|
Just submitted a proposal that would address the previous limitation here: #1406 |
|
Just a note: this is one of the smaller, more self-contained features from my recent batch of submissions. It adds support for loading templates from remote git repositories, which is useful for sharing templates across multiple projects. I don't think this conflicts with the v11 refactoring work, so it could potentially be reviewed independently whenever you have time. No rush see #1407 for the broader context on my use case and how this fits in. |
|
I agree this should not conflict. Thanks for putting the effort in here as it would be awhile before I would have gotten to this when it was asked for before. |
dd85343 to
e17d06a
Compare
| # we disallow them entirely. | ||
| if "::" in val: | ||
| raise ValueError("Chained protocols are not supported for template_dir.") | ||
| return val |
There was a problem hiding this comment.
We can instead check for explicit "local://" and "file://" in the string to prevent accidental local filesystem access outside the project's repo. This catches common mistakes. This would allow chained protocols instead of disallowing them, which avoids confusing users with a seemingly arbitrary restriction.
I also just realized one workaround around the current protection anyway: fsspec allows users to register custom protocols that could wrap the local filesystem. We cannot detect these, but my assumption is that this is not a real attack vector: it requires the user to deliberately install and configure such a protocol on their system. So chained protocols are not really necessary nor sufficient to trigger an attack, so we might as well just allow them. Users who do this are explicitly opting into local filesystem access, which is outside our threat model.
My assumption with this path traversal check is: our goal is to prevent accidental misconfiguration and obvious attacks, not to sandbox users from their own deliberate actions.
cc @codejedi365, can you confirm how reasonable this assumption sounds to you?
Purpose
Closes #1401
Add support for remote repositories as template sources for changelog generation, allowing users to share templates across multiple projects without copy-paste or manual CI setup.
Rationale
Currently,
template_dironly accepts local filesystem paths, requiring users to either copy templates into each project, use git submodules or add manual download steps in CI workflows.By leveraging universal_pathlib which uses fsspec under the hood, we can abstract the storage layer and support any fsspec-compatible backend with minimal code changes.
Security considerations:
simplecache::file://) are explicitly rejected to prevent bypass of path traversal checksfile://,local://, and bare pathsHow did you test?
Added tests covering:
UPathLoadertemplate loading from memory filesystemStorageOptionsConfigenvironment variable resolutiontemplate_diracceptance for various protocolsrecursive_renderwithUPathsourcesValidated end-to-end with a real GitHub-hosted template repository:
Successfully generated changelog using remote templates.
How to Verify
Install dependencies:
pip install -e .Create a test repository with a
pyproject.toml:Run
semantic-release changelogVerify templates are fetched from the remote repository
PR Completion Checklist
Reviewed & followed the Contributor Guidelines
Changes Implemented & Validation pipeline succeeds
Commits follow the Conventional Commits standard
and are separated into the proper commit type and scope (recommended order: test, build, feat/fix, docs)
Appropriate Unit tests added/updated
Appropriate End-to-End tests added/updated
Appropriate Documentation added/updated and syntax validated for sphinx build