Skip to content

build, doc: use new api doc tooling#57343

Open
flakey5 wants to merge 2 commits intonodejs:mainfrom
flakey5:flakey5/20250305/api-docs-tooling
Open

build, doc: use new api doc tooling#57343
flakey5 wants to merge 2 commits intonodejs:mainfrom
flakey5:flakey5/20250305/api-docs-tooling

Conversation

@flakey5
Copy link
Member

@flakey5 flakey5 commented Mar 6, 2025

Switches over to using the new doc generation tooling. For more background on this, please see #52343

Currently a draft just to get feedback on the approach to this integration.

cc @nodejs/web-infra


Notable Change info (by @avivkeller):

The Node.js Website and Website Infrastructure teams have introduced a brand-new documentation pipeline, modernizing how our API docs are generated. While the documentation site may look familiar today, this infrastructure we are hard at work making a completely refreshed user interface in the very near future!

@nodejs-github-bot
Copy link
Collaborator

Review requested:

  • @nodejs/nodejs-website
  • @nodejs/web-infra

@nodejs-github-bot nodejs-github-bot added build Issues and PRs related to build files or the CI. doc Issues and PRs related to the documentations. needs-ci PRs that need a full CI run. tools Issues and PRs related to the tools directory. windows Issues and PRs related to the Windows platform. labels Mar 6, 2025
@flakey5 flakey5 marked this pull request as draft March 6, 2025 06:24
@flakey5 flakey5 force-pushed the flakey5/20250305/api-docs-tooling branch from 77ede22 to 3423c21 Compare March 6, 2025 06:29
@flakey5 flakey5 force-pushed the flakey5/20250305/api-docs-tooling branch from 3423c21 to 451f8a7 Compare March 6, 2025 06:31
ovflowd

This comment was marked as outdated.

@flakey5 flakey5 force-pushed the flakey5/20250305/api-docs-tooling branch 3 times, most recently from cf2609b to a3ce99d Compare March 10, 2025 22:04
@flakey5 flakey5 marked this pull request as ready for review March 10, 2025 22:05
@flakey5

This comment was marked as resolved.

@flakey5

This comment was marked as resolved.

@ovflowd

This comment was marked as resolved.

@codecov
Copy link

codecov bot commented Mar 10, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.72%. Comparing base (9cc7fcc) to head (95a3dc8).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main   #57343   +/-   ##
=======================================
  Coverage   89.72%   89.72%           
=======================================
  Files         675      675           
  Lines      204797   204797           
  Branches    39344    39353    +9     
=======================================
+ Hits       183752   183758    +6     
- Misses      13324    13329    +5     
+ Partials     7721     7710   -11     

see 31 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@araujogui

This comment was marked as resolved.

@araujogui

This comment was marked as resolved.

@ovflowd

This comment was marked as resolved.

Copy link
Member

@ovflowd ovflowd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the result of many months of arduous work between many awesome folks, including @flakey5 @AugustinMauroy @araujogui @ovflowd @avivkeller and others.

I'm so proud of what we are achieving here and this is a huge step towards a modern tooling and a revamped API docs within Node.js

Approving, as I believe this is ready!

@ovflowd

This comment was marked as resolved.

Copy link
Member

@lpinca lpinca left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RSLGTM because it is hard to review and outside of my comfort zone.

@ovflowd
Copy link
Member

ovflowd commented Feb 9, 2026

Do you mean to say the "node:process's 'message' event params are no longer parsed correctly:" is an issue on the markdown or?

Yes. The new tooling is much stricter on what it considers valid vs invalid. The markdown for that particular event would be considered invalid by the new tooling.

Once this PR lands, we can land the new linter, which will identify invalid markdown before they land.

I'd argue we should fix that specific Markdown file before we land this PR, otherwise that section is broken.

@avivkeller
Copy link
Member

See #61756

@aduh95
Copy link
Contributor

aduh95 commented Feb 10, 2026

A few bugs I've found while browsing the JSON diff:

When a function has multiple signatures, on this PR it's no longer taken into account, the JSON no longer contains info about e.g. return type or history image image
REPLACEME is for some reason considered older instead of newer (FWIW we have a linter rule that enforces the order in the markdown is correct, so the markdown order should be used instead) image image
The tool identify as params things that clearly could not be valid JS identifiers image image

Also it looks like #57516 was never ported to doc-kit (i.e. having stability labels sticking to the top of the page), any reason for not doing it?

I guess the first item is potentially breaking, do we know how easy/hard is it to fix? I don't think it's a blocker for landing, it might be one for backporting though. The other ones are nice-to-have, and should be easy to fix anyway.

@avivkeller
Copy link
Member

avivkeller commented Feb 10, 2026

When a function has multiple signatures, on this PR it's no longer taken into account, the JSON no longer contains info about e.g. return type or history

It still does, however, it correctly places the return type, history, etc with the signature it is associated with. (i.e. you'll find all the meta items within the signature array.

is for some reason considered older instead of newer (FWIW we have a linter rule that enforces the order in the markdown is correct, so the markdown order should be used instead)

Noted. We can just remove the sortChanges function. However, REPLACEME changes are never included in the final release, so it shouldn't really matter

The tool identify as params things that clearly could not be valid JS identifiers

There's a give/take here. The more exclusive the parsing is, the higher the chance of a false-negative. The less exclusive, the higher the chance of a false positive. That being said, I don't think it'll break anything if we exclude [0-9'"] from starting the RegEx in the CAMEL_CASE constant.

@aduh95
Copy link
Contributor

aduh95 commented Feb 10, 2026

It still does, however, it correctly places the return type, history, etc with the signature it is associated with. (i.e. you'll find all the meta items within the signature array.

There are two cases:

  • each signature has its own doc (e.g. process.emitWarning):
    • ✅ current tooling attaches different metadata (history, return type, params) to each.
    • ✅ doc-kit attaches different metadata (history, return type, params) to each.
  • both signatures share the same doc (e.g. stream.pipeline):
    • ✅ current tooling interprets it as a fall-through and attach same metadata to both.
    • ❌ doc-kit attaches no metadata to the first signature.

That's a regression, not a major one, but still one worth fixing before we can safely backport.

REPLACEME changes are never included in the final release, so it shouldn't really matter

It definitely matters for the DX though

@ovflowd
Copy link
Member

ovflowd commented Feb 11, 2026

It definitely matters for the DX though

It is called time travel, Antoine, never heard of? Future versions actually came before! 😛

@ovflowd
Copy link
Member

ovflowd commented Feb 11, 2026

  • doc-kit attaches no metadata to the first signature.

@avivkeller are you working on a fix for this?

@avivkeller
Copy link
Member

are you working on a fix for this?

It's a little more complex than you'd think. We currently handle parsing "entry by entry" so, one heading section is passed at a time. Passing multiple would require re-thinking the way we handleEntry in legacy-json.

@ovflowd
Copy link
Member

ovflowd commented Feb 13, 2026

are you working on a fix for this?

It's a little more complex than you'd think. We currently handle parsing "entry by entry" so, one heading section is passed at a time. Passing multiple would require re-thinking the way we handleEntry in legacy-json.

@aduh95 are we fine as is-then?

@ovflowd ovflowd force-pushed the flakey5/20250305/api-docs-tooling branch from 5956e06 to d3830b4 Compare February 13, 2026 22:19
Copy link
Contributor

@aduh95 aduh95 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think all blockers have been resolved, and tradeoffs have been identified, let's proceed

@avivkeller avivkeller removed the tsc-agenda Issues and PRs to discuss during the meetings of the TSC. label Feb 14, 2026
@avivkeller avivkeller force-pushed the flakey5/20250305/api-docs-tooling branch from f60729c to efb7c12 Compare February 14, 2026 20:26
@nodejs-github-bot
Copy link
Collaborator

@avivkeller
Copy link
Member

Looks like swc (our minifier), does not support all builds, causing most of the CI failures.

cc @ovflowd @flakey5

@ovflowd
Copy link
Member

ovflowd commented Feb 14, 2026

Looks like swc (our minifier), does not support all builds, causing most of the CI failures.

cc @ovflowd @flakey5

What minifier? The HTML one?

@avivkeller
Copy link
Member

avivkeller commented Feb 14, 2026

Yes. SWC's HTML minifier doesn't have bindings for some of the more niche machines. It has the following bindings:

        "@swc/html-darwin-arm64": "1.15.11",
        "@swc/html-darwin-x64": "1.15.11",
        "@swc/html-linux-arm-gnueabihf": "1.15.11",
        "@swc/html-linux-arm64-gnu": "1.15.11",
        "@swc/html-linux-arm64-musl": "1.15.11",
        "@swc/html-linux-x64-gnu": "1.15.11",
        "@swc/html-linux-x64-musl": "1.15.11",
        "@swc/html-win32-arm64-msvc": "1.15.11",
        "@swc/html-win32-ia32-msvc": "1.15.11",
        "@swc/html-win32-x64-msvc": "1.15.11"

flakey5 and others added 2 commits February 15, 2026 17:57
Switches over to using the new doc generation tooling.
For more background on this, please see nodejs#52343

Signed-off-by: flakey5 <73616808+flakey5@users.noreply.github.com>

Co-authored-by: Claudio W <cwunder@gnome.org>
Co-authored-by: avivkeller <me@aviv.sh>
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
Co-authored-by: Joyee Cheung <joyeec9h3@gmail.com>
@ovflowd ovflowd force-pushed the flakey5/20250305/api-docs-tooling branch from fa670b4 to 95a3dc8 Compare February 15, 2026 17:00
@ovflowd
Copy link
Member

ovflowd commented Feb 15, 2026

@Renegade334 I removed a test you added 17h ago as this test becomes irrelevant with doc-kit (it tests for that already and that's also covered (or planned to be covered, if not yet) on the new linting package for @node-core's lint pkg)

@Renegade334
Copy link
Member

de nada 🫡

@aduh95
Copy link
Contributor

aduh95 commented Feb 15, 2026

this test becomes irrelevant with doc-kit (it tests for that already and that's also covered (or planned to be covered, if not yet) on the new linting package

That doesn't sound like a good reason to remove it, if it already passes why not keep it? If it doesn't, we should investigate why.

@ovflowd
Copy link
Member

ovflowd commented Feb 15, 2026

this test becomes irrelevant with doc-kit (it tests for that already and that's also covered (or planned to be covered, if not yet) on the new linting package

That doesn't sound like a good reason to remove it, if it already passes why not keep it? If it doesn't, we should investigate why.

It would have passed, the issue is that it relies on the old tooling internals, you can compare the imports. Also seems like René is fine with the removal.

@bjohansebas bjohansebas added the request-ci Add this label to start a Jenkins CI on a PR. label Feb 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

build Issues and PRs related to build files or the CI. commit-queue-squash Add this label to instruct the Commit Queue to squash all the PR commits into the first one. doc Issues and PRs related to the documentations. needs-ci PRs that need a full CI run. notable-change PRs with changes that should be highlighted in changelogs. request-ci Add this label to start a Jenkins CI on a PR. tools Issues and PRs related to the tools directory. web-agenda

Projects

None yet

Development

Successfully merging this pull request may close these issues.