http: improve performance of shouldUseProxy by Uzlopak · Pull Request #59426 · nodejs/node

Uzlopak · 2025-08-09T23:35:12Z

*wroom* *wroom*

This PR improves the performance of shouldUseProxy.

Actually started 2 days ago with improving the perf of ipToInt...

Benchmarks on my machine:

Details

before:

http/proxy-should-use-proxy.js n=1000000 no_proxy="" hostname="127.0.0.1": 152,298,960.42252606
http/proxy-should-use-proxy.js n=1000000 no_proxy="*" hostname="127.0.0.1": 109,371,173.71784446
http/proxy-should-use-proxy.js n=1000000 no_proxy="126.255.255.1-127.0.0.255" hostname="127.0.0.1": 629,054.1395199846
http/proxy-should-use-proxy.js n=1000000 no_proxy="127.0.0.1" hostname="127.0.0.1": 23,596,691.205888372
http/proxy-should-use-proxy.js n=1000000 no_proxy="example.com" hostname="127.0.0.1": 21,604,331.149891604
http/proxy-should-use-proxy.js n=1000000 no_proxy=".example.com" hostname="127.0.0.1": 16,123,322.001540164
http/proxy-should-use-proxy.js n=1000000 no_proxy="*.example.com" hostname="127.0.0.1": 17,621,371.114304848
http/proxy-should-use-proxy.js n=1000000 no_proxy="" hostname="localhost": 161,158,328.71077526
http/proxy-should-use-proxy.js n=1000000 no_proxy="*" hostname="localhost": 110,128,653.39418161
http/proxy-should-use-proxy.js n=1000000 no_proxy="126.255.255.1-127.0.0.255" hostname="localhost": 15,375,318.52086422
http/proxy-should-use-proxy.js n=1000000 no_proxy="127.0.0.1" hostname="localhost": 18,567,246.19312502
http/proxy-should-use-proxy.js n=1000000 no_proxy="example.com" hostname="localhost": 24,235,890.246413812
http/proxy-should-use-proxy.js n=1000000 no_proxy=".example.com" hostname="localhost": 17,635,878.40280526
http/proxy-should-use-proxy.js n=1000000 no_proxy="*.example.com" hostname="localhost": 17,837,685.127969157
http/proxy-should-use-proxy.js n=1000000 no_proxy="" hostname="www.example.com": 159,578,253.82453224
http/proxy-should-use-proxy.js n=1000000 no_proxy="*" hostname="www.example.com": 104,244,926.37337217
http/proxy-should-use-proxy.js n=1000000 no_proxy="126.255.255.1-127.0.0.255" hostname="www.example.com": 9,661,511.364715049
http/proxy-should-use-proxy.js n=1000000 no_proxy="127.0.0.1" hostname="www.example.com": 12,343,979.353065126
http/proxy-should-use-proxy.js n=1000000 no_proxy="example.com" hostname="www.example.com": 11,798,926.93479099
http/proxy-should-use-proxy.js n=1000000 no_proxy=".example.com" hostname="www.example.com": 8,826,112.633218365
http/proxy-should-use-proxy.js n=1000000 no_proxy="*.example.com" hostname="www.example.com": 7,564,523.5133493
http/proxy-should-use-proxy.js n=1000000 no_proxy="" hostname="example.com": 158,343,624.34938583
http/proxy-should-use-proxy.js n=1000000 no_proxy="*" hostname="example.com": 103,525,477.30938941
http/proxy-should-use-proxy.js n=1000000 no_proxy="126.255.255.1-127.0.0.255" hostname="example.com": 15,688,007.450171985
http/proxy-should-use-proxy.js n=1000000 no_proxy="127.0.0.1" hostname="example.com": 23,058,490.65161519
http/proxy-should-use-proxy.js n=1000000 no_proxy="example.com" hostname="example.com": 26,354,197.248516392
http/proxy-should-use-proxy.js n=1000000 no_proxy=".example.com" hostname="example.com": 13,011,310.511034966
http/proxy-should-use-proxy.js n=1000000 no_proxy="*.example.com" hostname="example.com": 15,525,732.107746532
http/proxy-should-use-proxy.js n=1000000 no_proxy="" hostname="myexample.com": 164,711,350.76978672
http/proxy-should-use-proxy.js n=1000000 no_proxy="*" hostname="myexample.com": 106,214,797.76065217
http/proxy-should-use-proxy.js n=1000000 no_proxy="126.255.255.1-127.0.0.255" hostname="myexample.com": 9,192,458.176383602
http/proxy-should-use-proxy.js n=1000000 no_proxy="127.0.0.1" hostname="myexample.com": 12,657,966.677396402
http/proxy-should-use-proxy.js n=1000000 no_proxy="example.com" hostname="myexample.com": 12,283,053.183298787
http/proxy-should-use-proxy.js n=1000000 no_proxy=".example.com" hostname="myexample.com": 7,655,382.777904308
http/proxy-should-use-proxy.js n=1000000 no_proxy="*.example.com" hostname="myexample.com": 9,110,204.836031983

after:

http/proxy-should-use-proxy.js n=1000000 no_proxy="" hostname="127.0.0.1": 295,829,017.92753434
http/proxy-should-use-proxy.js n=1000000 no_proxy="*" hostname="127.0.0.1": 274,345,562.37822485
http/proxy-should-use-proxy.js n=1000000 no_proxy="126.255.255.1-127.0.0.255" hostname="127.0.0.1": 10,364,300.176334057
http/proxy-should-use-proxy.js n=1000000 no_proxy="127.0.0.1" hostname="127.0.0.1": 25,335,366.78365188
http/proxy-should-use-proxy.js n=1000000 no_proxy="example.com" hostname="127.0.0.1": 36,742,652.18604451
http/proxy-should-use-proxy.js n=1000000 no_proxy=".example.com" hostname="127.0.0.1": 38,972,973.29529
http/proxy-should-use-proxy.js n=1000000 no_proxy="*.example.com" hostname="127.0.0.1": 34,475,165.76263014
http/proxy-should-use-proxy.js n=1000000 no_proxy="" hostname="localhost": 303,066,671.03082675
http/proxy-should-use-proxy.js n=1000000 no_proxy="*" hostname="localhost": 290,866,275.393353
http/proxy-should-use-proxy.js n=1000000 no_proxy="126.255.255.1-127.0.0.255" hostname="localhost": 21,210,277.974691644
http/proxy-should-use-proxy.js n=1000000 no_proxy="127.0.0.1" hostname="localhost": 27,279,087.267927937
http/proxy-should-use-proxy.js n=1000000 no_proxy="example.com" hostname="localhost": 37,369,598.7850396
http/proxy-should-use-proxy.js n=1000000 no_proxy=".example.com" hostname="localhost": 38,129,886.178095475
http/proxy-should-use-proxy.js n=1000000 no_proxy="*.example.com" hostname="localhost": 39,684,624.70012313
http/proxy-should-use-proxy.js n=1000000 no_proxy="" hostname="www.example.com": 284,585,619.1487418
http/proxy-should-use-proxy.js n=1000000 no_proxy="*" hostname="www.example.com": 300,751,789.24758214
http/proxy-should-use-proxy.js n=1000000 no_proxy="126.255.255.1-127.0.0.255" hostname="www.example.com": 11,227,351.6249346
http/proxy-should-use-proxy.js n=1000000 no_proxy="127.0.0.1" hostname="www.example.com": 13,975,908.915200153
http/proxy-should-use-proxy.js n=1000000 no_proxy="example.com" hostname="www.example.com": 14,612,559.456860516
http/proxy-should-use-proxy.js n=1000000 no_proxy=".example.com" hostname="www.example.com": 9,358,467.76951655
http/proxy-should-use-proxy.js n=1000000 no_proxy="*.example.com" hostname="www.example.com": 7,851,985.981221269
http/proxy-should-use-proxy.js n=1000000 no_proxy="" hostname="example.com": 292,501,575.8522399
http/proxy-should-use-proxy.js n=1000000 no_proxy="*" hostname="example.com": 258,859,331.17996883
http/proxy-should-use-proxy.js n=1000000 no_proxy="126.255.255.1-127.0.0.255" hostname="example.com": 21,097,655.7529689
http/proxy-should-use-proxy.js n=1000000 no_proxy="127.0.0.1" hostname="example.com": 32,412,925.107667632
http/proxy-should-use-proxy.js n=1000000 no_proxy="example.com" hostname="example.com": 24,466,027.7216731
http/proxy-should-use-proxy.js n=1000000 no_proxy=".example.com" hostname="example.com": 13,245,926.46698768
http/proxy-should-use-proxy.js n=1000000 no_proxy="*.example.com" hostname="example.com": 38,109,891.1669161
http/proxy-should-use-proxy.js n=1000000 no_proxy="" hostname="myexample.com": 208,814,694.70769596
http/proxy-should-use-proxy.js n=1000000 no_proxy="*" hostname="myexample.com": 302,527,648.75813913
http/proxy-should-use-proxy.js n=1000000 no_proxy="126.255.255.1-127.0.0.255" hostname="myexample.com": 11,585,614.499313131
http/proxy-should-use-proxy.js n=1000000 no_proxy="127.0.0.1" hostname="myexample.com": 15,337,032.830145737
http/proxy-should-use-proxy.js n=1000000 no_proxy="example.com" hostname="myexample.com": 14,267,590.980183072
http/proxy-should-use-proxy.js n=1000000 no_proxy=".example.com" hostname="myexample.com": 8,919,752.421567837
http/proxy-should-use-proxy.js n=1000000 no_proxy="*.example.com" hostname="myexample.com": 14,072,507.751840893

nodejs-github-bot · 2025-08-09T23:35:17Z

Review requested:

@nodejs/performance

codecov · 2025-08-10T01:32:44Z

Codecov Report

❌ Patch coverage is 88.52459% with 14 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.90%. Comparing base (a73b575) to head (986b7db).
⚠️ Report is 3 commits behind head on main.

Files with missing lines	Patch %	Lines
lib/internal/http.js	88.52%	14 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #59426      +/-   ##
==========================================
- Coverage   89.91%   89.90%   -0.01%     
==========================================
  Files         655      655              
  Lines      192866   192941      +75     
  Branches    37806    37833      +27     
==========================================
+ Hits       173412   173467      +55     
- Misses      12015    12021       +6     
- Partials     7439     7453      +14

Files with missing lines	Coverage Δ
lib/internal/http.js	`92.70% <88.52%> (-2.97%)`	⬇️

... and 35 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

joyeecheung · 2025-08-10T09:18:37Z

lib/internal/http.js

+  }
+
+  // See: https://about.gitlab.com/blog/we-need-to-talk-no-proxy
+  // TODO(joyeecheung): share code with undici.


I would prefer this to happen before we do any optimisation. IMO it's better to put this in undici and we can vendor it in in Node.js. It's not ideal if fetch proxy decision starts to diverge from http/https built-ins as we keep making them more convoluted.

mcollina

lgtm

jasnell · 2025-08-23T14:40:29Z

lib/internal/http.js

+  let octetShift = 0;
+  let code = 0;
+
+  for (let i = ip.length - 1; i >= 0; --i) {


Nit: a comment in here briefly explaining the algorithm would be helpful for folks coming into this code later. Optional tho.

aduh95 · 2025-11-08T20:46:28Z

This needs a rebase

nodejs-github-bot added http Issues or PRs related to the http subsystem. needs-ci PRs that need a full CI run. labels Aug 9, 2025

Uzlopak force-pushed the improve-should-use-proxy branch 2 times, most recently from ef99bf3 to adacfa0 Compare August 9, 2025 23:45

http: improve performance of shouldUseProxy

79f0534

Uzlopak force-pushed the improve-should-use-proxy branch from adacfa0 to 79f0534 Compare August 9, 2025 23:46

http: improve performance of shouldUseProxy

986b7db

Uzlopak mentioned this pull request Aug 10, 2025

http: trim off brackets from IPv6 addresses with string operations #59420

Merged

joyeecheung reviewed Aug 10, 2025

View reviewed changes

anonrig approved these changes Aug 22, 2025

View reviewed changes

mcollina approved these changes Aug 22, 2025

View reviewed changes

jasnell reviewed Aug 23, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Comments

http: improve performance of shouldUseProxy#59426

http: improve performance of shouldUseProxy#59426
Uzlopak wants to merge 2 commits intonodejs:mainfrom
Uzlopak:improve-should-use-proxy

Uzlopak commented Aug 9, 2025 •

edited

Loading

Uh oh!

nodejs-github-bot commented Aug 9, 2025

Uh oh!

codecov bot commented Aug 10, 2025 •

edited

Loading

Uh oh!

joyeecheung Aug 10, 2025

Uh oh!

mcollina left a comment

Uh oh!

jasnell Aug 23, 2025

Uh oh!

aduh95 commented Nov 8, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Uh oh!

Comments

Conversation

Uzlopak commented Aug 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nodejs-github-bot commented Aug 9, 2025

Uh oh!

codecov bot commented Aug 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

joyeecheung Aug 10, 2025

Choose a reason for hiding this comment

Uh oh!

mcollina left a comment

Choose a reason for hiding this comment

Uh oh!

jasnell Aug 23, 2025

Choose a reason for hiding this comment

Uh oh!

aduh95 commented Nov 8, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Uzlopak commented Aug 9, 2025 •

edited

Loading

codecov bot commented Aug 10, 2025 •

edited

Loading