Fixes: #59202

Summary

• preserve non-UTF8 filename bytes when using glob with encoding: 'buffer'
• keep matching behavior while avoiding UTF-8 replacement corruption in returned paths
• add regression coverage for non-UTF8 filenames and traversal in buffer mode
• document encoding and return types for glob APIs

Testing

• python3 tools/test.py test/parallel/test-fs-glob.mjs
• python3 tools/test.py --repeat=5 test/parallel/test-fs-glob.mjs
• python3 tools/test.py test/parallel/test-fsglob
• make lint-js NODE=$(command -v node)
• scripts/agent-preflight.sh
• make test-doc is blocked locally by an ignored local AGENTS.md markdown-lint warning; this file is not tracked in the patch.