Skip to content

rewrite tokenizer in wasm#343

Open
PupilTong wants to merge 3 commits intocsstree:masterfrom
PupilTong:p/hw/wasm-tokenizer
Open

rewrite tokenizer in wasm#343
PupilTong wants to merge 3 commits intocsstree:masterfrom
PupilTong:p/hw/wasm-tokenizer

Conversation

@PupilTong
Copy link

#342

The build/bundle part should be modified by reviewers/maintainers.

  • tried my best to keep all comments
  • tried my best to keep the code structure similiar with the original JS code

There are still some ways to improve the wasm tokenizer. I'm not sure in the original code charCodeAt will or won't exceed the length of the source. If the answer is not, some of the boundary checking code could be removed.

If this WASM impl could be accepted, could you please publish the C code in the npm tarball? Therefore we could easily intergrate the tokenizer with our other part of C code.

The reason why I rewrite the tokenizer is that we have to transform the style string on runtime in Browser for our cross platform rendering freamwork(https://github.com/lynx-family). Although the current css-tree's tokenizer is really fast, it still becomes one of a bottelneck of the rendering performance.

* tried my best to keep all comments
* tried my best to keep the code structure similiar with the original JS code

There are still some ways to improve the wasm tokenizer.
I'm not sure in the original code `charCodeAt` will or won't exceed the length of the `source`.  If the answer is not, some of the boundary checking code could be removed.

If this WASM impl could be accepted, could you please publish the C code in the npm tarball?  Therefore we could easily intergrate the tokenizer with our other part of C code.

The reason why I rewrite the tokenizer is that we have to transform the style string on runtime in Browser for our cross platform rendering freamwork(https://github.com/lynx-family). Although the current css-tree's tokenizer is really fast, it still becomes one of a bottelneck of the rendering performance.
@PupilTong
Copy link
Author

image

@Sherry-hue
Copy link

awesome

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants