Skip to content

Generate AST transformers from config#5337

Merged
lukastaegert merged 17 commits intomasterfrom
generate-ast-transformer
Jan 13, 2024
Merged

Generate AST transformers from config#5337
lukastaegert merged 17 commits intomasterfrom
generate-ast-transformer

Conversation

@lukastaegert
Copy link
Member

This PR contains:

  • bugfix
  • feature
  • refactor
  • documentation
  • other

Are tests included?

  • yes (bugfixes and features will not be merged without tests)
  • no

Breaking Changes?

  • yes (breaking changes will not be merged unless absolutely necessary)
  • no

List any relevant issue numbers:

Description

This changes how the converters from SWC AST to binary and from binary to JavaScript AST work. Previously, all of them were hand-crafted, which means it was extremely important that every single node was properly checked, all indices aligned and all fields were converted in the right way and order.
To make this easier, the JavaScript converter is now fully generated from a script that reads a config file and automatically assigns indices and orders fields. It also generates constants for the Rust code so that field order is aligned.

To have the same level of efficiency, field types in Rust can contain an "_INLINED_FIELDNAME" suffix which indicates if there is a field that is not referenced via its position but is rather directly added to a node as the last field. In that case, this field needs to be converted first. If the inlined field changes, the type name changes, which ensures we know that the Rust code needs to be updated.

The bigger picture here, though, is that we eventually want to

  • Consume the Buffer directly in the Rollup AST classes. To do that, we can now use the same script to generate constants and single property converters as needed.
  • Be able to convert an existing JavaScript AST to binary. Again, we can now use the same config file. This is important to handle the case where a plugin returns a JavaScript AST, though we need to think how to handle the case where the plugin adds non-standard properties and nodes.

@vercel
Copy link

vercel bot commented Jan 11, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
rollup ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jan 12, 2024 6:12am

@lukastaegert lukastaegert marked this pull request as draft January 11, 2024 05:58
@github-actions
Copy link

github-actions bot commented Jan 11, 2024

Thank you for your contribution! ❤️

You can try out this pull request locally by installing Rollup via

npm install rollup/rollup#generate-ast-transformer

Notice: Ensure you have installed Rust nightly. If you haven't installed it yet, please first see https://www.rust-lang.org/tools/install to learn how to download Rustup and install Rust, then see https://rust-lang.github.io/rustup/concepts/channels.html to learn how to install Rust nightly.

or load it into the REPL:
https://rollup-64rxtt286-rollup-js.vercel.app/repl/?pr=5337

@codecov
Copy link

codecov bot commented Jan 11, 2024

Codecov Report

Attention: 4 lines in your changes are missing coverage. Please review.

Comparison is base (7fa474c) 98.81% compared to head (1a8f720) 98.81%.

Files Patch % Lines
src/utils/buffer-to-ast.ts 96.63% 0 Missing and 4 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #5337      +/-   ##
==========================================
- Coverage   98.81%   98.81%   -0.01%     
==========================================
  Files         232      232              
  Lines        8983     8993      +10     
  Branches     2342     2345       +3     
==========================================
+ Hits         8877     8886       +9     
  Misses         46       46              
- Partials       60       61       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@lukastaegert lukastaegert force-pushed the generate-ast-transformer branch from b9af278 to 1a8f720 Compare January 12, 2024 06:11
@lukastaegert lukastaegert merged commit 3ba1511 into master Jan 13, 2024
@lukastaegert lukastaegert deleted the generate-ast-transformer branch January 13, 2024 06:00
@github-actions
Copy link

This PR has been released as part of rollup@4.9.6. You can test it via npm install rollup.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant