Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Now that we use recast->esprima, we can simplify the lexer grammar for regexes and action blocks #18

Open
GerHobbelt opened this issue Oct 15, 2017 · 3 comments
Assignees
Milestone

Comments

@GerHobbelt
Copy link
Owner

Re-use the (customized) esprima parser that comes with recast to help us parse lexer regexes and lexer and parser action blocks: we can re-use the esprima scanner for those parts!

Suggestion: use a simple lexer regex to match the start of the regex or action code block input and then consume as much as necessary using the esprima scanner to produce the appropriate jison-lex / jison grammar token.

@GerHobbelt
Copy link
Owner Author

Sideways related: #22 -- both these issues are about using the full power of recast+esprima in our code generator(s).

@GerHobbelt
Copy link
Owner Author

Also related: #17 -- using esprima+recast means we restrict ourselves to a tighter controlled set of accepted action codes: old jison didn't care all that much as long as you merely pooped out generated source code (it was, and is, quite another matter when you produce parsers live, though!)

@GerHobbelt
Copy link
Owner Author

Progress: ditch esprima (that was a bad choice, in that esprima doesn't track modern JS development as well as babel & friends. Consequently the choice for recast is dubious as it turns out babel/parse produces an ever so slightly different AST (some node types differ from esprima and b0rk recast), while the recast intention could be mimicked using a babel transform plugin -- iff you forego the recast/print exact source copy for unaltered content feature.

Not 100% sure if I should ditch recast as well... pretty sure it's the Sunk Cost Fallacy at work in my brain, though. 😁


Key goal is to enable jison to check action code early and as much as possible so that action code mistakes (typo's, variable references, etc.) can be reported soon and precisely related to jison user grammar source files plus the ability to accept and produce modern JavaScript in any part of your grammar spec code sets.

Since the generally accepted process for this is compiling your stuff through babel, jison should do the same, as babel closely tracks modern JS development and provides full downwards compatibility in the generated output (babel-transform).

Recast may be nicer in that it has a nicely documented and stable API (babel explicitly states they keep the plugin API spec adaptable to enable swift and clean compiler development 👍 + 💘? ) but swapping esprima for babel/parse triggered several nasty b0rks in my codebase, so I am on the fence deciding to go with babel+recast or babel+babel-custom-plugin to make it happen. Where 'it' is transforming all action blocks + jison grammar spec into one generated JS file that's ready to run.

@GerHobbelt GerHobbelt self-assigned this Jun 17, 2018
@GerHobbelt GerHobbelt added this to the 0.6.5 milestone Jun 17, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant