Adapt to Cloudflare Workers environment

[Mini256]

try to close #2179

Summary

• To fix the error window is not found, replace node Timers with web timers
• To fix the error Code generation from strings disallowed for this context, using the static parser in PR Add "interpreting" parser #2099
• To fix the error crypto.hash() is not a function, polyfill node crypto withwebcrypto, and modify the functions which directly/indirectly called the crypto library as async function.

TODOs:

• mysql_native_password auth plugin
• caching_sha2_password auth plugin
• sha256_password auth plugin
• static_text_parser (will be finished in PR Add "interpreting" parser #2099)
• static_binary_parser (will be finished in PR Add "interpreting" parser #2099)
• add integration tests
• repalce pg-cloudflare with mysql2-cloudflare?

Usage

For users, they need to:

1. Add node_compat = true flag to polyfill the missing node modules
2. Add useStaticParser flag to force mysql2 to use static parser.

For example: https://github.com/Mini256/mysql2-cloudflare-test/blob/1f339a85f4457b696f122503051991326dbc23dc/src/index.ts#L30

[shiyuhang0]

This PR can run on Cloudflare Workers with the following database:

• MySQL 5
• MySQL 8
• TiDB Serverless Cluster
• PlanetScale

@sidorares could you please approve to run the GitHub action?

[sidorares]

@shiyuhang0 @Mini256 thanks for working on this! Could you document simple steps I can follow to try it myself?

[shiyuhang0]

@shiyuhang0 @Mini256 thanks for working on this! Could you document simple steps I can follow to try it myself?

cc3458f

This commit adds a step-by-step tutorial. Since it's user-facing, it needs a new release of MySQL2 including this PR.
it's better to follow the develop guide in the doc to test locally without a new release.

[shiyuhang0]

CI is green. Maybe we can ignore the failure of CodeQL action, because the error is reported in the code that originally existed (just the location has changed)

As for the static parser. We'd like to merge your PR #2099 rather than cherry-pick it into this PR. What's your opinion? @sidorares
The static binary parser is also needed to make things work. Code generation from strings disallowed for this context still occurs when code goes through the binary parser.

[sidorares]

As for the static parser. We'd like to merge your PR #2099 rather than cherry-pick it into this PR. What's your opinion? @sidorares The static binary parser is also needed to make things work. Code generation from strings disallowed for this context still occurs when code goes through the binary parser.

I can probably merge it first, together with added binary protocol non-jit parser.
I don't like the flag name "useStaticParser", it reflects more implementation details rather than intention, but don't have better ideas. Can you think of a better flag name? "disableEvals" ? "strictMode" ( could be an umbrella flag, also prohibits cleartext plugin etc ). disableGenFunction ( again, too focused on implementation ). Naming things is hard 🤷‍♂️

re testing - thanks a lot for dev documentation. Are you aware of any way to test workers locally or in actions runtime? something similar to localstack. Ideally with the same checks, e.i errors when code tries to eval. Any container we can use for that?

[Mini256]

Are you aware of any way to test workers locally or in actions runtime?

Cloudflare Workers provide an unstable_dev API for E2E / integration tests:

https://developers.cloudflare.com/workers/wrangler/api/#unstable_dev

Test code in node-postgres:

https://github.com/brianc/node-postgres/blob/6cd0aeb212d1672edd33499b2f4f858cf7ed9a79/packages/pg/test/worker/src/index.test.js#L13

[shiyuhang0]

I can probably merge it first, together with added binary protocol non-jit parser.
I don't like the flag name "useStaticParser", it reflects more implementation details rather than intention, but don't have better ideas. Can you think of a better flag name? "disableEvals" ? "strictMode" ( could be an umbrella flag, also prohibits cleartext plugin etc ). disableGenFunction ( again, too focused on implementation ). Naming things is hard 🤷‍♂️

I prefer disableEvals among these names. You can also use other names, not good at naming for me :(

[sidorares]

I prefer disableEvals among these names. You can also use other names, not good at naming for me :(

I like it more than useStaticParser, I think I'll update in my branch ( together with a version for no evals binary parser ) and comment here when ready

[advancedtw]

I have tested this pr on multiple MySQL server ranging from old 5.7 to more recent MySQL and MariaDB version locally and on deployed workers so far it works flawlessly. The only issue is having to set node_compat = true instead of compatibility_flags = [ "nodejs_compat" ] but that's a small price to pay.
Also it's currently not possible to use drizzle with msyql driver but kysely works ok.

[elithrar]

FYI:

• Other drivers (like node-postgres and Postgres.js) also require node_compat = true today
• We have plans to allow nodejs_compat to work: this will likely require us to better audit specific packages and conditionally load/exclude Node.js APIs that we don't have support for in Workers, or that don't make sense in a serverless environment like Workers.

Otherwise, we'd love to see this merged + we're bringing MySQL support to Hyperdrive: https://developers.cloudflare.com/hyperdrive/

[shiyuhang0]

I have resolved the conflict with master branch.
@sidorares Hi, what's the process of no evals parser? I'd like to remove the related codes in the PR after that.

[wellwelwel]

Hi, @shiyuhang0 🙋🏻‍♂️

If it's okay, I'd like to make a few simple suggestions in the documentation:

• Change from .md to .mdx
• Change the title from Run on Cloudflare Workers to Cloudflare Workers
• I think it fits in the Examples section, what do you think?

[shiyuhang0]

Hi, @shiyuhang0 🙋🏻‍♂️

If it's okay, I'd like to make a few simple suggestions in the documentation:

• Change from .md to .mdx
• Change the title from Run on Cloudflare Workers to Cloudflare Workers
• I think it fits in the Examples section, what do you think?

already doing this, will push a commit soon

[sidorares]

@shiyuhang0 what do you think about useStaticParser flag name? Should we rename it to disableEval before its too late and somebody else depends on this name?

[shiyuhang0]

@shiyuhang0 what do you think about useStaticParser flag name? Should we rename it to disableEval before its too late and somebody else depends on this name?

Both it is fine for me. But we shouldn't worry about modifying parameter names in an unmerged PR. For me, I think it can even be modified before release.

I will adjust this PR after the name is determined

[m430]

I really need this feature. I need to use the node-mysql2 driver with Drizzle in Cloudflare Workers. If everything is confirmed to be correct, could you please merge this PR as soon as possible? Thanks

@sidorares

[advancedtw]

@m430 last time I checked Drizzle/Mysql was not compatible with cloudflare even using this updated driver. That is because of some node compatibility issue.
If you really need a mysql driver for cf today! you can use a git dependecy or @nora-soderlund cloudflare-mysql

@shiyuhang0 disableEval makes more sense to me

[sidorares]

@shiyuhang0 missed your message, I also prefer disableEval. Are you able to continue or you need someone to take over your work?

[shiyuhang0]

@shiyuhang0 missed your message, I also prefer disableEval. Are you able to continue or you need someone to take over your work?

@sidorares I have pushed a new commit to use disableEval instead of useStaticParser . Is there anything else I need to do?

[codecov]

Codecov Report

Attention: Patch coverage is 55.14706% with 183 lines in your changes missing coverage. Please review.

Please upload report for BASE (master@41d21eb). Learn more about missing BASE report.

Files	Patch %	Lines
lib/parsers/static_text_parser.js	11.45%	116 Missing ⚠️
lib/utils/nodecrypto.js	0.00%	33 Missing ⚠️
lib/stream.js	61.36%	17 Missing ⚠️
lib/auth_plugins/sha256_password.js	12.50%	7 Missing ⚠️
lib/commands/client_handshake.js	70.00%	3 Missing ⚠️
lib/connection.js	84.61%	2 Missing ⚠️
lib/utils/crypto.js	77.77%	2 Missing ⚠️
lib/auth_plugins/caching_sha2_password.js	92.30%	1 Missing ⚠️
lib/commands/change_user.js	80.00%	1 Missing ⚠️
lib/commands/query.js	88.88%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff            @@
##             master    #2289   +/-   ##
=========================================
  Coverage          ?   89.47%           
=========================================
  Files             ?       76           
  Lines             ?    16021           
  Branches          ?     1363           
=========================================
  Hits              ?    14334           
  Misses            ?     1687           
  Partials          ?        0           

Flag	Coverage Δ
compression-0	89.47% <55.14%> (?)
compression-1	89.47% <55.14%> (?)
tls-0	88.96% <52.45%> (?)
tls-1	89.08% <46.07%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[elithrar]

Add node_compat = true flag to polyfill the missing node modules

Is there a path to instead supporting our modern nodejs_compat mode by updating imports (conditionally) vs using the legacy polyfill mode?

https://developers.cloudflare.com/workers/runtime-apis/nodejs/

[wellwelwel]

I have pushed a new commit to use disableEval instead of useStaticParser . Is there anything else I need to do?

Just a note, while in the docs it says disableEval, in code it's useStaticParser 🙋🏻‍♂️

[shiyuhang0]

I have pushed a new commit to use disableEval instead of useStaticParser . Is there anything else I need to do?

Just a note, while in the docs it says disableEval, in code it's useStaticParser 🙋🏻‍♂️

I forget push the change in code

[focux]

Does this PR works in his current state? Or do we have a list of the things that are missing to collaborate?

[wellwelwel]

Does this PR works in his current state? Or do we have a list of the things that are missing to collaborate?

Hi, everyone! I'd like to propose simplifying the process 🙋🏻‍♂️

Due to many responsibilities in a single PR, rebasing is necessary constantly, in addition to the difficulty of reviewing, I've tried to figure out the changes and the suggestion would be to create one refactoring at a time:

• Adapt asynchronous methods (probably no new tests will be needed, just make sure that all existing tests work, especially the new ones).
• Create the disableEval option/feature (any doubts and ideas about the name can be discussed in its dedicated PR).
  • New tests are essential.
• Create the secureStream method.
  • I couldn't understand if this is a refactoring or a new feature, so if it's a new feature, tests are also important.
• Finally, include pg-cloudflare or mysql2-cloudflare optional dependencies, ensuring the download isn't done directly, plus the Cloudflare Workers documentation, closing the Support Cloudflare workers #2179.

---

By breaking down responsibilities, it's easier both to review and to prevent the constant need to rebase commits.
What do you think?

[Mini256]

Create the disableEval option/feature

@wellwelwel Yes, we can discuss this in PR #2099 (or another dedicated PR) and use it as a precursor PR to support the new non-eval parser in mysql2, avoiding this PR from handling too many tasks. After that, we can rebase this PR.