Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extract the scripting engine code from the functions unit #1312

Open
wants to merge 13 commits into
base: unstable
Choose a base branch
from

Conversation

rjd15372
Copy link
Contributor

@rjd15372 rjd15372 commented Nov 15, 2024

This commit creates a new compilation unit for the scripting engine code by extracting the existing code from the functions unit.

We're doing this refactor to prepare the code for running the EVAL command using different scripting engines.

This PR also fixes #1470.

Signed-off-by: Ricardo Dias [email protected]

@rjd15372 rjd15372 force-pushed the refactor-engine-1261 branch from 40bcded to e8db485 Compare November 15, 2024 09:47
Copy link

codecov bot commented Nov 15, 2024

Codecov Report

Attention: Patch coverage is 80.32787% with 36 lines in your changes missing coverage. Please review.

Project coverage is 70.91%. Comparing base (b3b4bdc) to head (2447b8c).
Report is 27 commits behind head on unstable.

Files with missing lines Patch % Lines
src/scripting_engine.c 74.77% 28 Missing ⚠️
src/module.c 0.00% 4 Missing ⚠️
src/functions.c 95.08% 3 Missing ⚠️
src/server.c 50.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##           unstable    #1312      +/-   ##
============================================
+ Coverage     70.83%   70.91%   +0.07%     
============================================
  Files           120      121       +1     
  Lines         64911    65079     +168     
============================================
+ Hits          45982    46152     +170     
+ Misses        18929    18927       -2     
Files with missing lines Coverage Δ
src/function_lua.c 98.89% <100.00%> (+<0.01%) ⬆️
src/util.c 71.66% <ø> (-0.09%) ⬇️
src/server.c 87.58% <50.00%> (+0.11%) ⬆️
src/functions.c 94.50% <95.08%> (+2.50%) ⬆️
src/module.c 9.60% <0.00%> (+0.01%) ⬆️
src/scripting_engine.c 74.77% <74.77%> (ø)

... and 52 files with indirect coverage changes

@hpatro
Copy link
Collaborator

hpatro commented Nov 15, 2024

@rjd15372 You could point the merge branch as your initial branch. It will show only the diff then.

@rjd15372
Copy link
Contributor Author

rjd15372 commented Nov 15, 2024

@hpatro

@rjd15372 You could point the merge branch as your initial branch. It will show only the diff then.

I don't know if that is possible because the branch that I used to open #1277 is from my own fork of the valkey repo.

@rjd15372 rjd15372 force-pushed the refactor-engine-1261 branch 5 times, most recently from 990447a to 54eb99f Compare November 27, 2024 14:52
@rjd15372 rjd15372 force-pushed the refactor-engine-1261 branch from 54eb99f to 1ae23e8 Compare December 5, 2024 12:24
@rjd15372 rjd15372 force-pushed the refactor-engine-1261 branch 2 times, most recently from d095057 to 375d681 Compare December 23, 2024 10:17
This commit creates a new unit for the scripting engine code by
extracting the existing code from the functions unit.

We're doing this refactor to prepare the code for runnning the `EVAL`
command using different scripting engines.

Signed-off-by: Ricardo Dias <[email protected]>
@rjd15372 rjd15372 force-pushed the refactor-engine-1261 branch from 375d681 to 125394c Compare December 23, 2024 12:08
…eate library callback

Instead of using C strings to return the errors that may be happen
during the code parsing of a scripting engine, use `robj` strings.

This commit also removes the `valkey_asprintf` function, since it's not
used in any place anymore.

Signed-off-by: Ricardo Dias <[email protected]>
@rjd15372 rjd15372 marked this pull request as ready for review December 23, 2024 14:38
@rjd15372
Copy link
Contributor Author

@zuiderkwast @PingXie this refactor is ready for review.

Copy link
Member

@PingXie PingXie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I took a quick look. I'd suggest renaming "engine" to "scripting engine" globally first. We have been using "engine", "core engine", and "Valkey server" interchangeably in our daily conversations, documentation, and the code. We should qualify this new "engine" as "scripting engine" to avoid confusion.

@rjd15372
Copy link
Contributor Author

I took a quick look. I'd suggest renaming "engine" to "scripting engine" globally first. We have been using "engine", "core engine", and "Valkey server" interchangeably in our daily conversations, documentation, and the code. We should qualify this new "engine" as "scripting engine" to avoid confusion.

Sounds good 👍

Copy link
Contributor

@zuiderkwast zuiderkwast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks pretty good. Can you list the changes in the PR top comment, in particular the module API changes?

cmake/Modules/SourceFiles.cmake Outdated Show resolved Hide resolved
src/function_lua.c Show resolved Hide resolved
src/scripting_engine.h Outdated Show resolved Hide resolved
src/scripting_engine.h Outdated Show resolved Hide resolved
* renames the prefix of scripting engine functions from `engine` to `scriptingEngine`
* fixes scripting_engine header file top macro name
* reorders source file list in cmake

Signed-off-by: Ricardo Dias <[email protected]>
Signed-off-by: Ricardo Dias <[email protected]>
Signed-off-by: Ricardo Dias <[email protected]>
Copy link
Member

@PingXie PingXie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM. Will give it once-over once the active comments are resolved. Thanks @rjd15372!

src/scripting_engine.c Show resolved Hide resolved
src/functions.c Show resolved Hide resolved
src/scripting_engine.c Outdated Show resolved Hide resolved
src/server.c Outdated Show resolved Hide resolved
* Changed return type of `engineIterCallback` to `void`
* removed `exit(1)` call after `serverPanic`

Signed-off-by: Ricardo Dias <[email protected]>
Copy link
Contributor

@zuiderkwast zuiderkwast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't finish reviewing but I have some pending comments since last week. Posting them now.

I think this is close to ready to merge.

src/scripting_engine.c Outdated Show resolved Hide resolved
src/scripting_engine.c Outdated Show resolved Hide resolved
src/scripting_engine.c Outdated Show resolved Hide resolved
* Inlines `scriptingEngineImpl` struct inside `scriptingEngine` struct
* Improves log message when engine is already registered

Signed-off-by: Ricardo Dias <[email protected]>
…_memory_overhead`

It also fixes a bug, where we would never decrement the memory overhead
of scripting engines that were unregistered.

Signed-off-by: Ricardo Dias <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: No status
Development

Successfully merging this pull request may close these issues.

Support Detailed Compilation Errors by Allowing Scripting Engine to Return robj Instead of C Strings
5 participants