Requiring jxcore addon from "embedded" C++ mechanism works on Mac but not on Windows

[csapoadam]

I am trying to run embedded javascript code within a C++ program (like in the project here https://github.com/obastemur/JXcoreWindowsEmbedded ). This works fine on both Mac and Windows unless the code uses require to import a previously compiled jxcore addon - in which case it only works on Mac (but not Windows).

For example, if I have a .js file that has:

var binding = require('./build/Release/binding');
console.log("1 + 2 = " + binding.sum(1,2));

(where binding is the addon I created), and run this through the jx interpreter, then I have the desired result. But if in C++, I do:

std::stringstream ss;
ss << "console.log("Adding 1 and 2 together "); \n";
ss << "var binding0 = process.requireGlobal('./build/Release/testadd'); \n";
ss << "var binding1 = process.requireGlobal('./build/Release/binding'); \n";
ss << "console.log(binding1.sum(1,2)); \n";
JX_Evaluate(ss.str().c_str(), "myscript", &result);

I get an invalid memory access error during runtime. Note that testadd.js is a simple js script that adds the two numbers together without using the binding addon. Thus, the above example only fails when requiring the addon. If I require the addon in the separate script (for example, inside testadd.js), the code still fails. The error message is as follows:

Error: invalid access to memory location
c:\users\adam\desktop\development\maxwhere-tests\build\release\binding.node (nat
ive:jxcore_js_object 3:11)
at (native:jxcore_js_object:3:12)
at process.dlopen (node.js:638:17)
at Module.prototype.load (module.js:347:7)
at Module._load (module.js:314:5)
at Module.prototype.require (module.js:379:10)
at require (module.js:397:12)
at myscript:3:16

Do you know what causes this? I am using the SM version on Windows7 (compiled with VS2012). The same example works on Mac.

[obastemur]

@csapoadam You require a native binary from an embedded app right? I believe the app is failing to locate reference binary for native addon. It's normal that this is only happening on Windows. On Windows, when you require a 'node' file, it tries to find the reference methods etc. from 'jx.exe'

Could you fork JXcore embedded windows project and edit it to something that I can reproduce the problem? That would help a lot!

[csapoadam]

Hi @obastemur , I forked that project and changed it to demonstrate the problem I am having on Win7 with the SM version. The sources can be found here:

https://github.com/csapoadam/JXcoreWindowsEmbedded

Thanks for your help.

[csapoadam]

Hi, do you have any updates on this? Could you reproduce the problem?
Thanks!

[obastemur]

@csapoadam Thanks for the work there. I am working on it.

[obastemur]

It worked for me. What I did.

1 - compiled the jxcore-addon project
2 - opened your sln under Visual Studio
3 - compiled it under debug x64
4 - compiled jxcore as dll (v8 x64) under Windows (vcbuild.bat --shared-library x64)
5 - copy Release\jx.dll into sample_solution\x64\Debug
6 - copy js files and build dir from the root folder into sample_solution_x64_Debug
7 - run application

[csapoadam]

Hi @obastemur , that's correct, with v8 it works fine. I just tried it on my windows computer. However, the issue was with the SpiderMonkey version. I tried this again now, with the most up-to-date sources, and the SM version still doesn't work. Actually, now (with the latest sources), even doing jx install to build the addon gives this error:

c:\users\adam.node-gyp\jxb303\src\jx\proxy\Mozilla/JXString.h(57): error C2071
: 'jxcore::JXString::ctx_' : illegal storage class [C:\Users\Adam\Desktop\Devel
opment\embedded-addon\build\binding.vcxproj]

this seems to be a new error (or rather, a recurrence of an earlier error), because when I reverted to the sources from June 6 (the ones I used when starting this ticket), jx install worked fine, but then of course I still had the problem described above, with the invalid memory access.

It's true that on Windows one can use V8 without a problem, but on some plaforms (like iOS), currently the only alternative is SpiderMonkey. That's why I tried it on Windows with SM, too, and I had these errors. I thought it might be important to investigate this.

So I will use the V8 version for the time being, but I think there are still definitely some problems with SM.

[obastemur]

@csapoadam I thought you had the issue with V8. Gonna check with SM then. Thanks for the details

[obastemur]

Just tested and it also works for SM.

Before compiling the solution, make sure you have the jx.lib from JXcore SM.

[csapoadam]

I don't know what's wrong on my end then... :S Truth is though, I am using a 32-bit version of Windows, with VS2012. Other than that, I don't think I'm doing anything differently.

With the very latest sources, the problem can't be with the VS solution (libs, etc.), because just doing 'jx install' in the main folder (in order to install the addon) fails with:
c:\users\adam.node-gyp\jxb303\src\jx\proxy\Mozilla/JXString.h(57): error C2071
: 'jxcore::JXString::ctx_' : illegal storage class [C:\Users\Adam\Desktop\Devel
opment\embedded-addon\build\binding.vcxproj]

With the older sources, installing the addon works fine... then I go on to build the embedded c++ application, which works fine... then I copy the executable to the main folder and try to run it, but I get this error:
Error: invalid access to memory location
c:\users\adam\desktop\development\maxwhere-tests\build\release\binding.node (nat
ive:jxcore_js_object 3:11)
at (native:jxcore_js_object:3:12)
at process.dlopen (node.js:638:17)
at Module.prototype.load (module.js:347:7)
at Module._load (module.js:314:5)
at Module.prototype.require (module.js:379:10)
at require (module.js:397:12)
at myscript:3:16

If it's not the same on your end, it could just be a problem with my configuration (win7 32-bit plus VS2012).

[obastemur]

32Bit. That's the problem :) I will try to install 32 bit Windows whenever possible. Thanks

[csapoadam]

OK, I am installing 64-bit windows now to try this out. I will post an update in a few days. Thanks for your help.

[csapoadam]

Hi @obastemur , it's working for me (somewhat), but actually it's not completely doing what it should. With the example I sent you, here is the code for the script:

JXValue result;
std::stringstream ss;
ss << "console.log("Adding 1 and 2 together "); \n";
ss << "var binding0 = process.requireGlobal('./test_direct'); \n";
ss << "var binding1 = process.requireGlobal('./test_require'); \n";
ss << "var binding2 = process.requireGlobal('./build/Release/binding'); \n";
ss << "console.log(binding2.sum(1,2)); \n";
JX_Evaluate(ss.str().c_str(), "myscript", &result);

so, it should print:
"Adding 1 and 2 together
res is 3
3"

but it is printing only the first two lines, which means that the binding2 variable is never initialized correctly:

This is with the V8 version on 64-bit Windows...
Are you getting the same results?

[csapoadam]

Actually it should print 4 lines but I forgot to use process.requireGlobal instead of require in the second script.

However, neither correcting that nor removing it from main.cpp helps. The bottom line is that addons do not seem to work in embedded applications on windows.

[obastemur]

I've just figured out why. @csapoadam it definitely doesn't work for Windows.

Sample native app loads jx.dll. JX.dll loads binding.node. Binding.node requires jx.exe. As a result app crashes. It is v8 or SM independent and mostly because of how node native addon works.

It's true that on Windows one can use V8 without a problem, but on some plaforms (like iOS), currently the only alternative is SpiderMonkey.

Dynamic library loading on iOS is not allowed. If you give me some more details on what you are trying to achieve, I would like to help.

[csapoadam]

Hi, thanks for that info. So it seems like with Mac, it's possible to do this but not on Windows? Is the fact that it worked on Mac a coincidence, or will this be possible in future versions, too? (I don't really know the details of how addons work).

The reason why we wanted to do this (require addon from embedded code) was because we are looking to develop a cross-platform solution (win, osx, ios, android) that is capable of:

• embedding node.js runtime into a native c++ application.
• handling realtime bidirectional communication (object sharing) between the c++ and node scope (this was why we wanted the addon and native application to run in the same process).
• doing all the above with full node compatibility and ability to use existing node modules right out of npm.

Do you see any possibility for doing this? Perhaps if we are loading binding.node from an embedded application, there would be no need to require jx.exe again (since jx.dll is already loaded?).

[obastemur]

Is the fact that it worked on Mac a coincidence, or will this be possible in future versions, too?

It's not a coincidence. It works.

Do you see any possibility for doing this?

Yes sure. The problem with Windows may require some time taking internal tricks but doable.

Perhaps if we are loading binding.node from an embedded application, there would be no need to require jx.exe again (since jx.dll is already loaded?)

This is the part that requires the time taking work on Windows.

[csapoadam]

Sounds great, thanks!

Once you have a roadmap, let me know what timeframe you see realistic for these modifications.

[csapoadam]

Hi @obastemur , do you have any updates on this? Thanks!

[obastemur]

@csapoadam could not dig on this yet. If you have a chance to look into I can provide you some details

[csapoadam]

Hi @obastemur , I've been thinking about this and would love to contribute, but unfortunately there are quite a few projects I have to get through now. Let's talk about this later, once we have more time.