Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Node 10.1.0, Chimp and fibers problem (can we upgrade fibers?) #116

Open
samhatoum opened this issue Jul 21, 2018 · 33 comments
Open

Node 10.1.0, Chimp and fibers problem (can we upgrade fibers?) #116

samhatoum opened this issue Jul 21, 2018 · 33 comments
Labels

Comments

@samhatoum
Copy link
Contributor

Issue by joscha
Friday May 18, 2018 at 07:40 GMT
Originally opened as xolvio/chimp#690


Expected behaviour

Chimp installs as expected.

Actual behaviour

The fibers package fails to compile.

Exact steps to reproduce / repository that demonstrates the problem

after install of chimp:

    pushd ./node_modules/chimp/node_modules/fibers
    node ./build
Version & tools:
  • Chimp: type chimp -v 0.51.1
  • Node.js: type node -v 10.1.0
  • Operation system: type uname -v ubuntu xenial
Console / Log Output
xenial: gyp info using [email protected]
--
  | xenial: gyp info using [email protected] \| linux \| x64
  | xenial: gyp info spawn /usr/bin/python
  | xenial: gyp info spawn args [ '/usr/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',
  | xenial: gyp info spawn args   'binding.gyp',
  | xenial: gyp info spawn args   '-f',
  | xenial: gyp info spawn args   'make',
  | xenial: gyp info spawn args   '-I',
  | xenial: gyp info spawn args   '/var/lib/buildkite-agent/builds/canva-org/canva-web-build/web/node_modules/fibers/build/config.gypi',
  | xenial: gyp info spawn args   '-I',
  | xenial: gyp info spawn args   '/usr/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',
  | xenial: gyp info spawn args   '-I',
  | xenial: gyp info spawn args   '/var/lib/buildkite-agent/.node-gyp/10.1.0/include/node/common.gypi',
  | xenial: gyp info spawn args   '-Dlibrary=shared_library',
  | xenial: gyp info spawn args   '-Dvisibility=default',
  | xenial: gyp info spawn args   '-Dnode_root_dir=/var/lib/buildkite-agent/.node-gyp/10.1.0',
  | xenial: gyp info spawn args   '-Dnode_gyp_dir=/usr/lib/node_modules/npm/node_modules/node-gyp',
  | xenial: gyp info spawn args   '-Dnode_lib_file=/var/lib/buildkite-agent/.node-gyp/10.1.0/<(target_arch)/node.lib',
  | xenial: gyp info spawn args   '-Dmodule_root_dir=/var/lib/buildkite-agent/builds/canva-org/canva-web-build/web/node_modules/fibers',
  | xenial: gyp info spawn args   '-Dnode_engine=v8',
  | xenial: gyp info spawn args   '--depth=.',
  | xenial: gyp info spawn args   '--no-parallel',
  | xenial: gyp info spawn args   '--generator-output',
  | xenial: gyp info spawn args   'build',
  | xenial: gyp info spawn args   '-Goutput_dir=.' ]
  | xenial: gyp info spawn make
  | xenial: gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
  | xenial: make: Entering directory '/var/lib/buildkite-agent/builds/canva-org/canva-web-build/web/node_modules/fibers/build'
  | xenial:   CXX(target) Release/obj.target/fibers/src/fibers.o
  | xenial: ../src/fibers.cc: In function ‘v8::Handle<v8::String> uni::NewLatin1String(v8::Isolate*, const char*)’:
  | xenial: ../src/fibers.cc:122:64: error: no matching function for call to ‘v8::String::NewFromOneByte(v8::Isolate*&, const uint8_t*)’
  | xenial:    return String::NewFromOneByte(isolate, (const uint8_t*)string);
  | xenial:                                                                 ^
  | xenial: In file included from /var/lib/buildkite-agent/.node-gyp/10.1.0/include/node/node.h:63:0,
  | xenial:                  from ../src/coroutine.h:1,
  | xenial:                  from ../src/fibers.cc:1:
  | xenial: /var/lib/buildkite-agent/.node-gyp/10.1.0/include/node/v8.h:2736:51: note: candidate: static v8::MaybeLocal<v8::String> v8::String::NewFromOneByte(v8::Isolate*, const uint8_t*, v8::NewStringType, int)
  | xenial:    static V8_WARN_UNUSED_RESULT MaybeLocal<String> NewFromOneByte(
  | xenial:                                                    ^
  | xenial: /var/lib/buildkite-agent/.node-gyp/10.1.0/include/node/v8.h:2736:51: note:   candidate expects 4 arguments, 2 provided
  | xenial: ../src/fibers.cc: In function ‘v8::Handle<v8::String> uni::NewLatin1Symbol(v8::Isolate*, const char*)’:
  | xenial: ../src/fibers.cc:126:64: error: no matching function for call to ‘v8::String::NewFromOneByte(v8::Isolate*&, const uint8_t*)’
  | xenial:    return String::NewFromOneByte(isolate, (const uint8_t*)string);
  | xenial:                                                                 ^
  | xenial: In file included from /var/lib/buildkite-agent/.node-gyp/10.1.0/include/node/node.h:63:0,
  | xenial:                  from ../src/coroutine.h:1,
  | xenial:                  from ../src/fibers.cc:1:
  | xenial: /var/lib/buildkite-agent/.node-gyp/10.1.0/include/node/v8.h:2736:51: note: candidate: static v8::MaybeLocal<v8::String> v8::String::NewFromOneByte(v8::Isolate*, const uint8_t*, v8::NewStringType, int)
  | xenial:    static V8_WARN_UNUSED_RESULT MaybeLocal<String> NewFromOneByte(
  | xenial:                                                    ^
  | xenial: /var/lib/buildkite-agent/.node-gyp/10.1.0/include/node/v8.h:2736:51: note:   candidate expects 4 arguments, 2 provided
  | xenial: ../src/fibers.cc: In static member function ‘static void Fiber::DestroyOrphans()’:
  | xenial: ../src/fibers.cc:406:67: warning: ‘v8::String::Utf8Value::Utf8Value(v8::Local<v8::Value>)’ is deprecated: Use Isolate version [-Wdeprecated-declarations]
  | xenial:       String::Utf8Value stack(uni::Deref(that.isolate, fatal_stack));
  | xenial:                                                                    ^
  | xenial: In file included from /var/lib/buildkite-agent/.node-gyp/10.1.0/include/node/v8.h:26:0,
  | xenial:                  from /var/lib/buildkite-agent/.node-gyp/10.1.0/include/node/node.h:63,
  | xenial:                  from ../src/coroutine.h:1,
  | xenial:                  from ../src/fibers.cc:1:
  | xenial: /var/lib/buildkite-agent/.node-gyp/10.1.0/include/node/v8.h:2822:28: note: declared here
  | xenial:                    explicit Utf8Value(Local<v8::Value> obj));
  | xenial:                             ^
  | xenial: /var/lib/buildkite-agent/.node-gyp/10.1.0/include/node/v8config.h:318:3: note: in definition of macro ‘V8_DEPRECATED’
  | xenial:    declarator __attribute__((deprecated(message)))
  | xenial:    ^
  | xenial: ../src/fibers.cc: In static member function ‘static uni::FunctionType Fiber::New(const Arguments&)’:
  | xenial: ../src/fibers.cc:433:99: error: no matching function for call to ‘v8::Function::NewInstance(int, v8::Handle<v8::Value> [1])’
  | xenial:      return uni::Return(uni::Deref(Isolate::GetCurrent(), tmpl)->GetFunction()->NewInstance(1, argv), args);
  | xenial:                                                                                                    ^
  | xenial: In file included from /var/lib/buildkite-agent/.node-gyp/10.1.0/include/node/node.h:63:0,
  | xenial:                  from ../src/coroutine.h:1,
  | xenial:                  from ../src/fibers.cc:1:
  | xenial: /var/lib/buildkite-agent/.node-gyp/10.1.0/include/node/v8.h:3848:44: note: candidate: v8::MaybeLocal<v8::Object> v8::Function::NewInstance(v8::Local<v8::Context>, int, v8::Local<v8::Value>*) const
  | xenial:    V8_WARN_UNUSED_RESULT MaybeLocal<Object> NewInstance(
  | xenial:                                             ^
  | xenial: /var/lib/buildkite-agent/.node-gyp/10.1.0/include/node/v8.h:3848:44: note:   candidate expects 3 arguments, 2 provided
  | xenial: /var/lib/buildkite-agent/.node-gyp/10.1.0/include/node/v8.h:3851:44: note: candidate: v8::MaybeLocal<v8::Object> v8::Function::NewInstance(v8::Local<v8::Context>) const
  | xenial:    V8_WARN_UNUSED_RESULT MaybeLocal<Object> NewInstance(


Join our Slack xolv.io/community #chimp channel, where you can find help and help others.

@samhatoum
Copy link
Contributor Author

Comment by miendt
Tuesday May 22, 2018 at 03:44 GMT


You should downgrade node version to v8.10.0

@samhatoum
Copy link
Contributor Author

Comment by joscha
Tuesday May 22, 2018 at 03:59 GMT


You should downgrade node version to v8.10.0

well it works on 9.x right now, but just in general I'd like to advance with Node versions instead of downgrading and staying on older versions.

@samhatoum
Copy link
Contributor Author

Comment by celador
Wednesday May 23, 2018 at 17:08 GMT


+1 on this one. Problem suddenly occurred during our CI process today.

@samhatoum
Copy link
Contributor Author

Comment by thebarty
Thursday Jun 14, 2018 at 08:27 GMT


Same problem over here.

@joscha what's the current state. What's your workaround?

@samhatoum
Copy link
Contributor Author

Comment by joscha
Thursday Jun 14, 2018 at 08:34 GMT


@joscha what's the current state. What's your workaround?

No good workaround unfortunately - we will most likely resort to the custom resolutions that yarn offers and then see if we can get on chimp vNext asap

@samhatoum
Copy link
Contributor Author

Comment by joscha
Tuesday Jun 26, 2018 at 03:58 GMT


Upgrading/resolving fibers to 2.0.2 doesn't seem to work - probably because of laverdet/node-fibers#379 - will try HEAD.

@samhatoum
Copy link
Contributor Author

Comment by joscha
Tuesday Jun 26, 2018 at 04:12 GMT


I take that back, this seems to work:

index d0d1e12a40a..d2b41f8f023 100644
--- a/web/package.json
+++ b/web/package.json
@@ -159,7 +159,7 @@
-    "node-gyp": "^3.6.2",
+    "node-gyp": "^3.7.0",
@@ -245,10 +245,13 @@
   "resolutions": {
+    "chimp/fibers": "^2.0.2",
+    "chimp/**/fibers": "^2.0.2",
+    "chimp/fibers/node-gyp": "^3.7.0"
   },
 }

fsevents still errors, possibly it needs a resolution as well or chokidar which uses fsevents in chimp.

@samhatoum
Copy link
Contributor Author

Comment by joscha
Tuesday Jun 26, 2018 at 07:08 GMT


another resolution for

  "fsevents": "^1.2.4",

fixes it. That is quite widespread though, so might be worth waiting for it to be picked up separately by projects.

@samhatoum
Copy link
Contributor Author

Comment by joscha
Wednesday Jul 11, 2018 at 05:06 GMT


@samhatoum any chance to get these deps bumped?

@mpeltonen
Copy link

Node.js 10.x LTS starting next week, it would be awesome to get Chimpy compiling with it. I hope we get a new release with these dependency updates soon 👍

sirphoenix added a commit to sirphoenix/chimpy that referenced this issue Dec 27, 2018
sirphoenix added a commit to sirphoenix/chimpy that referenced this issue Dec 27, 2018
@cy
Copy link

cy commented Feb 20, 2019

looks like [c2aaa17](/sirphoenix/chimpy/commit/c2aaa17650719c094256743d8c952468449d8b82) and [6ad8a6b](/sirphoenix/chimpy/commit/6ad8a6b78cfae388c868d03465d5612d1ef8611b) on @sirphoenix's fork fixes the issue? Can we get that or similar merged and released?

@wolasss
Copy link

wolasss commented Jun 11, 2019

Bump. Is there anything we can do as a community to speed this up?

@joscha
Copy link
Contributor

joscha commented Jun 11, 2019 via email

@mpeltonen
Copy link

Yeah, it's been sad to see the project effectively dying. Also currently investigating options for replacement.

@wolasss
Copy link

wolasss commented Jun 11, 2019

@mpeltonen @joscha thanks guys for pointing that out. I have moved to using webdriver.io and cucumber directly, works as expected on CI

@samhatoum
Copy link
Contributor Author

samhatoum commented Jun 11, 2019

@joschs it's a little unfair to say that you didn't get a dignified response. I got an email from you in late 2018 to which I responded with:

Hi Joscha

I understand your concerns. I'll do my best to answer/address them below.

Chimp was under heavy development at one point and I devoted a lot of unpaid time to contribute towards open source. Since then, I've not been able to give the same amount of free time (family, life and such) and I came to an agreement with TBSH to take over as maintainers. I was actually going to stop development altogether but TBSH were using it for many of their clients and didn't want to see it fade away. They have since added the "retry" function on the 3rd of August, which wasn't that long ago. I'm sure if you mention one of the maintainers on the PR that would give them a little kick up the backside to do the merge! That's how I operated for a while.

The blog post states that nothing will happen with Chimp 2.0 until 2019, and this is true, but this doesn't affect your project since your integration tests are written for Chimp1.0/Chimpy. Chimp 2.0 will not be backward compatible with your existing test suite so I wouldn't even look at that for your current needs, I would only look to Chimpy.

As for you moving off Chimp as fast as you can, I think the answer to that depends on how the low development of Chimp has been affecting you, how many tests you have, the cost of migrating these tests to a new framework, and if the new framework will meet your needs. If you're using Meteor, it's hard to find a better option despite the low updates over the past year.

As for getting Percy integration into Chimpy, your best options as I see them are as follows:

  1. Engage TBSH to do the integration for you in Chimpy
  2. Create your own PR for the integration in Chimpy
  3. Migrate over to WDIO runner and use their Percy integration. (WDIO will offer you the least path of resistance since Chimp was built with Webdriver.io in the background, but you'll have to find other ways to use any Meteor-specific features within Chimp)

I'm happy to help you guys get to where you need to get to. If you're interested in option 1, then I can provide an intro. If you're interested in options 2 or 3, then I can have a call with you and point you in the right direction.

Let me know which way you want to go.

Cheers

Sam

Now it's true that both I and The Brain have been neglecting Chimp and that's because I believe it's shelf life has ended, especially with tools like Webdriver.io having caught up to most of Chimps functionality, and Cypress offering a great alternative to the WD protocol.

In any case, I've just agreed with @lgandecki to get these PRs merged on a shared cost basis as I don't want the work of others to be wasted. Sorry about all the delays guys.

@wolasss or anyone else, if you're interested in being maintainers it would be great to add you.

Thanks to you all and please reach out if you need assistance/advice in migrating.

@lgandecki
Copy link
Member

From my perspective, we here at TBSH are still using it for a few projects, including one very large one - with almost 1k chimpy tests. But, as Sam mentioned, there are other similar projects with heavy investments behind them (wdio, cypress, to mention just two) that caught up and basically ran circles around chimp since it was released. We don't start new projects with chimpy, and we don't recommend it. The only reasonable usecase for chimpy is it's great Meteor integration, and since Meteor doesn't work on Node 10.1.0, this doesn't seem like a high priority issue for us.

Could you guys explain to use what usecase are you having for Node 10? Are you using chimp for non-meteor projects? Are those new projects, or something that you invested time with building chimp tests already in the past?

Nonetheless, I'm about to do some merging/testing and will release a new version today.

lgandecki added a commit that referenced this issue Jun 11, 2019
fix fibers build on node v10.x (#116)
@joscha
Copy link
Contributor

joscha commented Jun 11, 2019

@joschs it's a little unfair to say that you didn't get a dignified response.

excuse me, but my sentence read:

I opened pull requests to both, none of them even dignified with an answer.

in full. Your references is out of context.

But apart from that, given that this PR was opened a year ago, similar to xolvio/chimp#696 and further PRs (also a few months ago) such as #127 and #139 (and these are only mine, there are boatloads more from other people), I'd say that is very dignified.

Especially after ignoring my PRs and then offering me help for pay when I reach out through a mutual contact. Now I understand that OSS is hard and I understand needing compensation for some of the work and there is nothing wrong with that, but there are a bunch of really fine people and PRs in these two projects that tried to make it work and they were completely ignored and thrown off by the mess between the two projects, two companies and current state and went nowhere.

This is nothing personal, but if you come across this page: I can really not recommend setting on this project to anyone, paid or otherwise.

Migrating off is not even that easy because you have to choose between wdio4 & cucumber but no properly working watch mode or wdio5 and watch mode but no properly working cucumber framework.

@joscha
Copy link
Contributor

joscha commented Jun 11, 2019

The only reasonable usecase for chimpy is it's great Meteor integration, and since Meteor doesn't work on Node 10.1.0, this doesn't seem like a high priority issue for us.

We never used chimp with meteor. The reason we chose chimp was because:

  • it bundles all the tools we wanted in perfect orchestration (wdio, config, cucumber)
  • its watch mode is unrivalled by any other runner and works with Typescript and other transpilers out of the box

Could you guys explain to use what usecase are you having for Node 10?

Node 10 is the current LTS version. Well, was, now we're at 12 I suppose, but I haven't tried if Chimp is any more happy with that.

Are you using chimp for non-meteor projects? Are those new projects, or something that you invested time with building chimp tests already in the past?

Ours is 2.5 years old with hundreds of tests :(

@lgandecki
Copy link
Member

Just did a bit of merging, tweaking, testing and released the 1.0 version that as far as I can tell (verified with a simple repo so far - https://github.com/TheBrainFamily/chimpy-1.0-tests ) works on both node 10 and 8 (doesn't work on 6, but I guess no one should be upgrading chimp if they are still on node 6).

Sorry for the delay guys. This is a difficult project to work with, and to be honest we pushed the idea of "if it doesn't break, don't touch" to the limits, probably a bit too far.
With about 500 installs a week (large percentage of being us) on the new npm name (since Sam wanted to keep "chimp" for other project..), it never seemed like a good investment of our time to add more things to it, as long as it worked for us. It's one of our LEAST popular packages. And at the same time, one of the most difficult to work with (because of all the dependencies, hacks, almost no tests...).

@lgandecki
Copy link
Member

@joscha could you say a bit more about wdio5 and not properly working cucumber framework? I did actually work on the wdio/cucumber integration, and am one of the contributors with RW access to the webdriverio repositories, so I might be able to help a bit here.

@joscha
Copy link
Contributor

joscha commented Jun 11, 2019

@supercrabtree could you please share all your investigations with @lgandecki?

@joscha
Copy link
Contributor

joscha commented Jun 11, 2019

With about 500 installs a week (large percentage of being us) on the new npm name

FWIW we are still using the original chimp package but have since moved to a fork that contains #139. Also our node modules are cached, so the npm install stats would be off anyway. But if chimpy was maintained and we didn't have the caching, etc. it would be thousands of installs a week - chances are there are more companies like this.

@samhatoum
Copy link
Contributor Author

@joscha apologies for the wrong reference, and I'm glad you understand the difficulty of OSS.

I do want to correct something you're dating, which is that I never offered you help for pay, I offered an intro for custom dev work for getting Chimp to work with Percy, which was your requirement.

In any case, let's move forward and get these PRs in for you guys.

@joscha
Copy link
Contributor

joscha commented Jun 11, 2019

I do want to correct something you're dating, which is that I never offered you help for pay, I offered an intro for custom dev work for getting Chimp to work with Percy, which was your requirement.

Your email (quoted above) has three points 1) pay, 2) open PRs 3) migrate to wdio. With 3 not being an option with the missing cucumber support, 2 and 1 were left. Looking at the responsiveness to other pull requests, 2 was not an option, which left me feeling this convo that I either pay or forget this project.

@samhatoum
Copy link
Contributor Author

Yes, but not pay me @joscha, that's the key there. I was genuinely trying to help you and I still am by dedicating folks to get these PRs dealt with. Please see that

@supercrabtree
Copy link

Hi @lgandecki, unless I have missed something cucumber is not currently supported for webdriverio 5. In the docs https://webdriver.io/docs/frameworks.html it says up the top:

The wdio runner currently supports Mocha and Jasmine and Cucumber (not yet supported in v5).

I was currently watching this PR webdriverio/webdriverio#3667 but am I missing something? Is it already supported? Cheers :)

@mpeltonen
Copy link

Thanks a lot @lgandecki and @samhatoum for the merges and new release, much appreciated!

@samhatoum
Copy link
Contributor Author

Welcome @mpeltonen

@lgandecki I think you're talking about an older version of the cucumber 2 plugin for WDIO (this one https://github.com/lgandecki/wdio-cucumber2-framework). The guys are looking for a Webdriver 5 version.

There is an actively open PR mentioned above which looks like it may not be far off now. If it doesn't progress, then @joscha and @supercrabtree please let us know and we can dedicate some time towards helping with a cucumber plugin for Webdriver5, and perhaps you can then help us to write a migration guide that helps the community :)

@supercrabtree
Copy link

Fantastic @samhatoum thank you very much. I'd be more than happy to write a migration guide, there was few pitfalls along the way that took me a while to figure out so if I can help others avoid them that would be great 👍 I also wrote a utility function that allows the cucumber tests to be written in both the older v1 style and the newer (v4?) style, this was a huge help in breaking down giant PRs and allowed us to migrate piece by piece. I'd be happy to share this too 🙂

@samhatoum
Copy link
Contributor Author

Thanks @supercrabtree, that would be fab. Keep us posted with how you get on and then we can add to the README of this project to allow others to benefit from your work, and we can reference the utility function that you wrote.

@abjerstedt
Copy link

The cucumber v5 support for webdriverio is close, but simultaneously far away.

I delivered the code itself (before bug fixes), but we need to write lots and lots of Unit tests (We have a 100% code coverage requirement for merges). The problem is, I don't have the time to work on it at the moment (its summer!).

@samhatoum
Copy link
Contributor Author

Looks like it just got completed:
https://webdriver.io/blog/2019/07/11/cucumberjs-v5-released.html

@supercrabtree let us know how you get on with the conversion once you get to it and shout if you need any help

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

8 participants