Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect monorepo package resolution? @glimmer/syntax -> @glimmer/util, but in ember-source, except not found #817

Closed
NullVoxPopuli opened this issue May 20, 2021 · 11 comments

Comments

@NullVoxPopuli
Copy link
Collaborator

NullVoxPopuli commented May 20, 2021

ref: glimmerjs/glimmer.js#345

Reason why I'm thinking this might be a dependency resolution issue,
In this PR where renovate upgrades my app from 3.26.x to 3.27.x
I get the error

Build Error (PackagerRunner) in ../../../../home/me/Development/NullVoxPopuli/limber/frontend/node_modules/@glimmer/syntax/dist/modules/es2017/lib/parser.js

Module not found: Error: Can't resolve 'ember-source/@glimmer/util.js' in '/home/me/Development/NullVoxPopuli/limber/frontend/node_modules/@glimmer/syntax/dist/modules/es2017/lib/parser.js'
Log
=================================================================================

ENV Summary:

  TIME: Thu May 20 2021 07:39:40 GMT-0400 (Eastern Daylight Time)
  TITLE: ember
  ARGV:
  - /home/me/.volta/tools/image/node/14.17.0/bin/node
  - /home/me/Development/NullVoxPopuli/limber/frontend/node_modules/.bin/ember
  - serve
  EXEC_PATH: /home/me/.volta/tools/image/node/14.17.0/bin/node
  TMPDIR: /tmp
  SHELL: /bin/bash
  PATH:
  - /tmp/yarn--1621510740646-0.05927598237875187
  - /home/me/Development/NullVoxPopuli/limber/frontend/node_modules/.bin
  - /home/me/.config/yarn/link/node_modules/.bin
  - /home/me/Development/NullVoxPopuli/limber/node_modules/.bin
  - /home/me/.volta/tools/image/node/14.17.0/libexec/lib/node_modules/npm/bin/node-gyp-bin
  - /home/me/.volta/tools/image/node/14.17.0/lib/node_modules/npm/bin/node-gyp-bin
  - /home/me/.volta/tools/image/node/14.17.0/bin/node_modules/npm/bin/node-gyp-bin
  - /tmp/yarn--1621510740233-0.7965985169595542
  - /home/me/Development/NullVoxPopuli/limber/node_modules/.bin
  - /home/me/.config/yarn/link/node_modules/.bin
  - /home/me/Development/NullVoxPopuli/limber/node_modules/.bin
  - /home/me/.yarn/bin
  - /home/me/.volta/tools/image/node/14.17.0/libexec/lib/node_modules/npm/bin/node-gyp-bin
  - /home/me/.volta/tools/image/node/14.17.0/lib/node_modules/npm/bin/node-gyp-bin
  - /home/me/.volta/tools/image/node/14.17.0/bin/node_modules/npm/bin/node-gyp-bin
  - /home/me/.volta/tools/image/npm/7.11.1/bin
  - /home/me/.volta/tools/image/yarn/1.22.10/bin
  - /home/me/.volta/tools/image/node/14.17.0/bin
  - /home/me/.volta/bin
  - /home/me/.pythons/Python-3.6.3/bin
  - /home/me/.cargo/bin
  - /home/me/Applications
  - /home/me/apps/phantomjs/bin
  - /home/me/scripts/system-utils
  - /home/me/scripts/git
  - /home/me/scripts/rails
  - /home/me/scripts
  - /home/me/.volta/bin
  - /usr/local/sbin
  - /usr/local/bin
  - /usr/sbin
  - /usr/bin
  - /sbin
  - /bin
  - /usr/games
  - /usr/local/games
  - /snap/bin
  - /home/me/.dotnet/tools
  - /home/me/.dotnet/tools
  - /home/me/.fzf/bin
  PLATFORM: linux x64
  FREEMEM: 277475328
  TOTALMEM: 17388204032
  UPTIME: 421813.43
  LOADAVG: 2.24,0.84,0.35
  CPUS:
  - Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz - 3192
  - Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz - 3192
  - Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz - 3192
  - Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz - 3192
  - Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz - 3192
  - Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz - 3192
  ENDIANNESS: LE
  VERSIONS:
  - ares: 1.17.1
  - brotli: 1.0.9
  - cldr: 38.1
  - icu: 68.2
  - llhttp: 2.1.3
  - modules: 83
  - napi: 8
  - nghttp2: 1.42.0
  - node: 14.17.0
  - openssl: 1.1.1k
  - tz: 2020d
  - unicode: 13.0
  - uv: 1.41.0
  - v8: 8.4.371.23-node.63
  - zlib: 1.2.11

ERROR Summary:

  - broccoliBuilderErrorStack: ModuleNotFoundError: Module not found: Error: Can't resolve 'ember-source/@glimmer/util.js' in '/home/me/Development/NullVoxPopuli/limber/frontend/node_modules/@glimmer/syntax/dist/modules/es2017/lib'
    at /home/me/Development/NullVoxPopuli/limber/node_modules/webpack/lib/Compilation.js:1762:28
    at /home/me/Development/NullVoxPopuli/limber/node_modules/webpack/lib/NormalModuleFactory.js:730:13
    at eval (eval at create (/home/me/Development/NullVoxPopuli/limber/node_modules/webpack/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:10:1)
    at /home/me/Development/NullVoxPopuli/limber/node_modules/webpack/lib/NormalModuleFactory.js:273:22
    at eval (eval at create (/home/me/Development/NullVoxPopuli/limber/node_modules/webpack/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:9:1)
    at /home/me/Development/NullVoxPopuli/limber/node_modules/webpack/lib/NormalModuleFactory.js:402:22
    at /home/me/Development/NullVoxPopuli/limber/node_modules/webpack/lib/NormalModuleFactory.js:117:11
    at /home/me/Development/NullVoxPopuli/limber/node_modules/webpack/lib/NormalModuleFactory.js:646:24
    at /home/me/Development/NullVoxPopuli/limber/node_modules/webpack/lib/NormalModuleFactory.js:800:8
    at /home/me/Development/NullVoxPopuli/limber/node_modules/webpack/lib/NormalModuleFactory.js:920:5
  - code: [undefined]
  - codeFrame: Module not found: Error: Can't resolve 'ember-source/@glimmer/util.js' in '/home/me/Development/NullVoxPopuli/limber/frontend/node_modules/@glimmer/syntax/dist/modules/es2017/lib/parser.js'
  - errorMessage: ../../../../home/me/Development/NullVoxPopuli/limber/frontend/node_modules/@glimmer/syntax/dist/modules/es2017/lib/parser.js: Module not found: Error: Can't resolve 'ember-source/@glimmer/util.js' in '/home/me/Development/NullVoxPopuli/limber/frontend/node_modules/@glimmer/syntax/dist/modules/es2017/lib/parser.js'
        at PackagerRunner (@embroider/webpack)
  - errorType: Build Error
  - location:
    - column: [undefined]
    - file: ../../../../home/me/Development/NullVoxPopuli/limber/frontend/node_modules/@glimmer/syntax/dist/modules/es2017/lib/parser.js
    - line: [null]
    - treeDir: [undefined]
  - message: ../../../../home/me/Development/NullVoxPopuli/limber/frontend/node_modules/@glimmer/syntax/dist/modules/es2017/lib/parser.js: Module not found: Error: Can't resolve 'ember-source/@glimmer/util.js' in '/home/me/Development/NullVoxPopuli/limber/frontend/node_modules/@glimmer/syntax/dist/modules/es2017/lib/parser.js'
        at PackagerRunner (@embroider/webpack)
  - name: Error
  - nodeAnnotation: @embroider/webpack
  - nodeName: PackagerRunner
  - originalErrorMessage: Module not found: Error: Can't resolve 'ember-source/@glimmer/util.js' in '/home/me/Development/NullVoxPopuli/limber/frontend/node_modules/@glimmer/syntax/dist/modules/es2017/lib/parser.js'
  - stack: ModuleNotFoundError: Module not found: Error: Can't resolve 'ember-source/@glimmer/util.js' in '/home/me/Development/NullVoxPopuli/limber/frontend/node_modules/@glimmer/syntax/dist/modules/es2017/lib'
    at /home/me/Development/NullVoxPopuli/limber/node_modules/webpack/lib/Compilation.js:1762:28
    at /home/me/Development/NullVoxPopuli/limber/node_modules/webpack/lib/NormalModuleFactory.js:730:13
    at eval (eval at create (/home/me/Development/NullVoxPopuli/limber/node_modules/webpack/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:10:1)
    at /home/me/Development/NullVoxPopuli/limber/node_modules/webpack/lib/NormalModuleFactory.js:273:22
    at eval (eval at create (/home/me/Development/NullVoxPopuli/limber/node_modules/webpack/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:9:1)
    at /home/me/Development/NullVoxPopuli/limber/node_modules/webpack/lib/NormalModuleFactory.js:402:22
    at /home/me/Development/NullVoxPopuli/limber/node_modules/webpack/lib/NormalModuleFactory.js:117:11
    at /home/me/Development/NullVoxPopuli/limber/node_modules/webpack/lib/NormalModuleFactory.js:646:24
    at /home/me/Development/NullVoxPopuli/limber/node_modules/webpack/lib/NormalModuleFactory.js:800:8
    at /home/me/Development/NullVoxPopuli/limber/node_modules/webpack/lib/NormalModuleFactory.js:920:5

=================================================================================

And here is where I think it might be a resolution problem, ember-source is in my monorepo-root node_modules (with @glimmer/util.js):

❯ ls node_modules/ember-source/dist/dependencies/@glimmer/
destroyable.js  encoder.js  env.js  global-context.js  low-level.js  manager.js  node.js  opcode-compiler.js  owner.js  program.js  reference.js  runtime.js  util.js  validator.js  vm.js  wire-format.js

the same path does not exist in my app's local node_modules

ls frontend/node_modules/ember-source/
ls: cannot access 'frontend/node_modules/ember-source/': No such file or directory

in my app's package.json:

"ember-source": "3.27.1", (and higher, 3.28 beta, 4 alpha)
"ember-cli-babel": "^7.26.6",
"@glimmer/syntax": "^0.79.1",
@ef4
Copy link
Contributor

ef4 commented May 20, 2021

The problem here is probably that we are looking at this file at all:

../../../../home/me/Development/NullVoxPopuli/limber/frontend/node_modules/@glimmer/syntax/dist/modules/es2017/lib/parser.js

In a normal ember app, you don't have a real @glimmer/syntax package because that is packaged inside ember-source instead.

Did you add @glimmer/syntax to use it manually for some purpose? That would mean you have two of them, and the compatibility adjustments we need to make on the copy inside ember-source would break the unbundled copy.

@NullVoxPopuli
Copy link
Collaborator Author

NullVoxPopuli commented May 20, 2021

Yup, I added it for:

import { getTemplateLocals } from '@glimmer/syntax';

used for in-browser compliing strict mode stuff on https://limber.glimdown.com

how does one adjust builtin compat rules?

@ef4
Copy link
Contributor

ef4 commented May 20, 2021

It's hard to fix your case without breaking the built-in case.

One workaround you can try, since you have a monorepo anyway, is to import from @glimmer/syntax in a new package that is not an ember addon. By doing it in code that's "not ember code", you avoid some of these compatibility features. That new package can then reexport the things you need and the app can import them from the new package.

Another option is to find what you need from inside ember-source/dist/template-compiler.js. That is what embroider itself does to get access to @glimmer/syntax, and that is how mho gets access to the template compiler from within a service worker.

@NullVoxPopuli
Copy link
Collaborator Author

NullVoxPopuli commented May 20, 2021

awesome, thanks! I think I'll try the new package route, because I've been wanting to make the gjs compiler in here its own library anyway and publish that. I'll report back soon™!

@NullVoxPopuli
Copy link
Collaborator Author

This is related, but still interesting,
testing out just this small change

-import { getTemplateLocals } from '@glimmer/syntax';
-import { precompileTemplate } from '@ember/template-compilation';
+import { getTemplateLocals, precompileTemplate } from '@ember/template-compilation';

(and removing @glimmer/syntax from package.json)

results in the build succeeding, but, with a runtime error:
Uncaught Error: Could not find module `@ember/-internals/utils` imported from `(require)`
    missingModule loader.js:247
    findModule loader.js:258
    requireModule loader.js:24
    <anonymous> object.js:16
    js chunk.bd06e5b59763daf483e5.js:5928
    __webpack_require__ chunk.909881dca69b23cf60bd.js:31
    <anonymous> decorator.js:9
    js chunk.bd06e5b59763daf483e5.js:5917
    __webpack_require__ chunk.909881dca69b23cf60bd.js:31
    <anonymous> index.js:11
    js chunk.bd06e5b59763daf483e5.js:5939
    __webpack_require__ chunk.909881dca69b23cf60bd.js:31
    <anonymous> Ember
    js chunk.bd06e5b59763daf483e5.js:5862
    __webpack_require__ chunk.909881dca69b23cf60bd.js:31
    <anonymous> Ember
    js chunk.bd06e5b59763daf483e5.js:5873
    __webpack_require__ chunk.909881dca69b23cf60bd.js:31
    <anonymous> Ember
    js chunk.bd06e5b59763daf483e5.js:5895
    __webpack_require__ chunk.909881dca69b23cf60bd.js:31
    <anonymous> Ember
    js chunk.79c93d843abd4d8f4997.js:619
    __webpack_require__ chunk.909881dca69b23cf60bd.js:31
    <anonymous> limber.js:25

which comes from tracked-built-ins, so I'll follow up there
image

(probably more Ember global issues with 3.27 stuff)

@NullVoxPopuli
Copy link
Collaborator Author

🤔 I'm still having issues with module resolution it seems, even ember stuff:

for example, in tracked-built-ins, I'm getting an error that this utils module doesn't exist:
image

but, it's right here:
image

when debugging require,
image

I would expect that the @ember/* modules would appear in there -- yet they don't. 🤔
Gonna try an isolated repro

@NullVoxPopuli
Copy link
Collaborator Author

Just connecting the issues I'm finding together: tracked-tools/tracked-built-ins#96 (unexpected new issue 🤷 )

@NullVoxPopuli
Copy link
Collaborator Author

alright, with some git references in my package.json, and removing @glimmer/syntax and importing getTemplateLocals from the template compiler, I get this error:

Error: Cannot call `compile` with only the template compiler loaded. Please load `ember.debug.js` or `ember.prod.js` prior to calling `compile`.

which... I assume means that the template compiler just isn't compatible with ember 3.27 + embroider?

@ef4
Copy link
Contributor

ef4 commented May 28, 2021

I would expect that the @ember/* modules would appear in there -- yet they don't.

You really don't want them in there if you want true modules and tree shaking. We only put things in there that absolutely must be for compatibility. Under embroider ES module imports don't go through the runtime loader.

@NullVoxPopuli
Copy link
Collaborator Author

bug for the template compiler issue: emberjs/ember.js#19568

@NullVoxPopuli
Copy link
Collaborator Author

idk if I've created too many x/y problems by doing this, but, I think we can close this issue, as the recommendation is to either:

  • use the builtin export
  • or pre-build the thing using @glimmer/syntax

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants