Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

lowdb integration issues with typescript, next and jest #591

Open
ncoquelet opened this issue Jun 4, 2024 · 5 comments
Open

lowdb integration issues with typescript, next and jest #591

ncoquelet opened this issue Jun 4, 2024 · 5 comments

Comments

@ncoquelet
Copy link

ncoquelet commented Jun 4, 2024 •
edited
Loading

Hi,

I've been stuck for two days trying anything to add lowdb to my project, and this issue is my last chance.
This is a full ESM project using full Typescript and based on react, nextjs and jest.

Node v20.14.0
npm 10.7.0
Typescript 5.3.3
Next 14.1.0
Jest 29.7.0
Lowdb 0.2.0

I've created a repo that reproduce the problem by simplified the dependencies and customisation to avoid side effects.
https://github.com/ncoquelet/lowdb_esm_issue

Installation

> git clone https://github.com/ncoquelet/lowdb_esm_issue
> cd lowdb_esm_issue
> npm install

Problems

Without lowdb everything works fine, but when I instantiate any lowdb class, I get these errors, both in next build and in the jest execution.

During Next build

> npm run build

  node:fs/promises
  Module build failed: UnhandledSchemeError: Reading from "node:fs/promises" is not handled by plugins (Unhandled scheme).
  Webpack supports "data:" and "file:" URIs by default.
  You may need an additional plugin to handle "node:" URIs.
full log 
[email protected] build
next build

▲ Next.js 14.2.3

Creating an optimized production build ...

Failed to compile.

node:fs/promises
Module build failed: UnhandledSchemeError: Reading from "node:fs/promises" is not handled by plugins (Unhandled scheme).
Webpack supports "data:" and "file:" URIs by default.
You may need an additional plugin to handle "node:" URIs.
at /home/john/dev/projects/lowdb_esm_issue/node_modules/next/dist/compiled/webpack/bundle5.js:28:401757
at Hook.eval [as callAsync] (eval at create (/home/john/dev/projects/lowdb_esm_issue/node_modules/next/dist/compiled/webpack/bundle5.js:13:28858), <anonymous>:6:1)
at Object.processResource (/home/john/dev/projects/lowdb_esm_issue/node_modules/next/dist/compiled/webpack/bundle5.js:28:401682)
at processResource (/home/john/dev/projects/lowdb_esm_issue/node_modules/next/dist/compiled/loader-runner/LoaderRunner.js:1:5308)
at iteratePitchingLoaders (/home/john/dev/projects/lowdb_esm_issue/node_modules/next/dist/compiled/loader-runner/LoaderRunner.js:1:4667)
at runLoaders (/home/john/dev/projects/lowdb_esm_issue/node_modules/next/dist/compiled/loader-runner/LoaderRunner.js:1:8590)
at NormalModule._doBuild (/home/john/dev/projects/lowdb_esm_issue/node_modules/next/dist/compiled/webpack/bundle5.js:28:401544)
at NormalModule.build (/home/john/dev/projects/lowdb_esm_issue/node_modules/next/dist/compiled/webpack/bundle5.js:28:403572)
at /home/john/dev/projects/lowdb_esm_issue/node_modules/next/dist/compiled/webpack/bundle5.js:28:82055
at NormalModule.needBuild (/home/john/dev/projects/lowdb_esm_issue/node_modules/next/dist/compiled/webpack/bundle5.js:28:407670)

Import trace for requested module:
node:fs/promises
./node_modules/lowdb/lib/adapters/node/TextFile.js
./node_modules/lowdb/lib/node.js
./src/app/Repository.ts
./src/app/page.tsx

node:fs
Module build failed: UnhandledSchemeError: Reading from "node:fs" is not handled by plugins (Unhandled scheme).
Webpack supports "data:" and "file:" URIs by default.
You may need an additional plugin to handle "node:" URIs.
at /home/john/dev/projects/lowdb_esm_issue/node_modules/next/dist/compiled/webpack/bundle5.js:28:401757
at Hook.eval [as callAsync] (eval at create (/home/john/dev/projects/lowdb_esm_issue/node_modules/next/dist/compiled/webpack/bundle5.js:13:28858), <anonymous>:6:1)
at Hook.CALL_ASYNC_DELEGATE [as _callAsync] (/home/john/dev/projects/lowdb_esm_issue/node_modules/next/dist/compiled/webpack/bundle5.js:13:26012)
at Object.processResource (/home/john/dev/projects/lowdb_esm_issue/node_modules/next/dist/compiled/webpack/bundle5.js:28:401682)
at processResource (/home/john/dev/projects/lowdb_esm_issue/node_modules/next/dist/compiled/loader-runner/LoaderRunner.js:1:5308)
at iteratePitchingLoaders (/home/john/dev/projects/lowdb_esm_issue/node_modules/next/dist/compiled/loader-runner/LoaderRunner.js:1:4667)
at runLoaders (/home/john/dev/projects/lowdb_esm_issue/node_modules/next/dist/compiled/loader-runner/LoaderRunner.js:1:8590)
at NormalModule._doBuild (/home/john/dev/projects/lowdb_esm_issue/node_modules/next/dist/compiled/webpack/bundle5.js:28:401544)
at NormalModule.build (/home/john/dev/projects/lowdb_esm_issue/node_modules/next/dist/compiled/webpack/bundle5.js:28:403572)
at /home/john/dev/projects/lowdb_esm_issue/node_modules/next/dist/compiled/webpack/bundle5.js:28:82055

Import trace for requested module:
node:fs
./node_modules/lowdb/lib/adapters/node/TextFile.js
./node_modules/lowdb/lib/node.js
./src/app/Repository.ts
./src/app/page.tsx

node:path
Module build failed: UnhandledSchemeError: Reading from "node:path" is not handled by plugins (Unhandled scheme).
Webpack supports "data:" and "file:" URIs by default.
You may need an additional plugin to handle "node:" URIs.
at /home/john/dev/projects/lowdb_esm_issue/node_modules/next/dist/compiled/webpack/bundle5.js:28:401757
at Hook.eval [as callAsync] (eval at create (/home/john/dev/projects/lowdb_esm_issue/node_modules/next/dist/compiled/webpack/bundle5.js:13:28858), <anonymous>:6:1)
at Object.processResource (/home/john/dev/projects/lowdb_esm_issue/node_modules/next/dist/compiled/webpack/bundle5.js:28:401682)
at processResource (/home/john/dev/projects/lowdb_esm_issue/node_modules/next/dist/compiled/loader-runner/LoaderRunner.js:1:5308)
at iteratePitchingLoaders (/home/john/dev/projects/lowdb_esm_issue/node_modules/next/dist/compiled/loader-runner/LoaderRunner.js:1:4667)
at runLoaders (/home/john/dev/projects/lowdb_esm_issue/node_modules/next/dist/compiled/loader-runner/LoaderRunner.js:1:8590)
at NormalModule._doBuild (/home/john/dev/projects/lowdb_esm_issue/node_modules/next/dist/compiled/webpack/bundle5.js:28:401544)
at NormalModule.build (/home/john/dev/projects/lowdb_esm_issue/node_modules/next/dist/compiled/webpack/bundle5.js:28:403572)
at /home/john/dev/projects/lowdb_esm_issue/node_modules/next/dist/compiled/webpack/bundle5.js:28:82055
at NormalModule.needBuild (/home/john/dev/projects/lowdb_esm_issue/node_modules/next/dist/compiled/webpack/bundle5.js:28:407670)

Import trace for requested module:
node:path
./node_modules/lowdb/lib/adapters/node/TextFile.js
./node_modules/lowdb/lib/node.js
./src/app/Repository.ts
./src/app/page.tsx

node:url
Module build failed: UnhandledSchemeError: Reading from "node:url" is not handled by plugins (Unhandled scheme).
Webpack supports "data:" and "file:" URIs by default.
You may need an additional plugin to handle "node:" URIs.
at /home/john/dev/projects/lowdb_esm_issue/node_modules/next/dist/compiled/webpack/bundle5.js:28:401757
at Hook.eval [as callAsync] (eval at create (/home/john/dev/projects/lowdb_esm_issue/node_modules/next/dist/compiled/webpack/bundle5.js:13:28858), <anonymous>:6:1)
at Object.processResource (/home/john/dev/projects/lowdb_esm_issue/node_modules/next/dist/compiled/webpack/bundle5.js:28:401682)
at processResource (/home/john/dev/projects/lowdb_esm_issue/node_modules/next/dist/compiled/loader-runner/LoaderRunner.js:1:5308)
at iteratePitchingLoaders (/home/john/dev/projects/lowdb_esm_issue/node_modules/next/dist/compiled/loader-runner/LoaderRunner.js:1:4667)
at runLoaders (/home/john/dev/projects/lowdb_esm_issue/node_modules/next/dist/compiled/loader-runner/LoaderRunner.js:1:8590)
at NormalModule._doBuild (/home/john/dev/projects/lowdb_esm_issue/node_modules/next/dist/compiled/webpack/bundle5.js:28:401544)
at NormalModule.build (/home/john/dev/projects/lowdb_esm_issue/node_modules/next/dist/compiled/webpack/bundle5.js:28:403572)
at /home/john/dev/projects/lowdb_esm_issue/node_modules/next/dist/compiled/webpack/bundle5.js:28:82055
at NormalModule.needBuild (/home/john/dev/projects/lowdb_esm_issue/node_modules/next/dist/compiled/webpack/bundle5.js:28:407670)

Import trace for requested module:
node:url
./node_modules/steno/lib/index.js
./node_modules/lowdb/lib/adapters/node/TextFile.js
./node_modules/lowdb/lib/node.js
./src/app/Repository.ts
./src/app/page.tsx


> Build failed because of webpack errors

Process finished with exit code 1

Jest Execution

> npm run test
  /home/john/dev/projects/lowdb_esm_issue/node_modules/lowdb/lib/node.js:1
  ({"Object.<anonymous>":function(module,exports,require,__dirname,__filename,jest){export * from './adapters/node/DataFile.js';
^^^^^^
  SyntaxError: Unexpected token 'export'
full log 
[email protected] test
jest

FAIL  src/app/Repository.test.ts
● Test suite failed to run

    Jest encountered an unexpected token

    Jest failed to parse a file. This happens e.g. when your code or its dependencies use non-standard JavaScript syntax, or when Jest is not configured to support such syntax.

    Out of the box Jest supports Babel, which will be used to transform your files into valid JS based on your Babel configuration.

    By default "node_modules" folder is ignored by transformers.

    Here's what you can do:
     • If you are trying to use ECMAScript Modules, see https://jestjs.io/docs/ecmascript-modules for how to enable it.
     • If you are trying to use TypeScript, see https://jestjs.io/docs/getting-started#using-typescript
     • To have some of your "node_modules" files transformed, you can specify a custom "transformIgnorePatterns" in your config.
     • If you need a custom transformation specify a "transform" option in your config.
     • If you simply want to mock your non-JS modules (e.g. binary assets) you can stub them out with the "moduleNameMapper" config option.

    You'll find more details and examples of these config options in the docs:
    https://jestjs.io/docs/configuration
    For information about custom transformations, see:
    https://jestjs.io/docs/code-transformation

    Details:

    /home/john/dev/projects/lowdb_esm_issue/node_modules/lowdb/lib/node.js:1
    ({"Object.<anonymous>":function(module,exports,require,__dirname,__filename,jest){export * from './adapters/node/DataFile.js';
                                                                                      ^^^^^^

    SyntaxError: Unexpected token 'export'

    > 1 | import { JSONFileSyncPreset } from "lowdb/node";
        | ^
      2 |
      3 | type Data = {
      4 |   messages: string[]

      at Runtime.createScriptFromCode (node_modules/jest-runtime/build/index.js:1505:14)
      at Object.<anonymous> (src/app/Repository.ts:1:1)
      at Object.<anonymous> (src/app/Repository.test.ts:1:1)

Test Suites: 1 failed, 1 total
Tests:       0 total
Snapshots:   0 total
Time:        1.161 s
Ran all test suites.

Process finished with exit code 1
@vikinatora
Copy link

vikinatora commented Jun 5, 2024 •
edited
Loading

You should probably use the browser preset from here: https://github.com/typicode/lowdb/blob/main/src/examples/browser.ts

Also, https://github.com/ncoquelet/lowdb_esm_issue is private

@ncoquelet
Copy link
Author

I have try the browser preset but globaly I obtains the same issues.
This is a problem with node and esm, my repo is now public to quickly reproduce my state.
Thanks for your support

@MisterrrX
Copy link

Got the same issue using Typescript, Webpack and node with ESM. Have you found any fix for this @ncoquelet?

@ncoquelet
Copy link
Author

ncoquelet commented Jul 22, 2024 •
edited
Loading

Got the same issue using Typescript, Webpack and node with ESM. Have you found any fix for this @ncoquelet?

Nop, I'm using something else ...

shaman-apprentice added a commit to shaman-apprentice/SurnameForge that referenced this issue Sep 22, 2024
@shaman-apprentice
Add PoC for shared lib
1e20540 
Surprisingly, TypeScript project references are not supported, so treat it as local folder package (see typicode/lowdb#591)
@shaman-apprentice
Copy link

My workaround for NestJS - maybe it is useful for someone else as well:

// use dynamic imports in combination with TS `moduleResolution: Node16` in ts config
const { JSONFilePreset } = await import("lowdb/node");

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@ncoquelet @shaman-apprentice @vikinatora @MisterrrX