RangeError: Invalid String Length at JSON.stringify()

[jmikrut]

First off—thank you all very much for the work you do in this package. We're attempting to use it in Payload CMS in an effort to auto-generate types from our users' configs.

We've been working with a proof of concept, testing through a few different projects here and there.

Some Payload configs can be moderately large when converted to a JSON schema: 10-15K lines or so. We're seeing that when configs near that size, compiling JSON to TS with this package just crashes with the following error:

(node:18811) UnhandledPromiseRejectionWarning: RangeError: Invalid string length
    at JSON.stringify (<anonymous>)
    at stringify (/www/payload/node_modules/json-stringify-safe/stringify.js:5:15)
    at /www/payload/node_modules/json-schema-to-typescript/dist/src/optimizer.js:34:204

I believe this error comes from JS running out of memory. We have attempted turning off format as the docs instruct, but this did not help anything.

Here is an example of a decently large JSON schema that is causing the error above.

Looking at the stack trace, line 34 of optimizer.js looks to have to do with unions. In the example schema, we have a lot of unions, many generated from a single Payload "field" called link. You can see an example of the JSON schema for a link field on line 44 of the example above.

Is there a potential performance enhancement or bug here?

As it stands we can't compile the schema linked above.

Any ideas?

[jmikrut]

This might actually end up being a duplicate of #323

[jmikrut]

Update: I simply commented out the de-duplication measures that happens within the INTERSECTION and UNION optimization.ts and this has completely fixed my performance issues. Took me from ~120 seconds to compile down to 5.

Types look great—nice and succinct. In my case, I have control elsewhere over avoiding duplicates within intersections and unions. Could we create an option that allows for this package to skip that optimization?

[bcherny]

Sorry for the delay!

Thanks for the patch, but this is a bug that we should fix, and not something that consumers should be able to bail out of to avoid hitting the bug.

Root cause fix, which will be included in the next JSTT release: #447.

[bcherny]

Published as part of v11.0.0.

[jmikrut]

Beautiful @bcherny — thank you!