Next.js build fails when static file size is over 50MB

[collegewap]

I have a project which has mostly static pages and few server-less APIs. The total size of the static contents (mostly images) are more than 50MB. But, the server-less functions do not refer to these 50MB image files. According to the documentation, the 50MB limit enforced by AWS should not apply here (since these images get uploaded to CDN and not AWS). So the build should succeed.

However the build fails while using Next 12 and works perfectly fine while using Next 11. Here is the minimal reproducible sample:

Next.js version	Has API/SSR	Fails	Should Fail?	Repo
12	Yes	Yes	No	https://github.com/collegewap/next-50mb-issue
11	Yes	No	No	https://github.com/collegewap/next-50mb-issue/tree/next-11
12	No	No	No	https://github.com/collegewap/next-50mb-issue/tree/without-api

Error screenshot:

My suspect is something broke between Next 11 and Next 12 on the size calculation.

Note: This is NOT a duplicate of #103 and creating this new discussion as per the comment.

[Pieparker]

If you use next/image, on V12 at least, it'll bundle these in to a Lambda (see here). Haven't experimented with V11 myself!

How does the V12 reproduction look without next/image?

[collegewap]

It still fails as images are stored under public folder. But, Why would it bundle the image cache to Lambda? Shouldn't the image caching happen and CDN level?

[collegewap]

And, why does example with static only files do not fail if files needs to be uploaded to Lambda? The only difference between https://github.com/collegewap/next-50mb-issue and https://github.com/collegewap/next-50mb-issue/tree/without-api is one dummy API, which doesn't even refer to any of these 50MB images.

[Pieparker]

Sincerely appreciate the repros! I'll forward this to the applicable team, in order to dig in and better understand the scenario. We'll follow up with any questions or findings!

ref https://github.com/vercel/customer-issues/issues/238 (internal only)

[Pieparker]

Hey there! Thanks for your patience. V12, by default, features output file tracing and here's the docs. Looks like we're rolling up all of the dependencies in to the bundle.

Unrelated to the images!

Any chance that you can re-try this on the latest Next.js version? It's also worth assessing whether you need esbuild, sharp, jimp etc, given their size (and that they're not used!).

[Pieparker]

Continued in a standalone reply, #254 (comment)

[Pieparker]

Hey there,

Our team believe that this is due to how we trace dependencies of a function when we build the app in our runtime: accessing the entire "/posts" folder while building this route, so our runtime included all the "/posts" folder as its dependency, which is pretty large:

https://github.com/collegewap/next-50mb-issue/blob/304a267d76e77752b19d3c9b092c1f6e720d8625/pages/posts/%5Bslug%5D.tsx#L159

The reason is, we don't know which file it will access in advance, so everything is included:

https://github.com/collegewap/next-50mb-issue/blob/304a267d76e77752b19d3c9b092c1f6e720d8625/pages/posts/%5Bslug%5D.tsx#L70-L72

Our suggestion would be to move this step to a webpack plugin, or make it a pre-build script:

https://github.com/collegewap/next-50mb-issue/blob/304a267d76e77752b19d3c9b092c1f6e720d8625/utils/mdxUtils.tsx#L18-L30

We'll continue to explore how we can better document and mitigate this!

[collegewap]

Thanks for investigating and the update! Will explore the solutions and get back if I have any queries.