Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Next.js 13 Amplify - CustomerError: The size of the build output exceeds the max allowed size of 230686720 bytes #3863

Open
5 tasks done
jitendra-koodo opened this issue Dec 19, 2023 · 23 comments

Comments

@jitendra-koodo
Copy link

Before opening, please confirm:

  • I have checked to see if my question is addressed in the FAQ.
  • I have searched for duplicate or closed issues.
  • I have read the guide for submitting bug reports.
  • I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
  • I have removed any sensitive information from my code snippets and submission.

App Id

new_app

AWS Region

us-east-1

Amplify Hosting feature

Frontend builds

Frontend framework

Next.js

Next.js version

13

Next.js router

App Router

Describe the bug

#3647

Issue above has been abruptly closed without providing any details .. Creating this new ticket

200MB limit is still there : why can it not be removed?

Expected behavior

Build should not fail

Reproduction steps

Build
Front end build steps fails ;:

Build Settings

No response

Log output

# Put your logs below this line
2023-12-19T07:14:02.836Z [INFO]: # Caching completed
2023-12-19T07:14:19.932Z [ERROR]: !!! CustomerError: The size of the build output (236302771) exceeds the max allowed size of230686720 bytes. Please reduce the size ofyour build output and try again.

Additional information

No response

@jitendra-koodo
Copy link
Author

2023-12-19T07:14:02.836Z [INFO]: # Caching completed
2023-12-19T07:14:19.932Z [ERROR]: !!! CustomerError: The size of the build output (236302771) exceeds the max allowed size of230686720 bytes. Please reduce the size ofyour build output and try again.

@jitendra-koodo
Copy link
Author

#3647

Issue above has been abruptly closed without providing any details .. Creating this new ticket

200MB limit is still there : why can it not be removed?

@jitendra-koodo
Copy link
Author

Can anyone suggest how we can move out of this Amplify to some other service which does not have this limit? Any recommendation? Our production builds are failing and we need to find alternatives very quickly. Business loss. Thanks.

@mauerbac
Copy link
Member

mauerbac commented Dec 19, 2023

I'm sorry to hear you are experiencing this issue. We made changes last week that removed non-critical known to be large packages to help stay below the 200MB limit.

Can you look at your output and breakdown the different packages and let me know? Running du -csh .next/*should tell us that.

also, what is your Amplify App ID? It looks like a hash ID

@victorccccc
Copy link
Contributor

victorccccc commented Dec 19, 2023

Hi 👋 , sorry for the inconvenience. We added logic to remove unneeded dependency files that are typically large (e.g some unused heavy binaries under @esbuild and @swc packages).

To review your build output folder, you can:

  1. Run following command
aws amplify get-job --app-id ${APP_ID} --branch-name ${BRANCH_NAME} --job-id ${BUILD_ID}
  1. Find artifactsUrl (presigned S3 url) under job -> steps -> stepName: "BUILD"
  2. Download the file which is your build output.

The bundle size issue could happen if your app has heavy dependencies. You can check size of the dependency under the node_modules folder in the build output. You could try to use a lighter weight dependency or remove some of the files you wouldn't need.

If you need further assistance could you please provide your appId? Thanks

@jitendra-koodo
Copy link
Author

jitendra-koodo commented Dec 20, 2023

Thanks @victorccccc @mauerbac for quick revert..

App ID: dkc2twse2pm4r
Job url#/dkc2twse2pm4r/staging-in/1

==

2023-12-19T06:41:23.553Z [INFO]: # Executing command: du -sh *
2023-12-19T06:41:23.556Z [INFO]: 4.0K	amplify.sh
2023-12-19T06:41:23.556Z [INFO]: 4.0K	amplify.yml
                                 28K	apidata.json
                                 48K	blog-related-data.json
                                 0	__cache.tar
2023-12-19T06:41:23.556Z [INFO]: 548K	components
2023-12-19T06:41:23.556Z [INFO]: 4.0K	context
                                 4.0K	data.json
                                 0	eslintrc.js
                                 88K	fonts
                                 44K	helpers
                                 4.0K	jsconfig.json
                                 8.0K	lib
                                 8.0K	middleware.js
                                 36K	next.config.js
2023-12-19T06:41:23.653Z [INFO]: 424M	node_modules
                                 4.0K	package.json
2023-12-19T06:41:23.653Z [INFO]: 280K	pages
                                 24K	public
                                 4.0K	README.md
                                 12K	scripts
                                 4.0K	seo-hreflangs.json
                                 72K	services
                                 4.0K	sort.json
2023-12-19T06:41:23.654Z [INFO]: 456K	styles
                                 200K	yarn.lock
2023-12-19T06:41:23.654Z [INFO]: # Executing command: du -h -d1 public
2023-12-19T06:41:23.656Z [INFO]: 4.0K	public/.well-known
                                 24K	public
2023-12-19T06:41:23.656Z [INFO]: # Executing command: du -h -d1 .next/static
2023-12-19T06:41:23.657Z [INFO]: 2.0M	.next/static/chunks
2023-12-19T06:41:23.657Z [INFO]: 8.0K	.next/static/fFlvAR4iijcKnCOhx6OaR
                                 344K	.next/static/css
                                 2.3M	.next/static
2023-12-19T06:41:23.658Z [INFO]: # Executing command: du -h -d1 .next/server
2023-12-19T06:41:23.659Z [INFO]: 28M	.next/server/chunks
2023-12-19T06:41:23.662Z [INFO]: 166M	.next/server/pages
2023-12-19T06:41:23.662Z [INFO]: 194M	.next/server
2023-12-19T06:41:23.662Z [INFO]: # Executing command: du -h -d1 .next/cache
2023-12-19T06:41:23.663Z [INFO]: 72K	.next/cache/eslint
2023-12-19T06:41:23.664Z [INFO]: 0	.next/cache/swc
                                 280M	.next/cache/webpack
                                 280M	.next/cache
2023-12-19T06:41:23.664Z [INFO]: # Completed phase: build
2023-12-19T06:41:23.669Z [INFO]: ## Build completed successfully
2023-12-19T06:41:23.670Z [INFO]: # Starting caching...
2023-12-19T06:41:23.674Z [INFO]: # Creating cache artifact...
2023-12-19T06:41:39.686Z [INFO]: # Created cache artifact
2023-12-19T06:41:39.852Z [INFO]: # Uploading cache artifact...
2023-12-19T06:41:42.793Z [INFO]: # Uploaded cache artifact
2023-12-19T06:41:42.793Z [INFO]: # Caching completed
2023-12-19T06:41:59.624Z [ERROR]: !!! CustomerError: The size of the build output (236302652) exceeds the max allowed size of230686720 bytes. Please reduce the size ofyour build output and try again.
2023-12-19T06:41:59.731Z [INFO]: # Starting environment caching...

@jitendra-koodo
Copy link
Author

jitendra-koodo commented Dec 20, 2023

I tried to do some basic maths to find out upper limit on number of pages which could be released on my website with this 200MB limit.

I roughly measured storage when I had 70 pages (case 1) and 700 pages (case2) (each page similar size in KB) . In case 1, total size was ~95MB. In case 2, total size was ~175MB. So if there is basic overhead of size A and per page need size B, then

A + 70B = 95
A + 700B = 175

B = ~120KB (it matches with the size visible on Chrome)
max 800 pages possible with this limit.

This means, no matter what optimization we do, at some point when page count go up, it'll break 200MB barrier even when all other things are optimized.

Our page count will double in next one week, What's the solution here?

@gzManuel
Copy link

There is another duplicated issue, @jitendra-koodo I found two solutions

  1. Upgrade to Nextjs 14 and update the build image of amplify to Amazon Linux:2023
  2. Delete the deployed branch of amplify, remove package-lock.json and generate it again, update the build image of amplify to Amazon Linux:2023 and deploy the branch again.

@bivainis
Copy link

I've encountered this issue today and here's what made it work for me:

I noticed that the failing frontend build logs had the following lines as compared to non-failing ones:

[WARNING]: ⚠ Found lockfile missing swc dependencies, run next locally to automatically patch
[INFO]: Downloading swc package @next/swc-linux-x64-gnu...
[INFO]: Downloading swc package @next/swc-linux-x64-musl...

A few days prior, someone on the team used a different nodejs version which rewrote entire package-lock.json in version 2 format instead of version 3.
To mitigate that and to help out, I deleted my package-lock.json and regenerated it via npm install
I then committed that version to the pull request of that person.
I noticed that after re-creating the lock file, these packages were removed (based on git diff):

  • @next/swc-darwin-x64
  • @next/swc-linux-arm64-gnu
  • @next/swc-linux-arm64-musl
  • @next/swc-linux-x64-gnu
  • @next/swc-linux-x64-musl
  • @next/swc-win32-arm64-msvc
  • @next/swc-win32-ia32-msvc
  • @next/swc-win32-x64-msvc

The only swc package that was still remaining in package-lock.json was @next/swc-darwin-arm64, which is logical since I'm on a mac.

I pasted back the removed packages from the diff in to package-lock.json, ran npm ci locally to make sure I install exact versions from the lock file.

Committed this file, pushed to git, and the build went through. This time no swc lock file warning and no out of space warnings as well.

Now the question is, how did I have those swc packages initially? Maybe they were installed during create next app. But if so, that means it's unreliable to re-create package-lock.json, because it will not include swc for other architectures (unless there's a better way to do this?)

Hope this helps someone.

@ggorge-etiqa
Copy link

I'm facing the same error. After digging into node_modules and .next folders sizes I found out that adding the lines below to the amplify.yml build step fixes the error:

        - rm -rf .next/standalone/node_modules/next/next-swc-fallback
        - rm -rf node_modules/next/next-swc-fallback

Seems that Amplify is already trying to do the same under the hood but probably it's missing some file.

@ggorge-etiqa
Copy link

ggorge-etiqa commented Jan 16, 2024

Another little update is that rm -rf node_modules/next/next-swc-fallback is only really needed.

I'm wondering why amplify is bothering on node_modules directory size when the artifacts folder is set to .next:

  artifacts:
    baseDirectory: .next
    files:
      - "**/*"

@jitendra-koodo
Copy link
Author

Point is that even with all these small optimizations, at some point you are going to hit the 200MB limit and at that point you have to move out of Amplify

@ggorge-etiqa
Copy link

@jitendra-koodo I know this is frustrating, but every solution has it's own limits.
Docs report this build output limit here: https://docs.aws.amazon.com/amplify/latest/userguide/troubleshooting-ssr-deployment.html#build-output-too-large but I think it should be also mentioned there: https://docs.aws.amazon.com/amplify/latest/userguide/quotas-chapter.html

@jitendra-koodo
Copy link
Author

@ggorge-etiqa Not sure what you mean.. which other solution has 200MB limit? moreover this limit is not shown anywhere on console at any point of time. People go to that doc only after hitting this limit.

@dannypaz
Copy link

dannypaz commented Feb 4, 2024

Rebump. I ran into this bug with Nextjs 14 and a very basic starter app. After using #3863 (comment) I was able to resolve

@andre-mr
Copy link

andre-mr commented Feb 16, 2024

Another little update is that rm -rf node_modules/next/next-swc-fallback is only really needed.

I'm wondering why amplify is bothering on node_modules directory size when the artifacts folder is set to .next:

  artifacts:
    baseDirectory: .next
    files:
      - "**/*"

where did you added those lines? at the end of build phase? i tried only this line and got an error related to swc at the end of "cloning git" step.

don't know if it was a coincidence and a temp error, but in a desperate way i changed my build file to this:

version: 1
frontend:
  phases:
    preBuild:
      commands:
        - rm -rf .next
        - rm -rf node_modules
        - npm i
    build:
      commands:
        - npm run build
        - rm -rf .next/standalone/node_modules/next/next-swc-fallback
        - rm -rf node_modules/next/next-swc-fallback
  artifacts:
    baseDirectory: .next
    files:
      - '**/*'
  cache:
    paths:
      - node_modules/**/*

and it worked, lol. in the next update i'll try to change this, today we are very hurry here.

by the way: my local .next folder, after cleaned up and made from next build is 199mb size. if bundle limit is 200 it's at the limit anyway, but i couldn't find anything unused to remove from dependencies or imports.
AND i don't know if amplify is counting node_modules (another huge folder) in this 200mb size limit (if it's counting, it would never pass because i would have almost 400mb size .next+node_modules).

@turbolego
Copy link

We have the same issue.
Next.js version 14.1.0 with React 18.

@alexHewittProcter
Copy link

alexHewittProcter commented Mar 29, 2024

I'm also experiencing this issue even after upgrading to Next 14!
I've tried several different attempts including changing the base image I use, removing specific dependencies and attempting to remove the cache that exists between builds

Is there any update from AWS on this? I've wasted several hours trying to fix this problem and I have to question why we're being limited to 200mb for our artefacts when the service quotas allow for 5GB? https://docs.aws.amazon.com/amplify/latest/userguide/quotas-chapter.html

@nicolasleguizamon
Copy link

Hello,
We have same issue!
Next: 13.2.4

@ekpono
Copy link

ekpono commented Apr 23, 2024

I am having the same issue. This is annoying

Next: 14.0.4

[ERROR]: !!! CustomerError: The size of the build output (306216844) exceeds the max allowed size of230686720 bytes. Please reduce the size ofyour build output and try again.

@guy-a
Copy link

guy-a commented May 5, 2024

Adding rm -rf node_modules/canvas to the build (amplify.yml) worked for me.

i.e.

version: 1
frontend:
  phases:
    preBuild:
      commands:
        - npm ci
    build:
      commands:
        - npm run build
        - rm -rf node_modules/canvas

@ekpono
Copy link

ekpono commented May 6, 2024

Adding rm -rf node_modules/canvas to the build (amplify.yml) worked for me.

i.e.

version: 1
frontend:
  phases:
    preBuild:
      commands:
        - npm ci
    build:
      commands:
        - npm run build
        - rm -rf node_modules/canvas

I did the same thing and it worked, I'm guessing having jest pulls in canvas which is about 100MB+

@pachoclo
Copy link

pachoclo commented Jun 6, 2024

Running into this issue for a second time. Can we please get a clear reason why the 200mb limit is still being enforced? We really don't want to have to switch to a different hosting service...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests