Better monitoring / Improvements in nextjs server stdout (logs) output

[kristijorgji]

Describe the feature you'd like to request

Motivation:
Better nextjs logging (especially for unhandled exceptions) to understand exactly context of possible errors/warnings and be able to respond very fast to those situations.

Hi, right now after next build when we start next server with next start we see output logs only if error happens.

I would like to be able to have more control here over

1. log format. Example print json output instead of line by line example that i copy below

Error: {"size":0,"timeout":0}
at getStaticProps (/usr/src/app/.next/server/pages/post/[id].js:3576:11)
at runMicrotasks (<anonymous>)

2. Ability to add meta/context to nextjs logs after specifying a json format. Example enrich all logs with request url as well, or with custom tracing field whatever developer wants.
   In that case above log would look like {"message": "some message here", "url": "domain.com/someurl", "appVersion":"something", ...othermeta}
3. In above real log that I posted i see js file name, but my code is in typescript. How can we apply source maps so see ts file line in log output in those unhandled errors ?

PS: This is not about wrapping everything in a try catch and logging to custom logger. We are doing that sending to newrelic, but many things happen and are not handled. In those cases next server prints the error but without formatting context and is not very useful, therefore my suggestion to having a great way like in other frameworks for full visibility

Describe the solution you'd like

As described in feature request I want to be able to do those actions.
I think one way maybe can be configurations in next.config.js but as I don't know nextjs internals well I am open to any solution that is best fit

Describe alternatives you've considered

Unfortunately nothing because is impossible currently to add meta and format next output to json.

[kristijorgji]

Another example that is real life causing trouble for me to understand where and how error happens.
The below are logs straight from my web docker container.

I don't know where that error happens or in what context.
In previous php apps for example, I added to all logs MDC context requestId coming from frontend, also url and was easy to understand and fix issues.

web | TypeError: Cannot read property 'session' of undefined
web |     at /usr/src/app/.next/server/chunks/6606.js:1005:23
web |     at runMicrotasks (<anonymous>)
web |     at processTicksAndRejections (node:internal/process/task_queues:96:5)
web |     at async Object.apiResolver (/usr/src/app/node_modules/next/dist/server/api-utils/node.js:182:9)
web |     at async NextNodeServer.runApi (/usr/src/app/node_modules/next/dist/server/next-server.js:386:9)
web |     at async Object.fn (/usr/src/app/node_modules/next/dist/server/base-server.js:488:37)
web |     at async applyCheckTrue (/usr/src/app/node_modules/next/dist/server/router.js:110:32)
web |     at async Router.execute (/usr/src/app/node_modules/next/dist/server/router.js:250:25)
web |     at async NextNodeServer.run (/usr/src/app/node_modules/next/dist/server/base-server.js:600:29)
web |     at async NextNodeServer.handleRequest (/usr/src/app/node_modules/next/dist/server/base-server.js:30

That entire message could be json, and allow for adding extra meta like requestId: valueIsendFromWebHttpClient platform: web|android|etc url and other context then would be fantastic and proplery how a framework should work to allow proper monitoring

I am quite confident that you guys can implement this easily and hopefully this post gets visibility.

[regan-karlewicz]

I would also like this functionality. Enabling the developer to have fine-grained control over the applications output is important for monitoring integrations. All of my logs are output in JSON everywhere except for server-side errors in NextJS. This is disruptive when looking at logs for a request in most log explorers, since each line of the stack trace are on different lines.

I have been able to mitigate this by implementing my own HoF for getServerSideProps, but this doesn't work as well when handling rendering errors on the server side.

[kaufmann42]

I agree this seems to me something that would be needed for proper logging.

[regan-karlewicz]

I've found a workaround if you're using a custom server, albeit it's somewhat hacky and not guaranteed to work in future versions.

type ExposedNextServer = Omit<NextServer, 'getServer'> & {
  getServer: () => Promise<ExposedNextServer>;
  quiet?: boolean;
};

async function createNextServer() {
  const server = next({}) as unknown as ExposedNextServer;
  const innerServer = await server.getServer();

  if (!(innerServer.logError instanceof Function)) {
    throw Error('Expected NextServer.logError to be a function');
  }

  innerServer.logError = (err: Error) => {
    if (innerServer.quiet) return;
    logger.error(err, 'NextJSError');
  };

  return server;
}

All said and done, this logError implementation is the exact same as what exists in NextJS currently, with the logger method replaced

[tomasfejfar]

Where would you put this in the app? I read https://nextjs.org/docs/pages/guides/custom-server but that means no "convenience" of nextjs will work with it.

[regan-karlewicz]

If I recall correctly this would go in your custom server file (server.ts from the docs), and replace the line const app = next({ dev }) with const app = createNextServer().

This does come with a handful of trade offs like the documentation outlines, but for my case I was already using a custom server anyway. Sorry I can't offer much more guidance, it's been a while since I've used NextJS.

[tomasfejfar]

Thanks!

[tomasfejfar]

Our usecase is that we have Datadog agent listening on stderr for logs. If they are json, they are parsed, if they are plaintext, they get logged per-line. So now even if we log to JSON in the app, any uncaught exception still pass through and is logged as lines effectively preventing any meaningful correlation effort in datadog.