Build time decreased x5 when migrating from docosaurus 2.4 to 3.9

[martindekov]

Hey Docosaurus team in Harvester we try to move to latest 3.9 from 2.4, but our original build times on 2.4 from ~11 minutes in total for pipeline and ~5-7 minutes for the build itself:

https://github.com/harvester/docs/actions/runs/21025102637/job/60447639391 (~11 minutes)

Where we drew inspiration from this codebase to do the optimization:

docusaurus/website/docusaurus.config.ts

Lines 182 to 203 in 1f845e6

	webpack: {
	jsLoader: (isServer) => ({
	loader: require.resolve('swc-loader'),
	options: {
	jsc: {
	parser: {
	syntax: 'typescript',
	tsx: true,
	},
	transform: {
	react: {
	runtime: 'automatic',
	},
	},
	target: 'es2017',
	},
	module: {
	type: isServer ? 'commonjs' : 'es6',
	},
	},
	}),
	},

and that's how we made our builds that fast. But now following the documentation setting simply the experimental faster and v4 to true yields a lot slower results for the same step here is a build adopting this:

https://github.com/harvester/docs/actions/runs/20988131354/job/60326759049 (~50 minutes)

TL:DR we went from this in 2.4:

    webpack: {
      jsLoader: (isServer) => ({
        loader: require.resolve('swc-loader'),
        options: {
          jsc: {
            parser: {
              syntax: 'typescript',
              tsx: true,
            },
            transform: {
              react: {
                runtime: 'automatic',
              },
            },
            target: 'es2017',
          },
          module: {
            type: isServer ? 'commonjs' : 'es6',
          },
        },
      }),
    },

to this in 3.9:

  future: {
    v4: true,
    experimental_faster: true,
  },

Also went over this discussion: #11140 and attempted adopting the suggestions:

    ...
    function disableExpensiveBundlerOptimizationPlugin() {
      return {
        name: 'disable-expensive-bundler-optimizations',
        configureWebpack(_config, isServer) {
          return {
            optimization: {
              concatenateModules: false,
            },
          };
        },
      };
    },
    ...
        docs: {
          showLastUpdateTime: showLastUpdate,
          showLastUpdateAuthor: showLastUpdate,
    ...

also tried to fine tune:

future: {
  v4: true,
  experimental_faster: {
    swcJsLoader: true,
    swcJsMinimizer: true,
    swcHtmlMinimizer: true,
    lightningCssMinimizer: true,
    mdxCrossCompilerCache: true,
    rspackBundler: true,
    rspackPersistentCache: true,
    ssgWorkerThreads: true,
  },
},

So nothing did better than ~30 minutes of build time. The original code which is in the PR is the best build time I could achieve, but never around the current ~7-8 minutes. Is there a way to achieve that sub 10 minute speed in 3.9? Thank you in advance for the help!

[slorber]

Hey

I understand that your build time increased x5 (and did not decrease)

It would be helpful to link to the original PR: harvester/docs#945

With Docusaurus Faster, it's supposed to be... faster. And you shouldn't use a custom SWC loader because it uses an built-in Rspack SWC loader by default, which would be faster than using a js-based swc loader like you did before.

---

Have you tried running your build locally? Is it as slow as on CI runs?

Are you comparing cold or warm rebuilds?

• Cold build: docusaurus clear before building (it clears node_modules/.cache among other things)
• Warn rebuilds: it will be faster if you run docusaurus build a second time

Before trying to optimize on your CI, it would be easier to optimize locally. Can you please share the time it takes to build locally?

local build time	Cold build	Warm rebuild
v2.4
v3.9

You can get additional logs to figure out what is slow thanks to this env variable:

export DOCUSAURUS_PERF_LOGGER=true

Please share your full logs so that we identify which step is slow, for both a cold build and a warm rebuild:

export DOCUSAURUS_PERF_LOGGER=true
yarn docusaurus clear
yarn docusaurus build
yarn docusaurus build

[sserrata]

Hi @slorber, between v2 and v3 we made significant changes to the MDX output, to reduce its footprint, by migrating from inline MDX to theme components. However, as you've pointed out, some of these theme components accept JSON props which can get pretty large.

What prevents you from importing a json file instead?

Nothing really I suppose, although I would prefer not to write these json files alongside the generated MDX files. Would it be possible to store and load/reference them from the .docusaurus directory?

[slorber]

I have noticed that the size reduced yeah 👍 but it may make MDX compilation slower.

---

I don't know when content is generated in your plugin, but we have the actions.createData() API that could work for that: https://docusaurus.io/docs/api/plugin-methods/lifecycle-apis#createData

The data is namespaced under your .docusaurus/plugin-name/plugin-id, so normally you only need to generate unique json filenames within your plugin.

Technically, it's possible for Webpack/Rspack to have "virtual modules" that stay in memory (https://rspack.uihtm.com/api/javascript-api/#custom-file-systems). It would be more efficient, but I'm not sure it would be easy to use as a plugin author.

[sserrata]

Thanks @slorber - a quick experiment confirmed your hypothesis:

Added a new externalJsonProps: boolean option that:

1. Extracts large JSON props (responses, body, parameters, schema) from generated MDX
2. Writes them to separate .json files alongside the MDX
3. Replaces inline JSON with require("./filename.json")

Results with Harvester Spec

Tested with the Harvester OpenAPI spec (hopefully this is right one or similar enough):

Metric	Without	With	Improvement
Build Time	441s (7m 21s)	17s	25x faster
Largest MDX	506KB	59KB	88% smaller
Total MDX	7.6MB	1.4MB	82% smaller

Before/After

// Before: Inline JSON (MDX must parse to AST)
<StatusCodes responses={{"200":{"description":"OK","content":{...huge nested JSON...}}}} />

// After: External file (bypasses MDX parsing)
<StatusCodes responses={require("./endpoint.responses.json")} />

Given the dramatic speedup I'm going to move forward on officially supporting externalJsonProps. It does introduce a bit more "noise" to the filesystem but I think the performance boost more than justifies it:

[sserrata]

Ok, I opened PaloAltoNetworks/docusaurus-openapi-docs#1279 - appreciate any assistance with review or even just feedback on the approach. Given the possible performance gains, I opted to make externalJsonProps enabled by default, with the option to disable on a per-API basis.

[slorber]

Awesome 🥳 Thanks for giving this a try, the community will largely benefit from the speedup

I'll check the PR next week

[martindekov]

Wow this truly optimized the build time to ~30 seconds 😮

cold_build_v3.log

@sserrata I saw you added the commit after 4.6.0 release so I did some hacks to run my local builds with it as it's commit after release. When can we expect the release of this?

Also thank you very much for working on this all, really astonishing results, even way faster than the original!

[sserrata]

Hi @martindekov, glad to hear it improved your build perf. Heads up that we also publish canary releases with each merge to main so you don't have to wait for an official release.

[martindekov]

Nice, can't wait for this to get released so we can adopt it! I will watch the project pipeline or releases page:
https://github.com/PaloAltoNetworks/docusaurus-openapi-docs/releases

Or if there is already published canary release which is not found on those places let me know to look there. Truly amazing work and quality+speed.