Is there a way to always restore and cache some resources?

[jcarlson]

Summary

I am trying to add Turbopack to a Hugo project. The project has a large number of images that Hugo processes and resizes at build time, which is an expensive operation. Hugo already manages a cache directory for this operation at ./resources/_gen.

I'd like for Turbocache to always save this directory as part of the cache bundle so that it can be shared with Vercel's remote caching abilities.

Currently, every preview or production deployment has to rebuild all the images from scratch because the site content (HTML templates, Markdown, etc...) has changed, which creates a cache miss.

How can I configure Turbo to always save the contents of this resources directory and just assume that it is already a cacheable output?

Additional information

No response

Example

No response

[jcarlson]

In other words, the behavior I want to see is as follows:

1. restore the ./resources/_gen directory from cache, if available
2. build the site, which may update the ./resources/_gen directory if new images are processed
3. update the saved cache of ./resources/_gen if it changed

[anthonyshew]

It sounds like you're describing the default behavior of Turborepo. The only thing you should need is to include the directory in the outputs key of the task.

Is there nuance to this question that I'm missing?

[jcarlson]

Suppose my Hugo site includes about 1,000 images in the /assets/images directory. Those assets are the source images. At build time, Hugo generates optimized versions of those images based on what the template HTML calls for. Image processing can include resizing, cropping, format conversion, etc...

Hugo maintains a cache of the results of these expensive image transformations at /resources/_gen. But what input would I use as a cache key for that directory? The thing that would cause the cacheable /resources/_gen directory to change would be changes to my Hugo templates (/layouts/**) or site content (/content/**), not the source images in /assets/images.

If I make a small tweak to the copy in a Markdown file, that hardly seems like a valid reason to regenerate 1,000+ images because of a cache miss.

Put another way, an image can exist in /assets/images/... but if neither my content nor my templates actually use it, then it doesn't appear in the generated site output.

I think the issue here is that I'm asking Turbo to cache a cache. It is useful with a Hugo site like this to restore any cached copy of /resources/_gen before running hugo build, even if the cached resources are out of date or incomplete. Hugo will update that cache as part of its build.

So to cache Hugo build operations most effectively, I'd want to always restore any partial cache of /resources/_gen, let Hugo build the site, and update the cached copy of /resources/_gen after the build, because the build itself may have changed it.

Let me describe one last example to hopefully illustrate the issue.

If I set Turbo to cache resources like so:

{
  "tasks": {
    "build": {
      "inputs": ["assets/images/**"],
      "outputs": ["resources/**"]
    }
  }
}

Then I will always get a cache hit, so long as the set of source images doesn't change. And it rarely will. But what happens if my HTML templates now call for three image sizes to be generated, instead of two? My content and layouts change frequently, making a cache key on /content/** or /layouts/** quote volatile, but the content and templates rarely change in a way that would require all image processing to start over from scratch.

TL;DR: Caching the generated images in /resources/_gen using /content/** and /layouts/** as input keys results in exceedingly low cache hit rates for the expensive operation of processing images.