Precaching bug with `adapter-static` and `load()` functions #98

WhyAsh5114 · 2024-12-31T10:03:55Z

The main problem

There's a specific bug when using load() functions along with prerendering and the static-adapter that I spent
hours trying to fix, it occurs primarily due to @vite-pwa/sveltekit not being able to figure out how to cache
the returned data from the load(), typically stored in:

.svelte-kit/output/prerendered/dependencies/{page-route}/__data.json

This causes the app to break when offline and navigating to a page that depends on that data.

The hacky fix

To fix this, you need to manipulate the PWA config in vite.config.ts and the custom service-worker file.
After setting up the basic PWA, I added the following to your vite.config.ts:

export default defineConfig({
  plugins: [
    sveltekit(),
    SvelteKitPWA({
      /* other config */
      injectManifest: {
        globPatterns: [
          'client/**/*.{js,css,ico,png,svg,webp,webmanifest,ttf,woff,woff2}',
          'prerendered/**/*.{html,json}'
        ],
      }
    })
  ],
});

Notice the prerendered/**/*.{html,json} in the globPatterns array, this tells vite-plugin-pwa to cache the
prerendered data (typically in the form of __data.json files) so that the app can work offline. This should have
been enough, but for some reason you need this weird hack in your service-worker.ts:

/// <reference lib="WebWorker" />
/// <reference types="@sveltejs/kit" />

import { cleanupOutdatedCaches, precacheAndRoute } from 'workbox-precaching';
import { registerRoute } from 'workbox-routing';
import { StaleWhileRevalidate } from 'workbox-strategies';
declare let self: ServiceWorkerGlobalScope;

const manifest = self.__WB_MANIFEST;

manifest.forEach((entry) => {
	if (typeof entry === 'object' && entry.url.endsWith('__data.json')) {
		entry.url = entry.url.replace(/^prerendered\/dependencies\//, '');
	}
});

cleanupOutdatedCaches();
precacheAndRoute(manifest, { ignoreURLParametersMatching: [/.*/] });

self.addEventListener('message', (event) => {
	if (event.data && event.data.type === 'SKIP_WAITING') self.skipWaiting();
});

registerRoute(/.*/, new StaleWhileRevalidate());

This hack is necessary because the __data.json files are stored in the prerendered/dependencies/ folder, and the
service worker's default behaviour incorrectly configures their URLs. This hack manually alters the URLs so that the
service worker can cache them correctly.

The text was updated successfully, but these errors were encountered:

userquin · 2024-12-31T11:10:49Z

can you provide a minimal reproduction and so I can check the kit folder? Right now I'm fixing similar problem for Kit SPA mode...

Or just updload an screenshot like this one (this is for SPA using static adatper):

userquin · 2024-12-31T11:37:02Z

I'm going to update the buildGlobPatterns function to include json and js files and removing also 'prerendered/dependencies/' prefix in the manifest transform:

function buildGlobPatterns(globPatterns?: string[]) {
  if (globPatterns) {
    if (!globPatterns.some(g => g.startsWith('prerendered/')))
      globPatterns.push('prerendered/**/*.{html,json,js}')

    if (!globPatterns.some(g => g.startsWith('client/')))
      globPatterns.push('client/**/*.{js,css,ico,png,svg,webp,webmanifest}')

    if (!globPatterns.some(g => g.includes('webmanifest')))
      globPatterns.push('client/*.webmanifest')

    return globPatterns
  }

  return ['client/**/*.{js,css,ico,png,svg,webp,webmanifest}', 'prerendered/**/*.{html,json,js}']
}

WhyAsh5114 · 2024-12-31T11:46:15Z

Yep, that should fix it

WhyAsh5114 · 2024-12-31T11:54:02Z

Also, I had another doubt. When using this strategy, the $offlineReady store returned from the useRegisterSW() function is never true.

Am I misunderstanding something? How would I know if all the assets have been precached on the frontend?

userquin · 2024-12-31T12:04:42Z

$offlineReady will be called/activated only once, when the service worker is installed and activated first time.

Uhmmm, looks like the .svelte-kit/output/prerendered/dependencies/_app/env.js file is added after PWA plugin builds the service worker, and so there is no way to fix this: I'm going to update #97 with previous fix, can you test it in your local with this dependency https://pkg.pr.new/@vite-pwa/sveltekit@488aa8c ?

WhyAsh5114 · 2024-12-31T12:05:59Z

sure, I'll let you know

userquin · 2024-12-31T12:06:20Z

@WhyAsh5114 are you using SPA https://svelte.dev/docs/kit/single-page-apps?

WhyAsh5114 · 2024-12-31T12:06:54Z

not really, I'm using adapter-static, with export const prerender = true in root layout.ts

userquin · 2024-12-31T12:09:04Z

Am I misunderstanding something? How would I know if all the assets have been precached on the frontend?

You need to check it in the browser devtools: Application > Cache Storage

not really, I'm using adapter-static, with export const prerender = true in root layout.ts

Can you provide a minimal reproduction?

WhyAsh5114 · 2024-12-31T12:09:35Z

Uhmmm, looks like the .svelte-kit/output/prerendered/dependencies/_app/env.js file

Also, the env.js file isn't really of interest, the __data.json files are though IMO, since when I navigate using button clicks on the app, the SW tries to fetch that page's __data.json if it has a load() (or maybe $props()/export let data).

Ohkk, I'll make a minimal repro and link it here.

userquin · 2024-12-31T12:13:14Z

Since you're using static adapter (SPA requires it), maybe moving the buildEnd hook from pre to post works, I need to do some tests.

I have no idea why kit not applying adapter logic like normal build, looks like prerendered/dependencies files built without awaiting... (maybe a bug or missing await somewhere).

WhyAsh5114 · 2024-12-31T12:19:07Z

Minimal repro: https://github.com/WhyAsh5114/prerendered-pwa

See the service-worker.ts file, if you uncomment the hacky fix, only then the SW registers properly.

userquin · 2024-12-31T12:31:25Z

Thx, looks like we need to move the kit pwa build plugin from 'pre' to 'post' (register it after kit), this way we've now prerendered/dependencies/_app/env.js in the sw precache manifest.

userquin · 2024-12-31T12:39:32Z

There is a problem, now the service worker being generated in the kit output folder, but the adapter will not copy the service worker since it is being generated after the adapter runs (the adapter will copy the client folder without the sw).

WhyAsh5114 · 2024-12-31T12:42:44Z

hmm, but again, why would we need the env.js file? does it work if we just get __data.json?

Or are those files also generated after the PWA plugin builds?

WhyAsh5114 · 2024-12-31T12:43:32Z

I'm not even sure it contains anything useful, in my minimal repro it's just this:

export const env={}

userquin · 2024-12-31T12:50:56Z

I'm going to check your reproduction, looks like the $service-worker not including the _app/env.js file in the build/files:

WhyAsh5114 · 2024-12-31T12:52:15Z

Yes, my globPattern doesn't include JS from the prerendered/client dir, only html,json

                                globPatterns: [
					'client/**/*.{js,css,ico,png,svg,webp,webmanifest,ttf,woff,woff2}',
					'prerendered/**/*.{html,json}'
				]

userquin · 2024-12-31T13:00:25Z

ok, the fix will work for __data.json files, kit is generating them since we've the entries in the sw precache manifest.

userquin · 2024-12-31T13:04:41Z

And working offline:

WhyAsh5114 · 2024-12-31T13:08:27Z

Yep, perfect. I expected this by default, maybe we should also update the docs for this, let me know if I can help!

userquin · 2024-12-31T13:08:58Z

I'm going to release a patch version, in the meantime you can check it in your local with this dependency: https://pkg.pr.new/@vite-pwa/sveltekit@a7c0605 (I guess once PR merged the pkg.pr.new is removed)

userquin · 2024-12-31T13:09:23Z

Yep, perfect. I expected this by default, maybe we should also update the docs for this, let me know if I can help!

Checking the docs...

WhyAsh5114 · 2024-12-31T13:10:46Z

I'm going to release a patch version, in the meantime you can check it in your local with this dependency: https://pkg.pr.new/@vite-pwa/sveltekit@a7c0605 (I guess once PR merged the pkg.pr.new is removed)

I will do that once I get back, out rn

userquin · 2024-12-31T13:31:34Z

vite-pwa/docs#172

WhyAsh5114 · 2024-12-31T13:55:01Z

I'm going to release a patch version, in the meantime you can check it in your local with this dependency: https://pkg.pr.new/@vite-pwa/sveltekit@a7c0605 (I guess once PR merged the pkg.pr.new is removed)

Umm, I installed this and remove my custom URL manipulation. It doesn't seem to work, did you push the patch?

And the docs change looks good.

userquin · 2024-12-31T14:02:57Z

Remove the storage including the service worker and reload the page, you may have old sw (you should use a virtual, vanillajs or sveltekit for page reload): check if the old service worker is in skip waiting state (yellow dot in devtools)

WhyAsh5114 · 2024-12-31T14:04:16Z

Yes, I did try that, clear storage, empty cache and hard reload. I was still getting this error. Same minimal repro with following changes:

New package
vite.config.ts

import { sveltekit } from '@sveltejs/kit/vite';
import { defineConfig } from 'vite';
import { SvelteKitPWA } from '@vite-pwa/sveltekit';

export default defineConfig({
	plugins: [
		sveltekit(),
		SvelteKitPWA({
			manifestFilename: 'manifest.webmanifest',
			strategies: 'injectManifest',
			filename: 'service-worker.ts',
			injectManifest: {
				globPatterns: [
					'client/**/*.{js,css,ico,png,svg,webp,webmanifest,ttf,woff,woff2}',
					'prerendered/**/*.{html,json}'
				]
			}
		})
	]
});

service-worker.ts

/// <reference lib="WebWorker" />
/// <reference types="@sveltejs/kit" />

import { cleanupOutdatedCaches, precacheAndRoute } from 'workbox-precaching';
declare let self: ServiceWorkerGlobalScope;

cleanupOutdatedCaches();
precacheAndRoute(self.__WB_MANIFEST, { ignoreURLParametersMatching: [/.*/] });

self.addEventListener('message', (event) => {
	if (event.data && event.data.type === 'SKIP_WAITING') self.skipWaiting();
});

userquin · 2024-12-31T14:08:30Z

Try adding includeVersionFile: true to kit pwa options: you can also remove injectManifest options, looks like your options are the default ones

WhyAsh5114 · 2024-12-31T14:10:31Z

Still the same, it only happens when I go offline. The cache entry is there though, so is the version.json

userquin · 2024-12-31T14:12:31Z

Check if you have checked Disable cache in network tab, uncheck it (disabling cache will bypass the sw):

WhyAsh5114 · 2024-12-31T14:13:00Z

And working offline:

Try doing it in this way:

npm run bulid && npm run preview
go to localhost:4173
now go offline and try to click on the link

WhyAsh5114 · 2024-12-31T14:13:12Z

Check if you have checked Disable cache in network tab, uncheck it:

it is unchecked

userquin · 2024-12-31T14:14:03Z

WhyAsh5114 · 2024-12-31T14:15:29Z

You are getting that after removing the url manipulation from the service worker?
If so, I'm messing something up on my side, this package was the one I was supposed to install right?

		"@vite-pwa/sveltekit": "https://pkg.pr.new/@vite-pwa/sveltekit@a7c0605",

userquin · 2024-12-31T14:17:24Z

wait, I'm using local tgz file, checking changes...

userquin · 2024-12-31T14:17:52Z

In the meantime add the pwa icons...

userquin · 2024-12-31T14:23:10Z

Use this link https://pkg.pr.new/@vite-pwa/sveltekit@97

WhyAsh5114 · 2024-12-31T14:23:11Z

Ohk, need to head out for a bit, will get back in an hour or two and add the icons

userquin · 2024-12-31T14:24:01Z

userquin · 2024-12-31T14:26:07Z

Oh, we need font files (ttf,woff*), don't remove first globPattern.

userquin · 2024-12-31T15:54:40Z

@WhyAsh5114 kit pwa v0.6.7 released

WhyAsh5114 · 2024-12-31T16:07:04Z

Hey, it works!

I forgot to use registerRoute() on the paths so the sw wasn't participating in the fetch at all. My bad.

Once I added that, with v0.6.7, it works properly. Thank you very much!

WhyAsh5114 · 2024-12-31T17:06:47Z

Okay, maybe its not fully done yet. There's still some weird behavior. I've update the repro to include it. Steps to create an error:

npm run build && npm run preview
In a new private tab, cleared everything, empty cache and hard reload
Go offline
Stay on the root page, don't go to /dependent at all (important)
Hover on the link to go to /dependent, there'll be errors in the console
Even though the __data.json is cached, you'll still get this. But for some reason, manually changing the URL to localhost:4173/dependent will work without errors, and you can then go back and forth.
But starting from root (localhost:4173), and immediately going offline, and then trying to go to /dependent with the URL breaks it

WhyAsh5114 · 2024-12-31T17:09:04Z

For some reason, hovering (and clicking) on the link element, the request doesn't go through the service-worker.

userquin · 2024-12-31T17:52:41Z

Check https://github.com/WhyAsh5114/prerendered-pwa/pull/1 (about prefetching when hovering the links no idea what's happening, I get some errors but now it should be fixed in the PR => read my comment in the PR).

The PR including pwa icons (from kit pwa example) and removed manifest.webamanifest from static folder addding the content to pwa manifest option).

WhyAsh5114 · 2024-12-31T18:33:22Z

@userquin, I did merge it and tried to run it again. It still errors out when doing the steps mentioned above. But yeah, this issue has been fixed, its something I need to figure out on my own with workbox docs and all.

userquin · 2024-12-31T18:57:10Z

@WhyAsh5114 can you replace /./ with /.*/ in the sw? Check also https://developer.chrome.com/docs/workbox/modules/workbox-precaching#custom_manipulations

GoogleChrome/workbox#3291

WhyAsh5114 · 2024-12-31T19:01:04Z

yep, tried that, still issues

I'll look into the mentioned issue tomorrow, a bit tired rn

WhyAsh5114 · 2025-01-01T07:47:08Z

@userquin Okay, I finally fixed it. All we just needed to do was:

self.addEventListener('activate', (event) => {
	event.waitUntil(self.clients.claim());
	console.log('Service worker activated');
});

The service worker wasn't taking control on the initial page load, and therefore unable to serve cache resulting in errors as the requests were not being passed to it anyways.

There was no need to modify the registration logic in the root +layout.svelte or even include the version file in the kit config. Updated the repro to reflect all this with default config and still working.

WhyAsh5114 · 2025-01-01T07:50:00Z

In brief:
The install event is the first event a service worker gets, and it only happens once.
A promise passed to installEvent.waitUntil() signals the duration and success or failure of your install.
A service worker won't receive events like fetch and push until it successfully finishes installing and becomes "active".
By default, a page's fetches won't go through a service worker unless the page request itself went through a service worker. So you'll need to refresh the page to see the effects of the service worker.
clients.claim() can override this default, and take control of non-controlled pages.

https://web.dev/articles/service-worker-lifecycle

userquin · 2025-01-01T09:57:49Z

Happy new year 🎉

You're using auto update, the code should be:

self.skipWaiting()
self.addEventListener('activate', (event) => {
	event.waitUntil(self.clients.claim());
	console.log('Service worker activated');
});

/* THIS IS USED FOR PROMPT FOR UPDATE: CLIENTS WILL NOT EMIT THIS EVENT
self.addEventListener('message', (event) => {
	if (event.data && event.data.type === 'SKIP_WAITING') self.skipWaiting();
});
*/

userquin · 2025-01-01T10:02:22Z

About the logic in the layout, you should avoid calling r.update() when offline, the virtual callback you're using is deprecated, use the new one with the sw url: when your code base grows, if you call update while updating you may have some problems, increase the timeout in production, 30 minutes or even more should be fine.

WhyAsh5114 · 2025-01-01T10:44:39Z

Happy new year to you too! Thank you for the fixes :)

userquin added the bug Something isn't working label Dec 31, 2024

userquin mentioned this issue Dec 31, 2024

fix: add SPA support #97

Merged

userquin closed this as completed in #97 Dec 31, 2024

userquin reopened this Dec 31, 2024

userquin closed this as completed Dec 31, 2024

Precaching bug with adapter-static and load() functions #98

Precaching bug with adapter-static and load() functions #98

Comments

WhyAsh5114 commented Dec 31, 2024

The main problem

The hacky fix

userquin commented Dec 31, 2024

userquin commented Dec 31, 2024

WhyAsh5114 commented Dec 31, 2024

WhyAsh5114 commented Dec 31, 2024

userquin commented Dec 31, 2024

WhyAsh5114 commented Dec 31, 2024

userquin commented Dec 31, 2024

WhyAsh5114 commented Dec 31, 2024

userquin commented Dec 31, 2024

WhyAsh5114 commented Dec 31, 2024 • edited Loading

userquin commented Dec 31, 2024 • edited Loading

WhyAsh5114 commented Dec 31, 2024 • edited Loading

userquin commented Dec 31, 2024

userquin commented Dec 31, 2024

WhyAsh5114 commented Dec 31, 2024

WhyAsh5114 commented Dec 31, 2024

userquin commented Dec 31, 2024 • edited Loading

WhyAsh5114 commented Dec 31, 2024

userquin commented Dec 31, 2024

userquin commented Dec 31, 2024

WhyAsh5114 commented Dec 31, 2024

userquin commented Dec 31, 2024

userquin commented Dec 31, 2024

WhyAsh5114 commented Dec 31, 2024

userquin commented Dec 31, 2024

WhyAsh5114 commented Dec 31, 2024

userquin commented Dec 31, 2024 • edited Loading

WhyAsh5114 commented Dec 31, 2024 • edited Loading

userquin commented Dec 31, 2024 • edited Loading

WhyAsh5114 commented Dec 31, 2024 • edited Loading

userquin commented Dec 31, 2024 • edited Loading

WhyAsh5114 commented Dec 31, 2024

WhyAsh5114 commented Dec 31, 2024

userquin commented Dec 31, 2024

WhyAsh5114 commented Dec 31, 2024

userquin commented Dec 31, 2024

userquin commented Dec 31, 2024

userquin commented Dec 31, 2024

WhyAsh5114 commented Dec 31, 2024

userquin commented Dec 31, 2024

userquin commented Dec 31, 2024

userquin commented Dec 31, 2024

WhyAsh5114 commented Dec 31, 2024

WhyAsh5114 commented Dec 31, 2024

WhyAsh5114 commented Dec 31, 2024

userquin commented Dec 31, 2024 • edited Loading

WhyAsh5114 commented Dec 31, 2024

userquin commented Dec 31, 2024 • edited Loading

WhyAsh5114 commented Dec 31, 2024

WhyAsh5114 commented Jan 1, 2025

WhyAsh5114 commented Jan 1, 2025

userquin commented Jan 1, 2025 • edited Loading

userquin commented Jan 1, 2025 • edited Loading

WhyAsh5114 commented Jan 1, 2025

Precaching bug with `adapter-static` and `load()` functions #98

Precaching bug with `adapter-static` and `load()` functions #98

WhyAsh5114 commented Dec 31, 2024 •

edited

Loading

userquin commented Dec 31, 2024 •

edited

Loading

WhyAsh5114 commented Dec 31, 2024 •

edited

Loading

userquin commented Dec 31, 2024 •

edited

Loading

userquin commented Dec 31, 2024 •

edited

Loading

WhyAsh5114 commented Dec 31, 2024 •

edited

Loading

userquin commented Dec 31, 2024 •

edited

Loading

WhyAsh5114 commented Dec 31, 2024 •

edited

Loading

userquin commented Dec 31, 2024 •

edited

Loading

userquin commented Dec 31, 2024 •

edited

Loading

userquin commented Dec 31, 2024 •

edited

Loading

userquin commented Jan 1, 2025 •

edited

Loading

userquin commented Jan 1, 2025 •

edited

Loading