v5, createPersister and IndexedDB: absolutely amazing!

[frederikhors]

I'm using the latest v5 version: AMAZING!!!!!!! Thanks!

I tried the experimental_createPersister and it works amazingly too!

I'm using it like this, what do you think?

import {
  experimental_createPersister,
  type AsyncStorage,
} from "@tanstack/query-persist-client-core";
import { get, set, del, createStore, type UseStore } from "idb-keyval";

function newIdbStorage(idbStore: UseStore): AsyncStorage {
  return {
    getItem: async (key) => await get(key, idbStore),
    setItem: async (key, value) => await set(key, value, idbStore),
    removeItem: async (key) => await del(key, idbStore),
  };
}

export const queryClient = new QueryClient({
  defaultOptions: {
    queries: {
      gcTime: 1000 * 30, // 30 seconds
      persister: !browser
        ? undefined
        : experimental_createPersister({
            storage: newIdbStorage(createStore("db_name", "store_name")),
            maxAge: 1000 * 60 * 60 * 12, // 12 hours
          }),
    },
  },
});

[TkDodo]

looks lovely

[TheTimeWalker]

Hey! Thank you so much for the snippet. Works pretty great!

One suggestion: The advantage of IndexedDB is that you can safely put objects into the storage without having to rely on serializing/deserializing the content like e.g. with LocalStorage. I have noticed that right now it would by default still stringify the content and then save it to IndexedDB. My suggestion would be to additionally add this into your persister:

experimental_createPersister({
  storage: newIdbStorage(createStore("db_name", "store_name")),
  maxAge: 1000 * 60 * 60 * 12, // 12 hours,
  // Just pass the data
  serialize: (persistedQuery) => persistedQuery,
  deserialize: (cached) => cached,
})

The only problem is that TypeScript will complain that this serialize/deserialize function is not assignable, because it expects a string as a return type and cached type.

[TkDodo]

one problem is that we default to JSON.stringify and JSON.parse, so if you pass one of the methods but not the other, typescript complains. If you can figure it out, I'll be happy to merge it :)

[TheTimeWalker]

I think that shouldn't be a problem 👍🏻
Since if you change the serializer to not do stringify, you'll have to change the deserializer anyway because it would try to parse a string by default. 😛

[TkDodo]

yes, but we need to get this right on type level. This shouldn't err at runtime

[frederikhors]

@TkDodo @TheTimeWalker What do you think if I open an RFC in the issues to discuss it better?

[TkDodo]

Yes please 👍

[TheTimeWalker]

With the newest version (v5.10), you can now add a type to the persister, so now we can combine my serializer solution with your IndexedDB solution :)

import {
  experimental_createPersister,
  type AsyncStorage,
  type PersistedQuery
} from "@tanstack/query-persist-client-core";
import { get, set, del, createStore, type UseStore } from "idb-keyval";

function newIdbStorage(idbStore: UseStore): AsyncStorage<PersistedQuery> {
  return {
    getItem: async (key) => await get(key, idbStore),
    setItem: async (key, value) => await set(key, value, idbStore),
    removeItem: async (key) => await del(key, idbStore),
  };
}

export const queryClient = new QueryClient({
  defaultOptions: {
    queries: {
      gcTime: 1000 * 30, // 30 seconds
      persister: !browser
        ? undefined
        : experimental_createPersister<PersistedQuery>({
            storage: newIdbStorage(createStore("db_name", "store_name")),
            maxAge: 1000 * 60 * 60 * 12, // 12 hours,
            serialize: (persistedQuery) => persistedQuery,
            deserialize: (cached) => cached,
          }),
    },
  },
});

Under AsyncStorage and createrPersister, just add <PersistedQuery>

[frederikhors]

Amazing! Amazing you, tanstack's team, all the open source world!

[tam-eoh]

When I use persistQueryClient, my data immediately appears upon reloading. However, when I switch to experimental_createPersister, my data is not there when reload. Could this mean I've set up something incorrectly?

[TkDodo]

That is the intended behavior because the data is only loaded when a useQuery call needs it. That is more memory efficient because not everything is loaded upfront. If you want to load everything upfront, the PersitQueryClient is what you want. Both approaches have their pros and cons.

[hverlin]

Re-reading this question, seems similar to my question: #6213 (comment)

Is the solution to use both persister together? Or use setQueryData for critical queries if we want to avoid any loading states?

[joelbrewer]

I must be doing something wrong, but I'm getting the following error when using experimental_createPersister :

Here's my code:

...
import { QueryClient } from '@tanstack/react-query';
import { PersistQueryClientProvider } from '@tanstack/react-query-persist-client'
import { experimental_createPersister } from "@tanstack/query-persist-client-core";
import { get, set, del, createStore } from "idb-keyval";
...

function newIdbStorage(idbStore) {
  return {
    getItem: async (key) => await get(key, idbStore),
    setItem: async (key, value) => await set(key, value, idbStore),
    removeItem: async (key) => await del(key, idbStore),
  };
}

export const queryClient = new QueryClient({
  defaultOptions: {
    queries: {
      gcTime: 1000 * 30, // 30 seconds
    },
  },
});

const Root = () => {
  const persister = experimental_createPersister({
    storage: newIdbStorage(createStore("my_db", "my_store")),
    maxAge: 1000 * 60 * 60 * 12, // 12 hours
  }); 
 
...

  return (
    <UserProvider>
      <PersistQueryClientProvider
        client={queryClient}
        persistOptions={{ persister }}
      >
        <RouterProvider router={router}>
          <ReactQueryDevtools initialIsOpen={false} />
        </RouterProvider>
      </PersistQueryClientProvider>
    </UserProvider>
  );
}

export default Root;

[TkDodo]

experimental_createPersister is not something you can pass to PersistQueryClientProvider

[hverlin]

@frederikhors @TkDodo @TheTimeWalker Thank you for the suggested solutions here!

I have an additional question regarding this new experimental_createPersister API, do you restore the cache immediately when you load the app?

import { createStore, del, get, set, UseStore, keys, getMany } from 'idb-keyval';

export async function restoreQueryClientCache() {
  if (!idbStore) {
    return;
  }

  const cachedKeys = await keys(idbStore);
  const cachedValues = await getMany<PersistedQuery>(cachedKeys, idbStore);
  for (const queryData of cachedValues) {
    if (queryData?.buster !== UI_VERSION) {
      continue;
    }

    if (!queryData?.state?.data) {
      continue;
    }

    queryClient.setQueryData(queryData.queryKey, queryData.state.data, {
      updatedAt: queryData.state.dataUpdatedAt,
    });
  }
}

restoreQueryClientCache().catch(console.error);

root.render(...);

I am migrating from the solution which was suggested here.

import { get, set, del } from 'idb-keyval'
import { PersistedClient, Persister } from '@tanstack/react-query-persist-client'

export function createIDBPersister(idbValidKey: IDBValidKey = 'reactQuery') {
  return {
    persistClient: async (client: PersistedClient) => { await set(idbValidKey, client) },
    restoreClient: async () => { return await get<PersistedClient>(idbValidKey) },
    removeClient: async () => { await del(idbValidKey) },
  } as Persister
}

With the previous solution, the query would initially resolve with the data from the cache (which was quite nice as if one reloads the page, they immediately see the previous data - no loading state). After migrating to the new experimental_createPersister, the query will initially have no data until it reads from index db, leading to some flickering.

Is a workaround to run queryClient.setQueryData for all the queries stored in the cache before calling React render, as otherwise deserialization is lazy and will lead to flickering?

Wondering if there would be an option to make the query client immediately restore some of the queries?

[frederikhors]

@hverlin just to be clearer, are you talking about the loading that may occur between when the application starts and when the data is fetched from the cache?

[hverlin]

yes. Since fetching the data from the cache is async, if you don't use setQueryData you get some very short loading state