Save & Reload DNS cache

[gspannu]

It would be nice to have an (optional) ability to save and reload cache.

At the moment, adding whitelists/ blacklists/ making config changes requires a restart of Blocky and that loses all the built-up cache.

Is it possible to have the ability to save/reload cache - even if the memory file is saved as a binary blob file and reloaded as such.

I am aware that Blocky is supposed to run stateless and have minimal config (i.e. just 1 file)… but losing the built up cache on each restart needs some sort of resolution….

Thanks…

[kwitsch]

You could add Redis to your setup which will store your DNS cache and blocky will automatically pull it on startup.

Another option would be to run two blocky instances where the first would only cache and forward the requests(+ the IP of the requesting client) to the second which does the blocking.

A third option would be a combination of both. In that case both blocky instances use the same Redis instances which synchronizes the DNS cache between both instances in addition to the restore functionality mentioned above.

[gspannu]

Thank you for the multiple solutions.
Is there any basic guide / tutorial to just use Redis to store DNS cache and Blocky to pull on startup?

Apologies, newbie here...

[kwitsch]

You could just set up a Redis server with default options and add it to your blocky config according to the config documentation.

No special settings are required since it is the default behavior if redis is configured.

Ps: if you are using Docker I could provide a compose example later on.

[gspannu]

Thanks. I am not using Docker…

I have a test Blocky instance running on a Proxmox Virtual machine.
I will install Redis on the same VM and experiment there. Once it is all working, I will get Blocky/Redis working on my OPNsense router for my production setup.

Thanks…

A couple of quick questions:

• I believe that Blocky will automatically save & reload cache (once configured in the Blocky yaml file). Do I need to do anything special or is that default behaviour (once Blocky config contains the Redis section)

• If I want to completely clear the cache, do I just call the Blocky API to clear cache or is there some way to clear all the entries in Redis and instead restart Blocky, so that it reloads a clean setup?

• What is the recommended way to completely clear cache (if ever required), if Blocky is configured to use Redis?

• Are there any other recommendations I should follow (for the Solution 1 setup. Single instance of Blocky with Redis)

[kwitsch]

• correct, no additional configuration needed
• since Redis by default doesn't store it's entries on disk, you could simply restart blocky and redid at the same time to clear all caches(best would be: Stop blocky-> restart Redis-> start blocky)
• recommendation: limit DB count in Redis- to one and restrict it's accessibility only to blocky(should be done after you've got a working setup since limiting access makes finding issues harder)

[gspannu]

Have played around with Redis and got it working... but I have a few more queries about how/ what Blocky stores/ reloads in Redis.

I understand that Redis stores the keys with the same TTL as what Blocky returns.

I am having a system where every client DNS request will only be fetched from upstream the first time, it will always be available cached the next time. Setting prefetchMaxItemsCount: 0
Assume memory available to be large enough... the system will now serve the quickest DNS resolution, as every subsequent request is always available cached.

I have configured Blocky to always prefetch any query by using prefetchThreshold: 0 in config.yml.

prefetching: true
prefetchExpires: 30d
prefetchThreshold: 0 
prefetchMaxItemsCount: 0

Blocky works fine, i.e.

• Blocky prefetches the query from upstream prior to the TLL expiring; and keeps repeating this process until prefetchExpires, lets say 30 days.
• thereby ensuring that all clients ALWAYS receive a cached value (for the next 30 days)

---

I have noticed that Blocky updates Redis for the first time it fetches the entry from upstream.
These entries in Redis then expire after TTL (as expected).

However, when Blocky prefetches the entry to update its cache, it does not seem to update Redis.

---

This means that Redis does not actually store a complete picture of Blocky cache. And a restart of Blocky does not restore the complete DNS cache

Or have I misunderstood something?

[kwitsch]

I looked into it and it seems like you're correct.
The prefetched value isn't published to Redis.

This is a bug.

[kwitsch]

@gspannu I opened issue #1422 for it.