Performance degradation in cache DB

[rigon]

Performance of saving to bbolt degrades when the DB grows in size, even when it is just updating an existing entry. This is more noticeable when updating the thumb flag.

[rigon]

This happens because the underlying structure of indexes are maps. When reading/writing a key from an index, what effectively happens is reading the entire map from the DB, decoding into memory, make the necessary changes, encode in binary format and write it back to the DB.

Because the thumbnail flag only have two states (true of false), all photos will be referenced or by one (true) or the other (false). If the number of photos grows, so do the maps in the indexes.

In scenario where the thumbnails are already generated but the cache needs to update the thumbnail flag, what will happen is coping all the references from one key (false) to another (true), in batches. Each time a batch is ready to be committed, it must read, decode, apply changes, encode, write.

See timshannon/bolthold#105 and timshannon/bolthold#106