Set up 2-tier cache for multi-align

[tonydisera]

There are currently over ~500,000 files in the persist_cache directory. @anderspitman recommends that we change to a 2-tier caching scheme:

I recommend converting this to a tiered cache with 2 levels. So for example, if you have the hash b6b842c0d9572a0cd68888f7425b652f, it would be stored in cache/b6/b8/b6b842c0d9572a0cd68888f7425b652f. The reason I recommend 2 levels is because you currently have ~2000 files that start with b6 (and ~2000 for every other prefix). So if you only did one level you'd still have too many files per directory. But there are only 7 files that start with b6b8, so you should have plenty of room to grow.

[tonydisera]

@anderspitman, the node js multialign server currently uses the npm packaged node-cache, which does not support multiple levels. I searched for multi-level cache packages, and NPM shows multi-level-cache in the top 10 for popularity, quality, and performance. And it looks like the port will require minimal changes.

https://www.npmjs.com/package/multi-level-cache

Is there a package you recommend?

[anderspitman]

Not sure if that's what we want. They don't mention anything about how they store a large number of files on disk. Probably what we're looking for is a key value (kv) store with support for persisting to the filesystem and handling large numbers of keys.