Compressing timestamps

[fatihky]

I think compressing timestamps before writing to disk would be cool. Even when storing on ram. I don't know any compression libraries for doubles(data type that used for values), so I can't suggest anything to compress values.

How about this idea?

Example integer compression libraries:
https://github.com/lemire/simdcomp (BSD)
https://github.com/powturbo/TurboPFor (GPL V2)

[danni-m]

This is a great idea, the links seem interesting.
I was thinking of implementing something similar to beringei encoding algorithm.
The issue i see with these libraries is that they act on complete array, I would prefer to have something that looks more like a streaming interface, every read/write operation will be done ontop of the compressed data and not as a bulk.

Also, antirez is working on a timestamp based streaming data stracture that will have some compression, I might use that in the future.

[fatihky]

Cool, then I am closing this.
Is antirez's work this?

[fatihky]

Sorry I didn't ask, is closing this ok?

[danni-m]

@fatihky regarding RCP11, yes.
I'll reopen, I believe that this is an important feature, maybe in the future i'll break it down to 2 different features:

1. on disk compression
2. in memory compression

Thanks for the feedback!

[fatihky]

Thank you for information, and you are welcome.

[danni-m]

Sorry for the mess, i've mistaken this ticket with the rollups/compcations.