You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We currently we have multiple unrelated benchmarks for various scenarios:
generic Arctic top level calls
draft Arctic breakdown solution for keeping track of where time goes ((de)compress, numpy, serialization, MongoDB IO)
draft Arrow serialization benchmarks
The goal is to create a standard API for benchmarks:
requirements
specify experiement scenarios in an easy way (e.g. DSL or just a dict for fixed steps)
collection of results
plotting
break down to components (e.g. compress, numpy object creation, serialization, mongo IO)
make sure that when benchmark mode is disabled no impact on performance
reproducible benchmarks
goals
understand our code's bottlenecks
have a standard way to perform and repeat benchmarks
A skeleton of benchmarks exists in the top level directory, benchmarks.
There are some very basic examples and a readme (https://github.com/manahl/arctic/blob/master/benchmarks.md), but these should be expanded upon to include all the storage engines and some more involved use cases and examples (i.e. chunkstore with numerics only, vs chunkstore with strings, version store with pickled objects, etc).
The text was updated successfully, but these errors were encountered:
We currently we have multiple unrelated benchmarks for various scenarios:
The goal is to create a standard API for benchmarks:
requirements
goals
A skeleton of benchmarks exists in the top level directory, benchmarks.
There are some very basic examples and a readme (https://github.com/manahl/arctic/blob/master/benchmarks.md), but these should be expanded upon to include all the storage engines and some more involved use cases and examples (i.e. chunkstore with numerics only, vs chunkstore with strings, version store with pickled objects, etc).
The text was updated successfully, but these errors were encountered: