Chewing your input: DataWizard

[arranzeyuwang]

DataWizard is the data processing and precomputing library of AVA, which has the historical legacy of confusing code bases, unclear file structures, and unstandardized calls.

Its previous version in AVA 1.0 was divided into four libs: Analyzer, Transformer, Randomer, and Utils, which makes the problem much more complex. In the new plan, DataWizard will be the unique library that is expected to be exposed after the refactoring.

Its new structure is shown as follows:

The new DataWizard contains the following components and functions:

1. General functions:
   Analyzer: Data type analyzer,
   Random: Random number generator,
   Statistics: Statistics-related methods,
   Transformer: Data transformation, aggregation, etc.
2. DataSet (non-entity concept):
   DataFrame: similar to the Spreadsheet table structure, the input data will be converted to DataFrame
   Series: row or column in the table, excluding the header
   Cache: cached via weakMap structure, only the calculated information will be stored to avoid memory leaks

Back to the Big Picture.