Benchmarks Update 2023-12-09

[alexanderkozlenko]

Testing Runtime

.NET 8.0.0 (8.0.23.53103)

Libraries Tested

Package	Version	Name
Addax.Formats.Tabular	1.2.0	Addax
CsvHelper	30.0.1	CsvHelper
Microsoft.Data.Analysis	0.21.0	DataFrame

Benchmark for Strings

1,048,576 records with 4 fields of type string each:

| Method | Engine    | Mean        | Median      | Op/s    | Gen0      | Gen1      | Allocated    |
|------- |---------- |------------:|------------:|--------:|----------:|----------:|-------------:|
| read   | Addax     |   181.94 ms |   182.01 ms |  5.4963 |         - |         - |  234881445 B |
| write  | Addax     |    88.18 ms |    88.05 ms | 11.3411 |         - |         - |        499 B |
| read   | CsvHelper |   417.60 ms |   417.51 ms |  2.3946 |         - |         - |  335569256 B |
| write  | CsvHelper |   163.68 ms |   163.81 ms |  6.1094 |  333.3333 |         - |  369118197 B |
| read   | DataFrame | 1,472.43 ms | 1,470.42 ms |  0.6791 | 9000.0000 | 3000.0000 | 7190094944 B |
| write  | DataFrame |   299.01 ms |   300.00 ms |  3.3443 |  500.0000 |         - |  754982824 B |

Benchmark for Numbers

1,048,576 records with 4 fields of type float64 each:

| Method | Engine    | Mean       | Median     | Op/s   | Gen0      | Gen1      | Allocated    |
|------- |---------- |-----------:|-----------:|-------:|----------:|----------:|-------------:|
| read   | Addax     |   265.0 ms |   264.6 ms | 3.7735 |         - |         - |        728 B |
| write  | Addax     |   369.4 ms |   370.6 ms | 2.7067 |         - |         - |       1072 B |
| read   | CsvHelper |   591.5 ms |   591.2 ms | 1.6907 |         - |         - |  436233176 B |
| write  | CsvHelper |   541.2 ms |   541.3 ms | 1.8479 |         - |         - |  436227472 B |
| read   | DataFrame | 1,583.7 ms | 1,583.5 ms | 0.6314 | 8000.0000 | 1000.0000 | 6452465440 B |
| write  | DataFrame |   771.9 ms |   771.9 ms | 1.2955 | 1000.0000 |         - |  923806400 B |

Benchmark for Dates

1,048,576 records with 4 fields of type datetime each:

| Method | Engine    | Mean        | Median      | Op/s    | Gen0      | Gen1      | Allocated    |
|------- |---------- |------------:|------------:|--------:|----------:|----------:|-------------:|
| read   | Addax     |   571.75 ms |   572.29 ms |  1.7490 |         - |         - |       1072 B |
| write  | Addax     |    97.12 ms |    97.18 ms | 10.2968 |         - |         - |        499 B |
| read   | CsvHelper | 1,008.59 ms | 1,008.61 ms |  0.9915 |         - |         - |  536897096 B |
| write  | CsvHelper |   325.35 ms |   325.87 ms |  3.0736 |  500.0000 |         - |  536890976 B |
| read   | DataFrame | 2,061.74 ms | 2,061.62 ms |  0.4850 | 9000.0000 | 1000.0000 | 6769192408 B |
| write  | DataFrame |   668.32 ms |   668.44 ms |  1.4963 | 1000.0000 |         - | 1200630184 B |

Benchmark for Mixed Content

1,048,576 records with 4 fields of types string, boolean, float64, datetime each:

| Method | Engine    | Mean       | Median     | Op/s   | Gen0      | Gen1      | Allocated    |
|------- |---------- |-----------:|-----------:|-------:|----------:|----------:|-------------:|
| read   | Addax     |   298.6 ms |   299.4 ms | 3.3491 |         - |         - |   58720984 B |
| write  | Addax     |   165.9 ms |   166.0 ms | 6.0266 |         - |         - |        556 B |
| read   | CsvHelper |   614.9 ms |   616.2 ms | 1.6262 |         - |         - |  411069120 B |
| write  | CsvHelper |   378.9 ms |   379.5 ms | 2.6394 |         - |         - |  478171360 B |
| read   | DataFrame | 1,661.0 ms | 1,661.3 ms | 0.6021 | 9000.0000 | 3000.0000 | 6806759968 B |
| write  | DataFrame |   616.6 ms |   615.3 ms | 1.6218 | 1000.0000 |         - |  942417680 B |

Legend

Engine    : Value of the 'Engine' parameter
Mean      : Arithmetic mean of all measurements
Median    : Value separating the higher half of all measurements (50th percentile)
Op/s      : Operation per second
Gen0      : GC Generation 0 collects per 1000 operations
Gen1      : GC Generation 1 collects per 1000 operations
Allocated : Allocated memory per single operation (managed only, inclusive, 1KB = 1024B)
1 ms      : 1 Millisecond (0.001 sec)