|
1 | 1 | # DLIO benchmark |
2 | 2 |
|
3 | | -The benchmark is designed to measure the performance of training and evaluation of deep learning models on data stored |
4 | | -as HDF5 files. Based on collected and analysed I/O patterns from [DLIO Benchmark](https://github.com/argonne-lcf/dlio_benchmark), |
5 | | -this benchmark simulates the learning process and evaluation of deep learning models using PyTorch and Tensorflow |
6 | | -frameworks, while gathering valuable information about system performance. |
| 3 | +The benchmark is designed to measure the performance of training and evaluation of deep learning models on data stored |
| 4 | +as HDF5 files. Based on collected and analysed I/O patterns from [DLIO Benchmark](https://github.com/argonne-lcf/dlio_benchmark), |
| 5 | +this benchmark simulates the learning process and evaluation of deep learning models that use PyTorch and Tensorflow |
| 6 | +frameworks, while gathering valuable information about system performance. Most importantly, this extension allows users |
| 7 | +to test AI workloads without the need to install machine learning libraries, reducing complexity and enhancing the |
| 8 | +usability of the benchmark. Another advantage is that from our experiments, our extension ran faster than DLIO Benchmark, |
| 9 | +which we suspect was due to the difference in the overhead introduced by the C application in our extension and the |
| 10 | +Python application in the original benchmark. While the quicker runtime could be beneficial for faster testing, it also |
| 11 | +suggests that the benchmark might not fully capture the complexity of real AI workloads, such as high metadata |
| 12 | +operations introduced by the use of Python-based libraries. I/O pattern produced by this extension is based on the |
| 13 | +implementation of [DLIO benchmark version 1.1](https://github.com/argonne-lcf/dlio_benchmark/releases/tag/v1.1). |
| 14 | +Changes in the main DLIO Benchmark configurations after version 1.1 will not be reflected in this h5bench pattern. To |
| 15 | +reproduce them, DLIO Benchmark behavior can be studied using various I/O analysis tools. We recommend using Log VFD. |
| 16 | + |
7 | 17 |
|
8 | 18 | ## Configuration |
9 | 19 | As in the case with other extensions, the following parameters should be specified in the configuration section of the |
|
0 commit comments