Pinned Loading

1. argonne-lcf/dlio_benchmark argonne-lcf/dlio_benchmark Public

   An I/O benchmark for deep Learning applications

   Python 90 43

2. HDFGroup/vol-cache HDFGroup/vol-cache Public

   HDF5 Cache VOL connector for caching data on fast storage layers and moving data asynchronously to the parallel file system to hide I/O overhead.

   C 21 8

3. dlio_profiling dlio_profiling Public

   This repo demonstrate how to profile I/O for deep learning applications. This is based on VaniDL

   Python

4. E3SM-IO E3SM-IO Public

   Forked from Parallel-NetCDF/E3SM-IO

   Benchmark programs using the I/O pattern of E3SM

   C++

5. blendcorpus blendcorpus Public

   BlendCorpus is a modular and scalable data preprocessing and loading framework for large language model (LLM) training.

   Python 1

6. torchtitan torchtitan Public

   Forked from pytorch/torchtitan

   A PyTorch native platform for training generative AI models

   Python