Parallel I/O analysis in distributed deep learning applications on high-performance computing
Parraga Pinzon, Edixon Alexander (Universitat Autònoma de Barcelona. Departament d'Arquitectura de Computadors i Sistemes Operatius)
León, Betzabeth (Universitat Autònoma de Barcelona. Departament d'Arquitectura de Computadors i Sistemes Operatius)
Méndez, Sandra Adriana (Barcelona Supercomputing Center)
Rexachs, Dolores (Universitat Autònoma de Barcelona. Departament d'Arquitectura de Computadors i Sistemes Operatius)
Luque, Emilio (Universitat Autònoma de Barcelona. Departament d'Arquitectura de Computadors i Sistemes Operatius)

Date:	2025
Abstract:	Distributed deep learning (DDL) applications generate heavy input/output (I/O) workloads that can create bottlenecks in high-performance computing (HPC) systems. Their optimal I/O configuration depends on factors such as access patterns, storage hardware, dataset size, and execution scale. This study proposes a systematic methodology for characterizing and optimizing I/O behavior in DDL applications, represented through the deep learning I/O benchmark (DLIO), and validated with the real DeepGalaxy application. We evaluate access modes, file formats, and Lustre file system configurations, demonstrating that stripe counts optimized for the access pattern and application scale can reduce I/O and execution times, achieving up to 18 GiB/s of bandwidth and a 5X increase in IOPS. HDF5 provides balanced performance, while TFRecord stands out in bandwidth-intensive scenarios. Shared access minimizes contention and improves scalability in multi-node executions. The results are consolidated into configuration guidelines that offer practical recommendations for practitioners to tune DDL applications for efficient execution in HPC environments.
Grants:	Agencia Estatal de Investigación PID2023-147955NB-I00
Note:	Altres ajuts: acords transformatius de la UAB
Note:	Fundación Escuelas Universitarias Gimbernat; Centro Gallego de Supercomputación (CESGA) in the supercomputer FinisTerrae III to the project identified with RES-DATA-2022-1-0014
Rights:	Aquest document està subjecte a una llicència d'ús Creative Commons. Es permet la reproducció total o parcial, la distribució, la comunicació pública de l'obra i la creació d'obres derivades, fins i tot amb finalitats comercials, sempre i quan es reconegui l'autoria de l'obra original.
Language:	Anglès
Document:	Article ; recerca ; Versió publicada
Subject:	Distributed deep learning ; Parallel I/O ; HPC cluster ; I/O behavior pattern
Published in:	The journal of supercomputing, Vol. 81, Num. 16 (November 2025) , art. 1520, ISSN 1573-0484

DOI: 10.1007/s11227-025-07986-1

56 p, 8.4 MB

The record appears in these collections:
Articles > Research articles
Articles > Published articles