Fig. 1From: Accelerating neural network training with distributed asynchronous and selective optimization (DASO)MPI Groups An overview of a common node-based computer cluster with P nodes and four GPUs per node. GPU colors represent group membership. The dashed lines indicate GPU-to-GPU communication channelsBack to article page