Fig. 5From: Accelerating neural network training with distributed asynchronous and selective optimization (DASO)Cycling Flow Process flow diagram of the synchronization steps during the cycling phase where t is the batch number and S is the batches to wait before global synchronization. The weighted average is calculated as shown in Eq. (2)Back to article page