Skip to main content
Journal of Big Data

Table 1 Overview of distributed DL frameworks, adapted from [6, 17]

From: Large scale performance analysis of distributed deep learning frameworks for convolutional neural networks

Framework

Parallelism

Communication

DistBelief [18]

Model + Data

Asynchronous

FireCaffe [21]

Data

Synchronous

Horovod [5]

Model + Data

Synchronous

MXNet [23]

Model + Data

Bounded Asynchronous

Petuum [19]

Model + Data

Bounded Asynchronous

TensorFlow [22]

model + Data

Bounded Asynchronous

PyTorch-DDP [6]

Model + Data

Synchronous

DeepSpeed [7]

Model + Data

Synchronous

  1. Bounded asynchronous is a hybrid of synchronous and asynchronous communication
Back to article page