57. The three levels of Distribution/Parallelization
● For each subset of the population (e.g. region)
● For each combination of the hyperparameters
● For each subset of the training data
Each level has different requirements
ANN Training over distributed GPU’s