ML Research Archives - Cerebras https://www.cerebras.net/tag/ml-research/ Mon, 18 Jul 2022 16:31:32 +0000 en-US hourly 1 https://wordpress.org/?v=6.3.2 https://www.cerebras.net/wp-content/uploads/2022/05/cropped-cerebras-logo-fav-32x32.png ML Research Archives - Cerebras https://www.cerebras.net/tag/ml-research/ 32 32 Neural Network Batch Size and Parallelism at Wafer Scale https://www.cerebras.net/blog/data-model-pipeline-parallel-training-neural-networks/ https://www.cerebras.net/blog/data-model-pipeline-parallel-training-neural-networks/#comments Fri, 03 Apr 2020 18:11:49 +0000 https://cerebras.net/data-model-pipeline-parallel-training-neural-networks/ As neural networks grow in size and computational demand, scaling across large, distributed systems is the dominant approach to keeping training times in check. Recent state of the art models in areas covering computer vision, natural language processing and reinforcement…

The post Neural Network Batch Size and Parallelism at Wafer Scale appeared first on Cerebras.

]]> https://www.cerebras.net/blog/data-model-pipeline-parallel-training-neural-networks/feed/ 1 Error Compensation Mechanism in Online Normalization https://www.cerebras.net/machine-learning/error-compensation-mechanism-in-online-normalization/ https://www.cerebras.net/machine-learning/error-compensation-mechanism-in-online-normalization/#respond Fri, 03 Apr 2020 15:54:07 +0000 https://cerebras.net/error-compensation-mechanism-in-online-normalization/ Online Normalization [1] is a normalization technique that uses moving average statistics to normalize activations in a neural network. As we have shown in our NeurIPS2019 paper, Online Normalization outperforms or is competitive with state of the art normalization…

The post Error Compensation Mechanism in Online Normalization appeared first on Cerebras.

]]> https://www.cerebras.net/machine-learning/error-compensation-mechanism-in-online-normalization/feed/ 0