Whitepaper - Cerebras

Training Giant Neural Networks Using Weight Streaming on Cerebras Wafer-Scale Systems

Rebecca Lewington — Fri, 24 Mar 2023 18:00:44 +0000

In this paper, we survey existing approaches used to scale training to clusters of compute units and explore the limitations of each in the face of giant models. We present a new paradigm for giant model training, called weight streaming,…

The post Training Giant Neural Networks Using Weight Streaming on Cerebras Wafer-Scale Systems appeared first on Cerebras.

Deep Learning Programming at Scale

Rebecca Lewington — Mon, 06 Jun 2022 18:27:06 +0000

Deep learning has become one of the most important computational workloads of our generation, advancing applications across industries from healthcare to autonomous driving. But it is also profoundly computationally intensive. (Updated June 2022.)…

The post Deep Learning Programming at Scale appeared first on Cerebras.

Powering Extreme-Scale HPC with Cerebras WaferScale Accelerators

Rebecca Lewington — Wed, 06 Apr 2022 23:54:21 +0000

In this paper, we will explore the challenges facing HPC developers today and show how the Cerebras architecture can help to accelerate sparse linear algebra and tensor workloads, stencilbased partial differential equation (PDE) solvers, N-body problems, and spectral algorithms such…

The post Powering Extreme-Scale HPC with Cerebras WaferScale Accelerators appeared first on Cerebras.

The Cerebras Software Development Kit: A Technical Overview

Rebecca Lewington — Tue, 08 Feb 2022 01:06:52 +0000

Cerebras has introduced a new software development kit (SDK) which allows anyone to take advantage of the strengths of the CS-2 system. Developers can use the Cerebras SDK to create custom kernels for their standalone applications or modify the kernel…

The post The Cerebras Software Development Kit: A Technical Overview appeared first on Cerebras.

Cerebras Systems Enables Brain-scale AI

Rebecca Lewington — Wed, 22 Sep 2021 00:12:54 +0000

This research paper explores Cerebras System’s approach to create a brain-scale AI and the new technologies that could enable that feat. But first, let’s put this discussion into the proper context. Just how big is a 120 trillion-parameter model?…

The post Cerebras Systems Enables Brain-scale AI appeared first on Cerebras.

Limits to Scale-Out for Training Language Models

Rebecca Lewington — Fri, 25 Jun 2021 00:23:58 +0000

Natural language processing has revolutionized how data is consumed, meaning that computational demand has skyrocketed. Companies in every industry are using graphics processing unit (GPU) clusters to keep up. But is this really the best solution?…

The post Limits to Scale-Out for Training Language Models appeared first on Cerebras.

Train Large BERT Models Faster with Cerebras Systems

Rebecca Lewington — Tue, 25 May 2021 00:39:22 +0000

Unstructured text is one of the largest human-generated data sources. Web data, academic publications, emails, traditional media, texts, instant messages, digital records, social media — all hold an enormous volume of unstructured text.…

The post Train Large BERT Models Faster with Cerebras Systems appeared first on Cerebras.

Cerebras Systems: Achieving Industry Best AI Performance Through A Systems Approach

Rebecca Lewington — Wed, 07 Apr 2021 00:51:34 +0000

The CS-2 is a system solution that consists of innovations across three dimensions: a) the second
generation Cerebras Wafer Scale Engine (WSE-2) — the industry’s largest and only multi-trilliontransistor processor, b) the Cerebras System and c) the Cerebras software platform.…

The post Cerebras Systems: Achieving Industry Best AI Performance Through A Systems Approach appeared first on Cerebras.