AI Research Projects Archives - Cerebras https://www.cerebras.net/tag/ai-research-projects/ Tue, 18 Apr 2023 17:07:11 +0000 en-US hourly 1 https://wordpress.org/?v=6.3.2 https://www.cerebras.net/wp-content/uploads/2022/05/cropped-cerebras-logo-fav-32x32.png AI Research Projects Archives - Cerebras https://www.cerebras.net/tag/ai-research-projects/ 32 32 Cerebras-GPT: Open Compute-Optimal Language Models Trained on the Cerebras Wafer-Scale Cluster https://arxiv.org/abs/2304.03208#new_tab Fri, 07 Apr 2023 17:24:10 +0000 https://www.cerebras.net/?p=104639 We introduce Cerebras-GPT, a family of open compute-optimal language models scaled from 111M to 13B parameters. We train Cerebras-GPT models on the Eleuther Pile dataset following DeepMind Chinchilla scaling rules for efficient pre-training (highest accuracy for a given compute budget).…

The post Cerebras-GPT: Open Compute-Optimal Language Models Trained on the Cerebras Wafer-Scale Cluster appeared first on Cerebras.

]]> Sparse-IFT: Sparse Iso-FLOP Transformations for Maximizing Training Efficiency https://arxiv.org/abs/2303.11525#new_tab Wed, 22 Mar 2023 16:53:24 +0000 https://www.cerebras.net/?p=104632 Replacing dense layers with Sparse-IFT leads to significant improvements across computer vision (CV) and natural language processing (NLP) tasks, including ResNet-18 on ImageNet (+3.5%) and GPT-3 Small on WikiText-103 (-0.4 PPL), both matching larger dense model variants with 2x or…

The post Sparse-IFT: Sparse Iso-FLOP Transformations for Maximizing Training Efficiency appeared first on Cerebras.

]]> SPDF: Sparse Pre-training and Dense Fine-tuning for Large Language Models https://arxiv.org/abs/2303.10464#new_tab Tue, 21 Mar 2023 16:41:45 +0000 https://www.cerebras.net/?p=104624 Presented at the ICLR 2023 Workshop on Sparsity in Neural Networks.
In this work, we show the benefits of using unstructured weight sparsity to train only a subset of weights during pre-training (Sparse Pre-training) and then recover the representational capacity…

The post SPDF: Sparse Pre-training and Dense Fine-tuning for Large Language Models appeared first on Cerebras.

]]> Wafer-Scale Fast Fourier Transforms https://arxiv.org/pdf/2209.15040.pdf#new_tab Fri, 20 Jan 2023 19:33:42 +0000 https://www.cerebras.net/?p=104308 We have implemented fast Fourier transforms for one, two, and three-dimensional arrays on the Cerebras CS-2, a system whose memory and processing elements reside on a single silicon wafer. The wafer-scale engine (WSE) encompasses a two-dimensional mesh of roughly 850,000…

The post Wafer-Scale Fast Fourier Transforms appeared first on Cerebras.

]]> GenSLMs: Genome-scale language models reveal SARS-CoV-2 evolutionary dynamics https://www.biorxiv.org/content/10.1101/2022.10.10.511571v2#new_tab Thu, 24 Nov 2022 04:58:55 +0000 https://www.cerebras.net/?p=104094 Our work seeks to transform how new and emergent variants of pandemic causing viruses, specially SARS-CoV-2, are identified and classified. By adapting large language models (LLMs) for genomic data, we build genome-scale language models (GenSLMs) which can learn the evolutionary…

The post GenSLMs: Genome-scale language models reveal SARS-CoV-2 evolutionary dynamics appeared first on Cerebras.

]]> TensorFlow as a DSL for stencil-based computation on the Cerebras Wafer-Scale Engine https://arxiv.org/abs/2210.04795#new_tab Fri, 26 Aug 2022 17:10:18 +0000 https://www.cerebras.net/?p=104540 The Cerebras Wafer Scale Engine (WSE) is an accelerator that combines hundreds of thousands of AI-cores onto a single chip. Whilst this technology has been designed for machine learning workloads, the significant amount of available raw compute means that it…

The post TensorFlow as a DSL for stencil-based computation on the Cerebras Wafer-Scale Engine appeared first on Cerebras.

]]> Cerebras Goes Big at Supercomputing 2021 https://www.cerebras.net/blog/sc21 Tue, 16 Nov 2021 14:00:26 +0000 https://cerebras.net/?p=1846 Rebecca Lewington, Technology Evangelist | November 16, 2021
There’s a lot to do at the Cerebras Virtual Experience
The Supercomputing Conference is where the High-Performance Computing (HPC) world comes together to show off the latest advances in high performance computing,…

The post Cerebras Goes Big at Supercomputing 2021 appeared first on Cerebras.

]]> Scaling and Operationalizing AI in Government https://www.cerebras.net/blog/scaling-and-operationalizing-ai-in-government/ Mon, 18 Oct 2021 18:30:48 +0000 https://cerebras.net/?p=1725 Gil Haberman, Senior Director of Product Marketing
Scaling and Operationalizing AI in Government  
Today at AI World Government 2021, Andy Hock our VP of Product, shared our vision of how Cerebras Systems can empower government organizations to harness the power…

The post Scaling and Operationalizing AI in Government appeared first on Cerebras.

]]> Announcing Cerebras Cloud @ Cirrascale, Democratizing High-Performance AI Compute https://www.cerebras.net/blog/announcing-cerebras-cloud-cirrascale-democratizing-high-performance-ai-compute/ Thu, 16 Sep 2021 06:00:39 +0000 https://cerebras.net/?p=1641 Gil Haberman, Sr. Director of Product Marketing | September 16, 2021
Democratizing High-Performance AI Compute  
Today, we are thrilled to announce  the availability of Cerebras Cloud @ Cirrascale, delivering the world’s fastest AI accelerator as a cloud service! Nearly…

The post Announcing Cerebras Cloud @ Cirrascale, Democratizing High-Performance AI Compute appeared first on Cerebras.

]]> Scaling Up and Out: Training Massive Models on Cerebras Systems using Weight Streaming https://www.cerebras.net/blog/scaling-up-and-out-training-massive-models-on-cerebras-systems-using-weight-streaming/ Tue, 14 Sep 2021 22:45:33 +0000 https://cerebras.net/?p=1648 …

The post Scaling Up and Out: Training Massive Models on Cerebras Systems using Weight Streaming appeared first on Cerebras.

]]> Announcing the Cerebras Architecture for Extreme-Scale AI https://www.cerebras.net/blog/announcing-the-cerebras-architecture-for-extreme-scale-ai/ https://www.cerebras.net/blog/announcing-the-cerebras-architecture-for-extreme-scale-ai/#comments Tue, 24 Aug 2021 12:01:15 +0000 https://cerebras.net/?p=1586 Sean Lie, Co-Founder and Chief Hardware Architect | August 24, 2021
Today at the Hot Chips conference, we proudly unveiled the world’s first multi-million core AI cluster architecture! Our unique technology handles neural networks with up to an astonishing 120…

The post Announcing the Cerebras Architecture for Extreme-Scale AI appeared first on Cerebras.

]]> https://www.cerebras.net/blog/announcing-the-cerebras-architecture-for-extreme-scale-ai/feed/ 2 An AI Chip With Unprecedented Performance To Do the Unimaginable https://www.cerebras.net/blog/an-ai-chip-with-unprecedented-performance-to-do-the-unimaginable/ Tue, 17 Aug 2021 03:36:14 +0000 https://cerebras.net/?p=1565 Dhiraj Mallick,  VP Engineering & Business Development | August 16, 2021
AI accelerator chips have made machine learning a reality in nearly every industry. With unprecedented pace of compute demand, model size and data growth, the need for high performance…

The post An AI Chip With Unprecedented Performance To Do the Unimaginable appeared first on Cerebras.

]]> Innovative AI Projects land in Edinburgh https://www.cerebras.net/blog/innovative-ai-projects-land-in-edinburgh/ Thu, 17 Jun 2021 23:39:09 +0000 https://cerebras.net/?p=1284 Rupal Hollenbeck, Formerly CMO at Cerebras Systems | June 17, 2021
Data scientists and AI researchers from the private and public sector gathered June 10th virtually at the EPCC to learn more about The Edinburgh International Data Facility, and specifically…

The post Innovative AI Projects land in Edinburgh appeared first on Cerebras.

]]>