Build a breakthrough AI platform beyond the constraints of the GPU

We believe the design space for AI models is far richer and larger than what’s possible on the GPU. That’s why we built the CS-2, an AI accelerator that’s 100x more powerful than the fastest GPU. Each CS-2 has access to 12 terabytes of memory, making it possible to train the largest models without modifications. With 40GB of on-chip memory for activations, we can train at 50k sequence length, more than 20x longer than vanilla GPU implementations. As a result, training GPT-3-size models on Cerebras hardware is as simple and fast as training GPT-2 on a GPU.

In addition to blazing fast performance, Cerebras hardware has a secret weapon – native support for dynamic and unstructured sparsity. Sparse models can dramatically improve efficiency and reduce up to 90% of training and inference FLOPs.

When you join Cerebras, you won’t be repeating the work of others, you will be pushing the frontier of what’s possible for the entire AI industry.

We are leaders in open-source release and publishing

At Cerebras, we are big believers in open-sourcing our work and publishing at major conferences. Along with EleutherAI, we pioneered the norm of releasing state-of-the-art LLMs using the permissive Apache 2.0 license. We trained and released some of the most popular models and datasets on HuggingFace, where they have already been downloaded millions of times. Notably, these include:

    • Cerebras-GPT: the first family of open-source GPT models, released under the Apache 2.0 license.
    • BTLM-3B-8K: the world’s best 3B model on release. Rivals 7B models in accuracy, fits on a Raspberry Pi.
    • SlimPajama: our first open-source data effort, a 627B token dataset that performs on par with the 1.2T token RedPajama. We released both the model and pre-processing code.
    • Jais 13B and 30B: along with our partner G42 we built and released Jais, a bilingual Arabic-English LLM. Jais is state-of-the art in Arabic and rivals Llama2 in English performance.

On the research side , our work has been recognized by some of the most prestigious organizations in the world. In 2022 we were awarded the ACM Gordon Bell Special Prize for developing a generative model to predict Covid mutations. In 2023 we were named a finalist for the Gordon Bell Prize for achieving breakthrough speedups in seismic processing. Cerebras’s  chip architecture is also leading the field, as evidenced by having the most open-source AI paper citations of any startup, according to the State of AI Report for 2023. See our full list of published work here.

Work on one of the fastest AI supercomputers in the world

Every AI company complains about being GPU poor. Not Cerebras. At Cerebras, we didn’t stop after building the world’s largest AI chip; instead, using our Wafer Scale Cluster architecture, we connected those chips  and built one of the world’s fastest AI supercomputers – the Condor Galaxy 1. With 4 exaflops of AI compute and 82 terabytes of memory, there is practically no limit to the kind of models you can build and train on CG-1. And we are not stopping there, either:  in 2024, together with our partner G42, we plan to expand our supercomputing footprint to 36 exaflops. Working at Cerebras, you will have access to a scale of compute that’s hard to imagine at most companies. 

Job stability with startup vitality

We realize there’s no shortage of opportunities for ML talent. Large companies offer attractive pay and job security but limited scope for growth. Early-stage companies offer more room for personal and financial growth, but the odds of success and survival can make it feel like a lottery.

At Cerebras, you won’t have to pick one or the other. As a mid-stage startup, we have plenty of runway and numerous repeat customers, allowing us to operate in a fast-changing environment. We also continue to grow quickly – our revenue is doubling year on year, and we recently reached cash flow break-even. This gives us the flexibility to grow on our own terms without constant VC funding and dilution.

A Simple and Non-Corporate Work Culture

At Cerebras we are extremely passionate about our work. But we also respect that you have your own interests and beliefs, and recognize that a workplace should not be all-consuming. Our work culture is simple and decidedly un-cultish. We don’t have vacuous mission statements, or a dozen core values for you to memorize. We keep work and activism separate – you will not be roped into “the current thing” or be pressured to say things you are not comfortable with. We trust that you are a person of high integrity and that you will do what’s best for yourself and the team.

When we are not working, we still have plenty of fun. Our CEO Andrew doubles as our BBQ pitmaster. Our Halloween party is an annual hit with staff and kids alike. And our climbing team is always on the lookout for new members.

Join Us

There has never been a more exciting time in AI. Cerebras is unique among AI hardware companies in that we are building not just a GPU alternative but a truly breakthrough architecture that is unlocking new opportunities for the industry. If you’re interested in joining us, we’d love to hear from you!

2023