Cerebras Slays GPUs, Breaks Record for Largest AI Models Trained on a Single Device

Cerebras, the company behind the world’s largest accelerator chip in existence, the CS-2 Wafer Scale Engine , has just announced a milestone: the training of the world’s largest NLP (Natural Language Processing) AI model in a single device. While that in itself could mean many things (it wouldn’t be much of a record to break if the previous largest model was trained in a smartwatch, for instance), the AI model trained by Cerebras ascended towards a staggering – and unprecedented – 20 billion parameters. All without the workload having to be scaled across multiple accelerators. That’s enough to fit the internet’s latest sensation, the image-from-text-generator, OpenAI’s 12-billion parameter DALL-E (opens in new tab) . The most important bit in Cerebras’ achievement is the reduction in infrastructure and software complexity requirements. Granted, a single CS-2 system is akin to a supercomputer all on its own. The Wafer Scale Engine-2 – which, like the name implies, is etched in a single, 7 nm wafer, usually enough for hundreds of mainstream chips – features a staggering 2.6 trillion 7 nm transistors, 850,000 cores, and 40 GB of integrated cache in a package consuming around 15kW. Keeping up to 20 billion-parameter NLP models in a… Read full this story

Cerebras Slays GPUs, Breaks Record for Largest AI Models Trained on a Single Device have 299 words, post on www.tomshardware.com at June 22, 2022. This is cached page on X-Buy. If you want remove this page, please contact us.