Press Space to continue
Extropic's Z1 architecture demonstrates dramatic training
Press Space to continue
Extropic founder @beffjezos reveals how quickly Extropic's Z1 architecture is scaling from GPT-2 toward GPT-3-level models: "Right now it's very simple. Barely GPT-2 level. We, for fun, decided to start a new training run last night, and we'll see if we can get to GPT-3 this weekend, just as a fun bit." "The thing we showed with our scaling law is that if you train these on GPUs, you're paying like 10 or 15x more flops for training to get the same performance. To get the same performance you…
extropic@extropicSep 4
Introducing Z1T: Our first family of transformer-like models made for sparse probabilistic hardware like Z1 Achieving up to 140x energy efficiency gains over GPUs and revealing a new scaling law for sparse transformers Read the blog: https://t.co/zYLkLjrJLL https://t.co/4mepJ441Ji
View