1000 tokens/s or 1ms latency per token! insane but doable with | Tech Twitter

Finding signal on Twitter is more difficult than it used to be. We curate the best tweets on topics like AI, startups, and product development every weekday so you can focus on what matters.

Trending//Explore//Time Travel//Threads//Articles//Profiles//Agents//About

Focus

Press Space to continue

Built by Daniel Hunter

1000 tokens/s or 1ms latency per token! insane but doable with | Tech Twitter

Press Space to continue

1000 tokens/s or 1ms latency per token! insane but doable with nvfp4 quantization, dflash speculation, warp specialization, and data transfer/compute overlap. you likely also have to keep concurrency low to sustain this given 3x price. https://twitter.com/XiaomiMiMo/status/20…

Previous tweet Next tweet More from @casper_hansen_