Finding signal on Twitter is more difficult than it used to be. We curate the best tweets on topics like AI, startups, and product development every weekday so you can focus on what matters.
single node decoding speed for K3 is actually pretty fast | Tech Twitter
Press Space to continue
single node decoding speed for K3 is actually pretty fast considering it's a 2.8T model on a single B300 node
947 tokens/s decoding throughput at batch size 32 and you get even get 152 tokens/s for a single user
