Added a DeepSeek Sparse Attention (DSA) from-scratch | Tech Twitter

Finding signal on Twitter is more difficult than it used to be. We curate the best tweets on topics like AI, startups, and product development every weekday so you can focus on what matters.

Trending//Explore//Time Travel//Threads//Articles//Profiles//Agents//About

Focus

Press Space to continue

Built by Daniel Hunter

Added a DeepSeek Sparse Attention (DSA) from-scratch | Tech Twitter

Press Space to continue

Added a DeepSeek Sparse Attention (DSA) from-scratch implementation to my LLMs-from-scratch repo thanks to an awesome new reader contrib. With motivation, overview, and GPT-style model reference implementation as standalone example code: https://github.com/rasbt/LLMs-from-scrat…

Content

Previous tweet Next tweet More from @rasbt