Context length stagnation despite model improvements
Press Space for next Tweet
It's interesting how, for all of the huge model improvements we've seen over the past two year, the one thing that hasn't improved much at all is context length We've been stuck in the 200,000 up to 1m range for quite a long time now
924
112
35
96
Topics
Read the stories that matter.The stories and ideas that actually matter.
Save hours a day in 5 minutesTurn hours of scrolling into a five minute read.