I think faster inference is one of the next big unlocks for

Press Space to continue

I think faster inference is one of the next big unlocks for coding agents. Volantis is using optics to give each chip far more memory and much higher memory bandwidth. They're targeting up to 10,000 tokens per second per user on models over 10T parameters. That's crazy! At that speed, a coding agent that takes hours today could finish in minutes. Definitely one of the more exciting raises I have seen recently.

semiDL@semiDL1d ago

Excited to announce Volantis's $88M Series A. We are solving Al's memory bottleneck by using optics, enabling chips with huge amounts of fast & cheap memory. By boosting both the memory bandwidth and capacity per chip by orders of magnitude, we enable ultra-fast inference (up to 10,000 tps/user) for large models (>10T) - with low $/tok to boot. Initially, this will enable insanely fast agents - think coding agents that finish in minutes or even seconds instead of hours. More excitingly, optics is a fundamentally scalable way to increase memory systems. Not 2X/year, but by orders of magnitude across new generations. This will enable a structurally new Al industry, including restarting scaling laws, holding entire repos in context windows & more. Our team has pioneered many core semiconductor technologies: the 1st CoWoS product, early HBM, the 1st silicon photonics CPO systems, the 1st high volume tunable VCSELs, the 1st processors to directly communicate using light & more. We’ve already sent data >10× farther than equally tiny electrical wires inside a chip package. Our next iteration is already taped out and targets world-record bandwidth density over relevant distances, read more: https://t.co/ljYTimdcbJ