This will make language models generate text much faster. Normally, a model generates one token at a time. But with this tool, a smaller helper model guesses several upcoming tokens, and the main model checks those guesses together. When the guesses are right, you get several tokens in one pass. They made the code available, and you can run this with vLLM and SGLang.