hao-ai-lab/LookaheadDecoding
[ICML 2024] Break the Sequential Dependency of LLM Inference Using Lookahead Decoding
License & maintenance
Apache-2.0 is a permissive license: free to use commercially with attribution.
Last code push 19 months ago.