SSealbook

jmaczan/tiny-vllm

Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM

aiattentionbatchingcoursecppcudahpcinferencellmllm-inference

License & maintenance

Apache-2.0 is a permissive license: free to use commercially with attribution.

Last code push 25 days ago.