pytorch/ao

PyTorch native quantization for training and inference

brrrcudadtypesfloat8inferencellamamxpytorchquantizationsparsity

License & maintenance

Custom or modified license that GitHub does not recognise: read the LICENSE file in the repository before using it in a product.

Last code push 3 days ago.

	Repository	Stars	Language	License	Activity
1	vllm-project/vllmA high-throughput and memory-efficient inference and serving engine for LLMs	★ 94.1k	Python	Apache-2.0	Active
2	sgl-project/sglangSGLang is a high-performance serving framework for large language models and multimodal models.	★ 37.3k	Python	Apache-2.0	Active
3	SYSTRAN/faster-whisperFaster Whisper transcription with CTranslate2	★ 25.9k	Python	MIT	Active
4	ollama/ollamaGet up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.	★ 183.7k	Go	MIT	Active
5	unslothai/unslothLocal UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4, MiniMax-H3, Gemma 4, FLUX and more.	★ 78.2k	Python	Apache-2.0	Active
6	hiyouga/LlamaFactoryUnified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)	★ 75.6k	Python	Apache-2.0	Active
7	ggml-org/whisper.cppPort of OpenAI's Whisper model in C/C++	★ 54.5k	C++	MIT	Active
8	mudler/LocalAILocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.	★ 49.6k	Go	MIT	Active