magnitudedev/magnitude
Open source inference engine that tunes its kernels on your hardware to find the fastest way to run each model. 2x faster than Ollama. Works on Apple Silicon, NVIDIA, AMD, or just a CPU.
License & maintenance
Apache-2.0 is a permissive license: free to use commercially with attribution.
Last code push today.