Sealbook / Open source / AI agents & LLM apps

syv-ai/HyperQwen

Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.

consumer-gpucudainference-optimizationkv-cachellm-inferencelocal-llmquantizationqwenqwen3rtx-3090

GitHub ↗

License & maintenance

Apache-2.0 is a permissive license: free to use commercially with attribution.

Last code push 2 days ago.

Similar projects

	Repository	Stars	Language	License	Activity
1	vllm-project/vllmA high-throughput and memory-efficient inference and serving engine for LLMs	★ 94.1k	Python	Apache-2.0	Active
2	nomic-ai/gpt4allGPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.	★ 77.3k	C++	MIT	Stale
3	openinterpreter/openinterpreterA coding agent for open models like Kimi K3 and GLM 5.3	★ 68.6k	Rust	Apache-2.0	Active
4	HKUDS/nanobotUltra-lightweight, open-source, self-hosted personal AI agent framework in Python with WebUI, tools, memory, MCP, multi-agent workflows, aut	★ 49.1k	Python	MIT	Active
5	ray-project/rayRay is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.	★ 44.1k	Python	Apache-2.0	Active
6	sgl-project/sglangSGLang is a high-performance serving framework for large language models and multimodal models.	★ 37.3k	Python	Apache-2.0	Active
7	labring/FastGPTFastGPT is a knowledge-based platform built on the LLMs, offers a comprehensive suite of out-of-the-box capabilities such as data processing	★ 29.9k	TypeScript	Other	Active
8	QwenLM/qwen-codeAn open-source AI coding agent that lives in your terminal.	★ 28.6k	TypeScript	Apache-2.0	Active