Sealbook
Sealbook / Open source / Agentes de IA y apps con LLM

syv-ai/HyperQwen

Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.

consumer-gpucudainference-optimizationkv-cachellm-inferencelocal-llmquantizationqwenqwen3rtx-3090
GitHub ↗

Licencia y mantenimiento

Apache-2.0 es una licencia permisiva: uso comercial libre con atribución.

Último cambio de código hace 2 días.

Proyectos similares

RepositorioEstrellasLenguajeLicenciaActividad
1vllm-project/vllmA high-throughput and memory-efficient inference and serving engine for LLMs★ 94.1kPythonApache-2.0Activo
2nomic-ai/gpt4allGPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.★ 77.3kC++MITSin cambios
3openinterpreter/openinterpreterA coding agent for open models like Kimi K3 and GLM 5.3★ 68.6kRustApache-2.0Activo
4HKUDS/nanobotUltra-lightweight, open-source, self-hosted personal AI agent framework in Python with WebUI, tools, memory, MCP, multi-agent workflows, aut★ 49.1kPythonMITActivo
5ray-project/rayRay is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.★ 44.1kPythonApache-2.0Activo
6sgl-project/sglangSGLang is a high-performance serving framework for large language models and multimodal models.★ 37.3kPythonApache-2.0Activo
7labring/FastGPTFastGPT is a knowledge-based platform built on the LLMs, offers a comprehensive suite of out-of-the-box capabilities such as data processing★ 29.9kTypeScriptOtherActivo
8QwenLM/qwen-codeAn open-source AI coding agent that lives in your terminal.★ 28.6kTypeScriptApache-2.0Activo