# vllm-project/vllm

> High-throughput inference and serving engine for LLMs with an OpenAI-compatible API

Navigation: [Artificial Intelligence](category/artificial-intelligence.txt) · [Index](../llms.txt)

## Facts

- GitHub: https://github.com/vllm-project/vllm
- HTML: https://awesomeopensource.dev/project/vllm-project-vllm
- Website: https://vllm.ai
- License: Apache-2.0
- Language: Python
- stars: 96871
- Topics: amd, blackwell, cuda, deepseek, deepseek-v3, gpt, gpt-oss, inference, kimi, llama, llm, llm-serving, model-serving, moe, openai, pytorch, qwen, qwen3, tpu, transformer
- Activity score: 5.0/5 (Thriving)
- Last commit (any branch): 2026-11-13
- Last release: v0.33.0rc3 2026-11-02

## Description

A Python inference and serving engine for large language models on NVIDIA, AMD, Intel and CPU hardware. PagedAttention, continuous batching and prefix caching keep GPU memory tight while raising throughput, and an OpenAI-compatible server adds tensor parallelism, multi-LoRA and structured output. It loads 200+ Hugging Face architectures, supports FP8, INT4 and GGUF quantization, and runs from a wheel or container.

## Maintenance activity

- Recency: 100% (weight 40%) — last commit: 0 days ago
- Commit volume: 100% (weight 25%)
- Releases: 98% (weight 25%) — last release: 11 days ago
- Bug fixes: 100% (weight 10%)
- Commits: 5349 / 90d, 14088 / 365d

## Similar projects

- [openclaw/openclaw](openclaw-openclaw.md): Open-source AI assistant that lives on your own hardware and in your chat apps · Activity score 5.0/5 (Thriving)
- [n8n-io/n8n](n8n-io-n8n.md): Fair-code workflow automation with a visual canvas, custom code and 1500+ integrations · Activity score 5.0/5 (Thriving)
- [langgenius/dify](langgenius-dify.md): Visual platform for building LLM apps with RAG pipelines, agents and model integrations · Activity score 5.0/5 (Thriving)
- [open-webui/open-webui](open-webui-open-webui.md): Self-hosted web UI for chatting with Ollama and OpenAI-compatible models offline · Activity score 5.0/5 (Thriving)
- [langchain-ai/langchain](langchain-ai-langchain.md): Python framework for building agents and LLM applications from composable components · Activity score 5.0/5 (Thriving)
- [paperclipai/paperclip](paperclipai-paperclip.md): Manage a team of AI agents with goals, budgets and task tracking · Activity score 5.0/5 (Thriving)
