vLLM
High-throughput LLM inference server for GPUs with an OpenAI-compatible API.
Quick answer
vLLM is a high-throughput LLM inference server for GPUs with an OpenAI-compatible API, listed under Local model runtimes. A typical small-business use: serve one open model to the whole company from a single GPU server. It needs a GPU server; we do not estimate that cost because GPU prices vary too much by card and provider. License: Apache-2.0, which allows internal business use.
Last updated · facts verified against official docs on · sources
Activity data comes from GitHub and was last refreshed Oct 22, 2026.
What is vLLM?
vLLM is the inference engine many companies use to serve open models at scale. PagedAttention memory management and continuous batching let it serve many concurrent users per GPU. It exposes an OpenAI-compatible server and supports NVIDIA, AMD and Intel GPUs, CPUs and other accelerators.
Where it fits in a 10–200 person business
- Serve one open model to the whole company from a single GPU server
- Back-end for chat UIs and RAG apps when Ollama becomes a bottleneck under concurrency
These use cases are our editorial suggestions, not claims made by the project.
Can a business use vLLM? (license)
| License | Apache-2.0 |
|---|---|
| Type | Permissive: MIT / Apache-2.0: use, modify and sell with few obligations. |
| Internal business use | Allowed under the license text |
| What to watch | Permissive. Internal and commercial use allowed. Models keep their own licenses. |
This is a plain-English summary of the license text, not legal advice. Third-party models you run with it have their own licenses.
What hardware does vLLM need?
The project does not publish a minimum CPU/RAM figure, so we leave it blank rather than guess.
GPU: NVIDIA: compute capability 7.5 or higher (e.g. T4, RTX 20xx, A100, L4, H100, B200).
VRAM depends on the model and context length.
Source: docs.vllm.ai/en/latest/getting_started/installation/gpu/
How much does it cost to host vLLM per month?
Needs a GPU server for practical use. Not estimated, because GPU prices vary too much by card and provider.
How to deploy vLLM
Method: pip / uv (or Docker)
uv pip install vllm
vllm serve <model-id>From the official instructions: github.com/vllm-project/vllm#readme. Always check the source for the current version before you run it.
Known gotchas when self-hosting vLLM
- In Docker, run with --ipc=host (or a large --shm-size), because PyTorch uses shared memory between processes, especially for tensor-parallel inference. source
Alternatives in Local model runtimes
- Ollama The simplest way to download and run open-weight LLMs locally behind an API. Compare Ollama vs vLLM
- LocalAI OpenAI-compatible API for LLMs, vision, voice and images on any hardware. Compare LocalAI vs vLLM
vLLM FAQ
How much RAM does vLLM need?
vLLM does not publish an official RAM figure, so we do not state one. VRAM depends on the model and context length.
Does vLLM need a GPU?
For practical use, yes. NVIDIA: compute capability 7.5 or higher (e.g. T4, RTX 20xx, A100, L4, H100, B200).
How much does it cost to self-host vLLM?
The software is free to download. It needs a GPU server; we do not estimate that cost because GPU prices vary too much by card and provider.
Is vLLM free for commercial use?
vLLM is licensed under Apache-2.0 (Permissive). Internal business use is allowed under the license text. Permissive. Internal and commercial use allowed. Models keep their own licenses. This is a summary, not legal advice.
What are common problems when self-hosting vLLM?
In Docker, run with --ipc=host (or a large --shm-size), because PyTorch uses shared memory between processes, especially for tensor-parallel inference.
Is vLLM still maintained?
Yes, it is actively developed. Last commit: Oct 22, 2026. Latest release: v0.32.0 (Oct 18, 2026). Checked Oct 22, 2026.
How do I install vLLM?
The official method is pip / uv (or Docker). The exact commands are in the "How to deploy" section above and in the official instructions: https://github.com/vllm-project/vllm#readme
What are self-hosted alternatives to vLLM?
Other apps in Local model runtimes: Ollama, LocalAI.
Sources
- GitHub repository: github.com/vllm-project/vllm
- Official documentation, Hardware requirements, Gotcha 1: docs.vllm.ai/en/latest/getting_started/installation/gpu/
- License text: github.com/vllm-project/vllm/blob/main/LICENSE
- Deploy instructions: github.com/vllm-project/vllm#readme
- Hetzner Cloud pricing (cost estimate): www.hetzner.com/cloud/regular-performance
- GitHub stars, commits and releases: GitHub REST API, fetched
Facts on this page were checked against the project's own repository and docs on 2026-10-22. Spotted something out of date? Tell us.