There is a moment most Nigerian businesses hit a few weeks after they first get a local language model running on a single workstation. The demo works beautifully for one person. Then five people try it at once, and the whole thing crawls. That is the gap between running a model and serving one, and it is exactly the gap vLLM was built to close.
vLLM is a high-throughput serving engine. It is not really for single-user chat on your laptop — for that, the lighter, more flexible llama.cpp approach is often the better fit. vLLM is for when you have an application, a team, or customers, and many requests arriving at the same time. Choosing hardware for it means thinking about concurrency from the very first decision, and it leans heavily on the same VRAM realities we cover in our GPU VRAM guide.
What vLLM Actually Does Differently
If you only remember one technical idea from this article, make it this one: vLLM's headline feature is PagedAttention. When a model generates text, it keeps a running memory of the conversation so far — the KV cache. In naive serving, this cache is allocated in big rigid blocks, and a great deal of GPU memory is wasted on padding and fragmentation. PagedAttention manages the KV cache in small pages, much like an operating system manages virtual memory. The result is that far less GPU memory is wasted, so a single card can hold many more concurrent requests.
The second idea is continuous batching. Instead of waiting for a batch of requests to all finish before starting the next, vLLM slots new requests in as soon as space frees up, keeping the GPU busy almost all the time. Together these two features are why vLLM can serve several or many users from one card where a simpler setup would choke after two or three.
The practical takeaway for hardware: vLLM is GPU-centric and NVIDIA-first. It is built around CUDA, the GPU and its VRAM dominate every sizing decision, and your CPU and system RAM are very much the supporting cast. Spend your money on the graphics card.
The Two Halves of Your VRAM Budget
This is where most people get their hardware wrong, so it is worth being precise. The VRAM a serving box needs is made of two separate things, and only one of them is obvious.
- Model weights — roughly the parameter count multiplied by the bytes per parameter. At FP16, that is 2 bytes per parameter, so a 7-billion-parameter model needs about 14GB, and a 13-billion-parameter model about 26GB. Quantisation shrinks this considerably, but it is the easy part to plan for.
- The KV cache — the memory that holds the running context for every active request. This grows with the context length and, critically, with the number of concurrent users. This is the half people forget.
That second item is why a card which comfortably "fits" a model can still run out of memory the moment real traffic arrives. The model weights are fixed, but the KV cache balloons as more people connect and as conversations get longer. More VRAM does not just let you load a bigger model — it directly buys you more concurrent users and longer context windows. When you size a serving GPU, you are really sizing the headroom left over after the weights, because that headroom is your capacity.
A Practical VRAM-Sizing Framework
Rather than chase exact figures, think in tiers. These are deliberately rough, leaving generous room for the KV cache on top of the weights.
- Single 24GB card (used RTX 3090 or RTX 4090) — comfortably serves small-to-mid models, say 7B to 13B class at FP16 or larger models when quantised, to several concurrent users. This is the genuine sweet spot for most Nigerian businesses starting out.
- Dual 24GB cards (48GB total) — lets you run larger models, or run a mid-size model with far more KV cache headroom for higher throughput and more simultaneous users.
- Three or four cards — for serving large models at scale, or running several models side by side. This is where you start thinking about it as proper infrastructure rather than a workstation.
The honest planning rule: take the model-weight figure, then leave at least as much again free for the cache and overhead if you expect real concurrency. A 13B model that needs 26GB for weights alone is not a comfortable fit on a 24GB card under multi-user load, even though the arithmetic looks close. For a deeper look at why memory speed matters as much as memory size here, see our piece on GPU memory bandwidth.
When One Card Is Not Enough: Tensor Parallelism
vLLM has a clean answer for models too big for a single GPU. With tensor parallelism, it splits a single model across two or four cards, so each holds a slice of the weights and they work on every request together. A model that would never fit on one 24GB card can run perfectly well spread across two.
This splitting means the cards talk to each other constantly, so a good GPU-to-GPU interconnect helps throughput. The good news for Nigerian builders is that plain PCIe works for this — you do not strictly need exotic interconnects to get going, though they help at larger scales. The trade-offs of moving from one card to several are worth understanding before you commit, and we walk through them in our guide to scaling from single to multi-GPU. If you are building the machine yourself, the dual-GPU rig build walkthrough covers the physical and BIOS details.
The Rest of the Box
The GPU dominates, but a serving machine still needs a sensible foundation around it. Skimp here and you bottleneck the expensive part.
- Fast NVMe storage — model weight files are large, tens of gigabytes each. A quick NVMe drive means the model loads into VRAM in seconds rather than minutes, which matters every time you restart the service or swap models.
- Enough system RAM — a sensible rule of thumb is to have at least as much system RAM as total VRAM, comfortably more if you can. It is used while loading and staging models even though inference itself lives on the GPU.
- A capable CPU and PCIe lanes — not a bottleneck for inference itself, but multi-GPU setups need enough lanes to feed every card, so check the motherboard and CPU support the configuration you are planning.
Power, NEPA and Running 24/7
Here is the part the hardware guides written abroad never mention. A serving box is meant to stay up. Unlike a chat tool you open when you need it, an inference server backing an application ideally runs 24/7, ready for requests at any hour. In Nigeria that collides directly with the realities of NEPA.
A single 24GB card under sustained load can pull 300 to 450 watts on its own, and the whole machine more. Multiply that across a dual or quad setup and you are running a serious, constant power draw. Plan for it deliberately:
- A proper UPS sized for the full machine, so a flicker from the grid does not corrupt a model mid-load or crash live requests.
- Stable, conditioned power — voltage swings are hard on power supplies running near capacity for hours on end.
- Honest running-cost maths — a box drawing several hundred watts around the clock on generator or inverter is a real monthly line item, not an afterthought. Factor it into whether self-hosting beats an API for your volume.
Rough Naira Tiers
Prices for used and imported GPUs in Nigeria move constantly, so treat these as shape rather than exact quotes. The point is the relative jumps between tiers.
- Hobby and single-user serving — a single used 24GB card in an otherwise modest build. Often in the low single-digit millions of Naira all-in. Enough to serve a small model to a handful of users and learn how vLLM behaves under your real workload.
- Small-business multi-user serving — a single strong 24GB card or a dual-card setup, with the better CPU, RAM, NVMe and UPS that a 24/7 box deserves. A meaningful step up in total cost, justified once the system is doing real work for staff or customers.
- Departmental infrastructure — three or four cards, server-grade foundations and serious power provisioning. This is a capital project, sized against the API bills it replaces.
Before fixating on tokens per second, read our honest look at what tokens-per-second numbers really mean — single-user speed and multi-user throughput are different measurements, and vLLM is optimised for the second one. For the broader business framing of all this, our overview of an AI inference server for business applications ties the hardware to the use cases.
Frequently Asked Questions
Can I run vLLM on an AMD GPU? vLLM is NVIDIA-first and built around CUDA, and that is where it is most mature and best supported. There is growing support for other accelerators, but if your goal is a dependable serving box today, an NVIDIA card is by far the safest choice. If you are weighing brands, our GPU brand comparison for 2026 goes deeper.
Why not just use llama.cpp instead? They solve different problems. llama.cpp is excellent for single-user use, runs on modest and even CPU-only hardware, and is wonderfully flexible. vLLM is the tool when you need to serve many requests at once with high efficiency. If you only ever have one person using the model at a time, llama.cpp is likely the simpler answer.
How many users can one 24GB card really serve? Honestly, it depends — on the model size, the context length and how heavy each request is. A small quantised model with short prompts will serve far more concurrent users than a large model handling long documents. Rather than trust a headline number, test with your own workload, which is exactly why starting on a single card before scaling is wise.
The One Thing to Remember
If you take away nothing else, remember that serving is about the KV cache, not just the weights. Anyone can tell you a 7B model needs about 14GB of VRAM. The skill is buying enough headroom on top of that for the concurrent users and context lengths your application will actually see — because that headroom, managed by PagedAttention, is what turns a single GPU into a server that quietly handles a whole team at once.
Not sure which tier fits your workload? Use our configurator to spec a vLLM-ready serving build sized for your concurrency and budget, or contact us to talk through your models, your expected users and the power realities of running it 24/7 in Nigeria.