You bought a capable NVIDIA GPU, you installed PyTorch, you wrote your first three lines of code to check that the GPU is visible — and it returns false. The framework simply cannot see the hardware sitting in your machine. If you have spent an afternoon staring at that exact problem, you are not alone, and you have almost certainly not broken anything. What you have hit is the single most common rite of passage in AI development: a version mismatch somewhere in a stack of software layers that all have to agree with one another.
This guide exists to save you that lost day. Once you understand how the pieces line up, the whole thing stops feeling like dark magic and becomes a short checklist. If you are still deciding which GPU to build around, our guides on NVIDIA vs AMD vs Intel GPUs and building an AI development workstation are worth reading alongside this one, because the GPU you pick decides which version battles you will even have to fight.
Why this is so painful for newcomers
The frustration comes from the fact that nothing throws a loud, obvious error. PyTorch installs cleanly. Python runs. The GPU is physically present and powering your displays. Everything looks fine. But under the surface, the framework was built expecting one version of NVIDIA's compute platform, and your machine is offering a different one — or no compatible one at all. The result is a quiet failure where the GPU is simply ignored and everything falls back to the CPU, running tens of times slower.
The cure is not to randomly reinstall things until something works, which is how most people lose their afternoon. The cure is to understand which layers exist and the one rule that governs how they fit together.
The stack, from top to bottom
Think of GPU-accelerated AI as a tower of four layers. Each has its own version number, and they must be mutually compatible. From the part you touch most down to the part closest to the metal:
- Your AI framework — PyTorch, TensorFlow or similar. This is what your code imports. Crucially, each framework build is compiled against a specific CUDA version.
- CUDA toolkit and runtime — NVIDIA's general-purpose GPU computing platform. This is what lets the framework run mathematical operations on the graphics card instead of the processor.
- cuDNN — NVIDIA's specialised deep-learning library, full of fast implementations of the operations neural networks rely on. Frameworks that use CUDA almost always need a matching cuDNN.
- The GPU driver — the lowest layer, the software that lets your operating system talk to the card at all. Everything above it depends on the driver being recent enough.
When people say "a CUDA problem", they usually mean that two of these four layers disagree about which version of CUDA is in play.
The mental model that fixes most of it
Here is the part that newcomers are rarely told plainly: you usually do not need to install and match all of this by hand. The old, painful way was to install a system-wide CUDA toolkit, install cuDNN separately, line up the versions yourself, and pray. The modern, sane way is to let the framework bring its own CUDA with it.
When you install PyTorch through its official installer using pip or conda, you choose a CUDA build — and that installer bundles the matching CUDA runtime and cuDNN automatically. You do not install them yourself. That removes two whole layers from your worry list in one move. What remains your responsibility is just the bottom layer: the GPU driver.
And the driver is the forgiving one. NVIDIA drivers are broadly backward-compatible, meaning a newer driver happily supports older CUDA versions. So the rule that solves the overwhelming majority of cases is short: keep your NVIDIA driver reasonably up to date, then install the PyTorch build for the CUDA version you actually want. Get those two things right and the GPU appears, no manual toolkit juggling required.
Where it still bites
The bundled-CUDA approach removes most pain, but a few situations still trip people up. Knowing them in advance is half the battle:
- A driver that is too old for the CUDA version your framework build expects. Backward compatibility runs one way — a new driver supports old CUDA, but an old driver cannot support a CUDA newer than itself. This is the classic culprit.
- Mixing a system-wide CUDA toolkit with the framework's bundled one. If you installed CUDA globally and also let PyTorch bring its own, the two can collide and confuse which libraries get loaded. Pick one approach per project.
- Software that must compile against a specific CUDA. Some high-performance inference engines and kernels — vLLM, flash-attention and the like — need a particular CUDA to build from source. Our guide on serving local LLMs with vLLM covers this territory in depth.
- Brand-new GPUs. A just-released card may need a newer driver and CUDA than your chosen framework has shipped support for yet. Here the fix is patience, or running a nightly framework build.
A practical checklist that saves the day
When the GPU refuses to show up, work through these in order rather than reinstalling at random:
- Use a virtual environment. Create a fresh conda environment or Python venv per project so one project's versions never poison another's. This single habit prevents most long-term mess.
- Check the framework's official install page. PyTorch's site gives you the exact pip or conda command for each CUDA build. Copy it; do not guess the command.
- Match the framework's CUDA build to what your code needs. If a library you depend on wants a particular CUDA, install the PyTorch build for that CUDA, not the newest available.
- Confirm your driver is current enough, but resist installing the absolute bleeding-edge release. Current and stable beats brand new and untested.
- Verify GPU visibility first, then run real work. Confirm the framework can see the card before you debug your training script — it separates a setup problem from a code problem.
Why the GPU brand decides your difficulty
Everything described so far lives inside NVIDIA's CUDA ecosystem, and there is a reason this guide is NVIDIA-shaped: CUDA is the path of least resistance for AI work. The frameworks, the tutorials and the prebuilt installers all assume it. AMD cards use a different stack called ROCm, which is improving but remains less mature and less universally supported, so you trade some compatibility for value. If you are weighing that trade, read our pieces on running AI on AMD GPUs with ROCm and how to choose a GPU in Nigeria before you commit, and how much VRAM you actually need to size the card itself.
The Nigerian angle: bandwidth is the real cost
There is a local dimension that makes getting this right even more valuable here than elsewhere. CUDA toolkits and framework wheels are not small files — they run to gigabytes each. On a metered or slow connection, a single wrong install can quietly burn a chunk of your data allowance and an hour of waiting, and reinstalling three times to find the right version turns one mistake into a real ₦ cost.
So plan your downloads. Decide on your target CUDA build before you start pulling files. Cache installers and wheels locally so a second project on the same machine does not re-download the same multi-gigabyte package. Getting the versions right the first time is not just a time saver here — it is a bandwidth saver, and on Nigerian connections that is money.
Frequently Asked Questions
Do I need to install the CUDA toolkit separately before installing PyTorch? In most cases, no. When you install PyTorch via its official pip or conda command for a chosen CUDA build, the matching CUDA runtime and cuDNN come bundled. You only need a recent-enough GPU driver. A separate system-wide toolkit is mainly for software you intend to compile from source yourself.
My driver is newer than the CUDA version PyTorch wants — is that a problem? Almost never. NVIDIA drivers are backward-compatible, so a newer driver supports older CUDA versions fine. The dangerous direction is the opposite: a driver that is too old for the CUDA your framework expects. When in doubt, update the driver, not downgrade it.
Why does my framework still run on the CPU even though it installed without errors? A clean install does not guarantee the GPU is being used. It usually means the framework build does not match your CUDA or driver situation, so it silently fell back to the processor. Verify GPU visibility explicitly, and if it returns false, recheck that you installed the GPU-enabled build matching your driver.
The One Thing to Remember
If you take away a single sentence, make it this: let the framework bring its own CUDA, keep your NVIDIA driver reasonably current, and isolate every project in its own virtual environment. That trio dissolves the vast majority of "PyTorch cannot find my GPU" misery, because you stop hand-matching layers that the installer was always willing to match for you.
Would you rather skip the version wars entirely and start with a machine configured to run AI work out of the box? Build your ideal setup with our configurator, or get in touch and we will spec a GPU and software stack matched to the frameworks you actually plan to use.