Repository navigation
[BUG]: Program.compile("ptx") fails on machines without a CUDA driver, although NVRTC does not need one #3015
Description
Is this a duplicate?
- I confirmed there appear to be no duplicate issues for this bug and that I agree to the Code of Conduct
Type of Bug
Runtime Error
Component
cuda.core
Describe the bug
On a Linux machine with no NVIDIA driver installed (a GitHub Actions ubuntu runner), compiling a Program to PTX with the NVRTC backend raises DynamicLibNotFoundError for libcuda.so.1. Compiling the same source to cubin with an explicit arch works on the same machine, so NVRTC itself is available.
The failure comes from the loadability check that runs before PTX compilation. _can_load_generated_ptx() calls driver_version(), which calls cuDriverGetVersion and needs libcuda. The check only decides whether to emit a RuntimeWarning, but when the driver library is missing it raises instead, and the whole compile fails. The same check is on the cache hit path in Program.compile.
This matters for CI jobs that compile kernels to PTX without a GPU, for example to inspect the generated PTX in tests. My workaround was to call cuda.bindings.nvrtc directly with the flags from ProgramOptions.as_bytes("nvrtc", "ptx"), which works without a driver.
How to Reproduce
On a machine without an NVIDIA driver. I used the ubuntu latest runner on GitHub Actions with Python 3.12.
pip install "cuda-core[cu12]==1.2.1"
from cuda.core import Program, ProgramOptions
src = 'extern "C" __global__ void k(float* x) { x[0] = 2.0f * x[0] + 1.0f; }'
# Works without a driver
Program(src, code_type="c++", options=ProgramOptions(arch="sm_75")).compile("cubin")
# Fails without a driver
Program(src, code_type="c++", options=ProgramOptions(arch="compute_75")).compile("ptx")A public run on a runner with no libcuda shows both cases, the cubin compile passes and the ptx compile fails. https://github.com/VolodymyrLinuxovich/heat-risk-cuda/actions/runs/37217831804
Trimmed traceback from the failing run
cuda/core/_program.pyx:222: in cuda.core._program.Program.compile
cuda/core/_program.pyx:738: in cuda.core._program._program_compile_uncached
cuda/core/_program.pyx:1192: in cuda.core._program.Program_compile
cuda/core/_program.pyx:815: in cuda.core._program._can_load_generated_ptx
cuda/core/_utils/version.pyx:37: in cuda.core._utils.version.driver_version
cuda/bindings/driver.pyx:23154: in cuda.bindings.driver.cuDriverGetVersion
...
cuda.pathfinder._dynamic_libs.load_dl_common.DynamicLibNotFoundError: "cuda" is an NVIDIA driver library and can only be found via system search. Ensure the NVIDIA display driver is installed.
The _can_load_generated_ptx code on main looks the same as in 1.2.1, so I expect main behaves the same way, but I have only run 1.2.1.
Expected behavior
PTX compilation succeeds without a driver, the same as cubin compilation. When the driver version cannot be determined, the loadability check could skip the warning (or warn that loadability is unknown) instead of failing the compile.
I would be glad to open a PR for this if that approach sounds right to you.
Operating System
Ubuntu (GitHub Actions ubuntu latest runner), no NVIDIA driver
nvidia-smi output
Not available, no driver on that machine.
Activity
efegokdemir commentedon Oct 4, 2026
I’d like to work on this issue. Could a maintainer assign it to me before I start, per the contribution guide?
VolodymyrLinuxovich commentedon Oct 4, 2026
Thanks for the interest. I offered a fix in the report and would be glad to open the PR. I hit this in my own CI, so I can verify the fix there. Happy to leave it to the maintainers to decide.
leofang commentedon Oct 5, 2026
This is not a bug. NVRTC happens to work today without driver installed, but it is not a documented behavior that the NVRTC team guarantees. Also,
cuda.corecould choose to fall back to the driver for JIT-compiling PTX, and in order for us to decide if fallback is needed, we need the driver to be present in the first place.In general, ALL
cuda.corefunctionalities require a working NVIDIA GPU, necessary CUDA Toolkit libraries, and driver properly installed. The only guarantee we offer on CPU-only machines is thatfrom cuda import coreworks.For the specific use case, I recommend all JIT-compilation users to generate CUBIN instead of PTX. It'd offer you better support for CUDA Minor Version Compatibility.
VolodymyrLinuxovich commentedon Oct 5, 2026
Thanks for explaining. That makes sense, and I switched my CI to compile cubin instead.