Resolving AssertionError: Torch Not Compiled With CUDA Enabled In Python 3

The “AssertionError: torch not compiled with CUDA enabled” can be a significant hurdle when working with PyTorch, a renowned open-source machine learning library known for its proficiency in training deep neural networks. It relies on CUDA, a parallel computing platform by NVIDIA, for efficient GPU acceleration.

In this article, we’ll explore the origins of this error, present illustrative examples, and provide a systematic approach to rectify it.

What is AssertionError: torch not compiled with CUDA enabled?

The error message “AssertionError: torch not compiled with CUDA enabled” serves as a clear indicator that the existing PyTorch installation is devoid of the necessary framework for CUDA support.

This essentially means that the PyTorch library has been set up in a way that it lacks the ability to offload certain computations to the GPU, which is a significant aspect of accelerating the training and inference processes of deep neural networks.

What Causes of the AssertionError: torch not compiled with CUDA?

The “AssertionError: torch not compiled with CUDA enabled” can be caused due to any of the reasons mentioned below.

Incorrect Installation:

This is one of the most common reasons for encountering the error. When PyTorch is initially installed, users must specify whether or not they want CUDA support. If CUDA support is not selected during installation, PyTorch will default to a CPU-only build.

Missing GPU Drivers:

Even if PyTorch is installed with CUDA support, it relies on NVIDIA GPU drivers to function properly. If these drivers are missing, outdated, or incompatible, it can lead to the assertion error.

Incompatible PyTorch and CUDA Versions:

Using incompatible versions of PyTorch and CUDA can also trigger this error. It’s important to ensure that the PyTorch version you’re using is compatible with the CUDA version installed on your system.

Below are few examples mentioned to explain the AssertionError: torch not compiled with CUDA more properly.

Example 1: Basic Assertion Error

In this example, a Python script (likely named example.py) is being executed. At line 5, a tensor is being created using torch.tensor([1.0, 2.0]), and then an attempt is made to move it to the GPU using .cuda().

However, an ‘AssertionError: torch not compiled with CUDA enabled’ is raised, stating that PyTorch was not compiled with CUDA support.

Traceback (most recent call last):
File "example.py", line 5, in <module>
x = torch.tensor([1.0, 2.0]).cuda()

This error indicates that the PyTorch installation being used lacks the necessary components for CUDA support. As a result, it’s unable to utilize the GPU for accelerated computations.

Example 2: Importing torch.cuda

In this example, the script starts by trying to import torch.cuda. However, it encounters an AssertionError, stating that PyTorch was not compiled with CUDA support.

Syntax:

Traceback (most recent call last):
File "example.py", line 2, in <module>
import torch.cuda

This error is caused when the code attempts to directly access torch.cuda, indicating that the PyTorch installation lacks CUDA support.

How to resolve such error?

The AssertionError: torch not compiled with CUDA more properly. can be resolved by using one of the

Verifying GPU and Drivers

Begin by confirming that your system has an NVIDIA GPU installed and check that the necessary drivers are properly installed. You can do this by running the command nvidia-smi in your terminal.

Reinstall PyTorch with CUDA Support 

To rectify this issue, you’ll need to reinstall PyTorch with CUDA enabled. Use the following pip command, replacing the version numbers with the appropriate ones for your system:

pip install torch==1.9.0+cu111 torchvision==0.10.0+cu111 torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html

This will ensure that you have a PyTorch installation with CUDA support.

Verify Installation 

After reinstalling PyTorch, you can verify if CUDA is working correctly by running a short Python script:

import torch

print(torch.cuda.is_available())

print(torch.cuda.device_count())

If CUDA is set up correctly, the script will print True for torch.cuda.is_available() and indicate the number of available GPUs.

Set the Device 

In your Python code, it’s essential to specify the device (CPU or GPU) on which tensors and models should operate. This can be done as follows:

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

This line of code checks if CUDA is available and assigns the appropriate device. You can then use tensor.to(device) to move tensors to the specified device.

Passing the pin_memory value as false

Another way to resolve this error is to pass the pin_memory value in a code while using the get_iterator function. This function is used to call upon an iterator. In this process of using this function ano,ther function called pin_memory gets enabled, which usually passes the dataset while training to the GPU. Thus it should always remain true, but in some instances, it may raise the AssertionError: Torch Not Compiled With CUDA Enabled. To avoid this error, you need to pass the pin_memory value as false while calling for the get_iterator function.

Syntax:

import torch
from your_module import create_batches, get_coco_data  # Import necessary functions and modules

def get_iterator(data, batch_size=32, max_length=30, shuffle=True, num_workers=4, pin_memory=True):
    cap, vocab = data
    return torch.utils.data.DataLoader(
        cap,
        batch_size=batch_size, shuffle=shuffle,
        collate_fn=create_batches(vocab, max_length),
        num_workers=num_workers, pin_memory=pin_memory
    )


train_data = get_iterator(get_coco_data(vocab, train=True), batch_size=args.batch_size, pin_memory=False)

Solution For Mac

While working on Mac, facing this error might be different as the Mac system does not have CUDA. Thus, in the case of Mac, the best solution is to not use a .cuda() function on Mac. Instead, you can externally add NVIDIA CUDA, but it will be incompatible with the integrated intel; thus, using platforms like Google Collaborate will be beneficial.

Solution For Detectron2

For platforms like Detectron2 any of the above solutions won’t work but instead you can use the following code which can help you resolve the AssertionError: Torch Not Compiled With CUDA Enabled.

Syntax:

import torch

# Assuming cfg is a configuration object
cfg = torch.hub.load('facebookresearch/detectron2:main', 'config', 'COCO-Detection/faster_rcnn_R_50_FPN_3x.yaml')

# Set the device to CPU
cfg.MODEL.DEVICE = "cpu"

# Now you can create and use the model with CPU computation
model = torch.hub.load('facebookresearch/detectron2:main', 'model', config=cfg)

FAQs

Conclusion

In conclusion, The “AssertionError: torch not compiled with CUDA enabled” error is a common stumbling block for those looking to harness the GPU-accelerated power of PyTorch. By ensuring a proper installation and configuration, you can unlock the full potential of your system for deep learning tasks.

Always remember to check for GPU compatibility, install the correct drivers, and verify your PyTorch installation with CUDA support. With these steps, you’ll be on your way to seamless GPU-accelerated machine learning workflows.

References

1. AssertionError
2. CUDA

To learn more about some common errors follow Python Clear’s errors section.