Picture this: You are running a local LLM, compiling a massive Rust project, or docker-composing a complex microservices mesh on your local development machine. Suddenly, your system slows to a crawl, your mouse cursor freezes, and the dreaded Linux Out-Of-Memory (OOM) killer steps in to mercilessly terminate your IDE. We’ve all been there. System RAM is precious, and when it runs out, your SSD-backed swap space feels painfully slow.
But then you look at your desk. Inside your machine sits a modern Nvidia GPU with 8GB, 12GB, or even 24GB of ultra-fast GDDR6 VRAM. While you aren't actively training a model or playing a AAA game, that incredibly fast memory is just sitting there, completely idle. What if you could hijack that VRAM and use it as ultra-high-speed system swap space?
Thanks to an ingenious open-source project making waves on Hacker News, you can now do exactly that on Linux. Today, we are diving deep into how this works, why VRAM swap is orders of magnitude faster than traditional NVMe swap, and how to set it up on your own Linux development rig.
The Memory Bottleneck: RAM vs. NVMe vs. VRAM
To understand why this is a game-changer, we need to look at the massive performance disparity between different storage tiers in a modern PC. When your system runs out of physical RAM (DDR4 or DDR5), Linux relies on "swap space" (traditionally a partition or a file on your SSD) to temporarily hold idle memory pages.
While modern PCIe Gen4 and Gen5 NVMe SSDs are incredibly fast compared to old spinning hard drives, they still crawl compared to system memory. Let's look at the theoretical bandwidth numbers:
- PCIe Gen4 NVMe SSD: ~7 GB/s
- DDR5 System RAM: ~50 to 80 GB/s
- Nvidia RTX 4070 Ti VRAM (GDDR6X): ~504 GB/s
VRAM is designed to push massive amounts of graphical and mathematical data to GPU cores simultaneously. It operates at bandwidths that make even the fastest DDR5 system RAM look slow, and utterly obliterates NVMe SSDs. By mounting a swap partition directly inside your GPU's VRAM, you can handle memory overflows with a fraction of the performance degradation you would experience using a standard SSD swap file.
How Does It Work? Enter nvswp
Historically, leveraging GPU memory as system storage required complex ramdisk drivers or proprietary CUDA blocks. However, the open-source community has streamlined this process using a lightweight utility designed to allocate a block of memory on an Nvidia GPU via CUDA and expose it to the Linux kernel as a block device using the Linux Network Block Device (NBD) or user-space block devices (ublk).
The system architecture looks something like this:
+-------------------------------------------------------------+
| Linux Kernel |
| +------------------+ +----------------+ |
| | VM Subsystem | | Block Device | |
| | (Memory Manager) | | Driver (NBD) | |
| +--------+---------+ +--------+-------+ |
+-----------|--------------------------------------|----------+
| (Paging/Swapping) |
v v
+-------------------------------------------------------------+
| User-Space Daemon |
| +-------------------------------------------------------+ |
| | nvswp | |
| +-------------------------------------------------------+ |
+-----------------------------|-------------------------------+
| (CUDA Memory Allocation)
v
+-------------------------------------------------------------+
| Nvidia GPU VRAM |
| [================ Allocated Swap Block ================] |
+-------------------------------------------------------------+
The daemon runs in user space, allocates a chunk of VRAM using CUDA, and bridges it to the kernel's virtual memory subsystem. When the OS needs to swap pages out of system RAM, it writes them directly to the GPU over the high-speed PCIe bus.
Step-by-Step Guide: Setting Up VRAM Swap on Linux
Before we begin, a quick disclaimer: This is experimental tech. Do not run this on production servers. However, for a local development machine where you want to squeeze out every drop of performance, it is incredibly fun to play with.
Prerequisites
To follow along, you will need:
- A Linux distribution (Ubuntu, Arch, and Fedora work best).
- Proprietary Nvidia Drivers installed and functioning.
- The CUDA Toolkit installed (specifically,
nvccmust be in your PATH). - Kernel support for the Network Block Device (
nbd) module (virtually all mainstream kernels have this enabled by default).
Step 1: Install and Load the NBD Kernel Module
First, we need to ensure the Linux kernel is ready to handle network block devices. Load the module with the following command:
sudo modprobe nbd max_part=8To ensure this module loads automatically every time your system boots, add it to your modules configuration:
echo "nbd" | sudo tee -a /etc/modules-load.d/nbd.confStep 2: Build the VRAM Block Device Utility
We will use a popular open-source tool designed for this purpose. Clone the repository and compile it using your local CUDA compiler:
git clone https://github.com/mrcatofes/nvswp.git
cd nvswp
makeThis will generate an executable binary named nvswp in your directory.
Step 3: Allocate VRAM and Initialize Swap
Now, let's allocate a chunk of VRAM. For this example, let's assume we want to allocate 4GB (4096 Megabytes) of our GPU's VRAM as a swap device. Run the daemon (you will need root privileges to interact with /dev/nbd0):
sudo ./nvswp --size 4096 --device /dev/nbd0At this point, /dev/nbd0 is mapped directly to a 4GB allocation in your GPU's VRAM. Open a new terminal window to format and enable this block device as swap space:
# Format the block device as swap
sudo mkswap /dev/nbd0
# Enable the swap space with a high priority
sudo swapon -p 32767 /dev/nbd0Note the -p 32767 flag. This tells the Linux kernel to prioritize this swap space over your existing SSD-backed swap. Linux will now exhaust your ultra-fast GPU VRAM swap before it ever touches your slower NVMe drive.
Verifying the Setup
To verify that your system is successfully leveraging your GPU for swap, you can check the active swap spaces:
swapon --showYou should see an output similar to this, showcasing your high-priority NBD device:
NAME TYPE SIZE USED PRIO /dev/dm-1 partition 16G 0B -2 /dev/nbd0 partition 4G 0B32767
You can also run nvidia-smi in your terminal. You will notice that your GPU memory usage has jumped by exactly 4GB, listed under a process managed by nvswp.
The Pro Developer Catch: Is This Actually Practical?
While this is an incredibly cool engineering hack, there are a few architectural trade-offs that you need to keep in mind before making this a permanent fixture of your development environment.
1. The PCIe Bus Bottleneck
Even though GDDR6 VRAM can run at 500+ GB/s internally on the graphics card, the data traveling between your system RAM and your GPU must pass through the PCIe slot.
- A PCIe 4.0 x16 slot has a maximum bandwidth of about 31.5 GB/s.
- A PCIe 3.0 x16 slot tops out at around 15.8 GB/s.
2. CUDA Context Suspensions
If you are actively developing AI models or running PyTorch, your GPU will need to allocate memory for CUDA contexts. If your VRAM is fully allocated to system swap, you might run into "CUDA Out of Memory" errors when you try to run your models. This tool is best used when your GPU is otherwise idling, acting as a dynamic buffer for heavy compilation tasks or Docker local runs.
3. Suspend and Hibernate
Most GPU drivers do not preserve VRAM state when your computer goes into sleep or suspend mode. If your system swaps out critical OS pages to the GPU and you close your laptop lid or put your PC to sleep, you will likely resume to a completely frozen system or a kernel panic. Always disable VRAM swap before suspending your machine.
Automating the Workflow
If you want to use this regularly, you can write a simple bash script to easily toggle your GPU swap on and off:
#!/bin/bash
# toggle-gpu-swap.sh
if [ "$1" == "start" ]; then
echo "Loading NBD module..."
sudo modprobe nbd
echo "Starting nvswp daemon..."
sudo ./nvswp --size 4096 --device /dev/nbd0 &
sleep 2
echo "Activating Swap..."
sudo mkswap /dev/nbd0
sudo swapon -p 32767 /dev/nbd0
echo "GPU Swap Active!"
elif [ "$1" == "stop" ]; then
echo "Deactivating Swap..."
sudo swapoff /dev/nbd0
echo "Stopping daemon..."
sudo killall nvswp
echo "GPU Swap Disabled."
else
echo "Usage: $0 {start|stop}"
fiWrapping Up
Using your Nvidia GPU's VRAM as Linux swap space is a prime example of the power and flexibility of the Linux kernel. It allows developers to repurpose incredibly expensive hardware that would otherwise sit idle during non-graphics-intensive tasks, providing a high-speed buffer that prevents system freezes during resource-heavy local workflows.
What do you think? Would you trust your idle GPU VRAM to handle your system's memory overflows, or do you prefer sticking to traditional NVMe swap files? Let me know in the comments below!
Looking for more deep dives into Linux internals, developer workflows, and system optimization? Subscribe to the "Coding with Alex" newsletter at sysseder.com to get our latest technical guides delivered straight to your inbox!