My eGPU Setup

index

Intro

I have been using the 2022 Legion 5i Pro since Oct. 2022 and it has served me well. However, with the rise of deep learning, VRAM has become crucial. The 6GB VRAM on the RTX 3060 Mobile just isn’t enough for running medium-sized LLMs, text to image models and experimenting with training/finetuning etc.

At this point I had 2 options, either upgrade to a PC or a high-end laptop. A PC isn’t suitable as I have to travel and high-end laptops with 16GB VRAM mean RTX 5080 Mobile or above, which have ridiculous prices here. For me, an eGPU was a much better option. Fortunately my laptop does have a TB4 port.

So I needed a few things - eGPU Dock, GPU, Power Supply and an USB4 cable.

After some research, I found an eGPU dock with a built in power supply and cables included - Aoostar AG02. As for the GPU, it had to be NVIDIA for CUDA. The used market here is terrible ( no RTX 3090 … ) so I went for the budget option RTX 5060Ti 16GB. I bought a higher end Aorus Elite card, as there is a service center nearby and I want it to run cool even with an ambient of 30°C.

Hardware Setup

Fast forward 2 weeks and both the dock and GPU have arrived.

After unboxing and ensuring there is no visible damage, we are ready to connect everything. Remove the stickers and plastic cap from the GPU. Open the side screw from the eGPU dock, attach the GPU and secure it. Ensure the GPU is inserted properly and sits firmly in place. Carefully attach the GPU’s power cable(s) and the eGPU Dock’s power to the wall socket. Lastly, connect the eGPU to the laptop with the included cable.

Danger (Power Cable)

It is mandatory and extremely crucial that the 8-Pin / 12VHPWR GPU power cable is fully inserted and firmly attached. Ensure the same for the cable end connecting to the PSU of the eGPU Dock.

eGPU Setup

GPU Power Connector

NVIDIA Driver Setup

I’m dual booting Windows 11 and Ubuntu 24.04. We will use Ubuntu for all ML/DL stuff.

Warning (GPU Check)

I tested the GPU on Windows 11 first, although it’s plug-and-play, I updated the NVIDIA drivers just to be sure. A couple of Furmark/3DMark tests with HwInfo monitoring and you’re good to go.

$ nvidia-smi
Sun Jun 29 18:10:48 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 575.57.08              Driver Version: 575.57.08      CUDA Version: 12.9     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 3060 ...    On  |   00000000:01:00.0 Off |                  N/A |
| N/A   48C    P0             12W /   80W |       1MiB /   6144MiB |      9%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

For Ubuntu 24.04 and RTX 50 Series, we need to use a beta driver as shown in this post on the NVIDIA Dev Forum.

Warning (Secure Boot)

Secure Boot must be turned off for beta drivers to work. You could generate an MOK and sign the drivers yourself and …

The current driver was installed from the package manager. We have to stick to one method - either install the driver from a runfile or install it from the package manager. I’ll remove the installed NVIDIA Driver.

$ sudo apt purge '*nvidia*'
12 collapsed lines
The following packages will be REMOVED:
  libnvidia-cfg1-575* libnvidia-common-575* libnvidia-compute-575* libnvidia-decode-575* libnvidia-egl-gbm1*
  libnvidia-egl-wayland1* libnvidia-egl-xcb1* libnvidia-egl-xlib1* libnvidia-encode-575* libnvidia-extra-575*
  libnvidia-fbc1-575* libnvidia-gl-575* libnvidia-gl-575:i386* libnvidia-gpucomp-575* nvidia-compute-utils-575*
  nvidia-dkms-575* nvidia-driver-575* nvidia-firmware-575* nvidia-kernel-common-575* nvidia-kernel-source-575*
  nvidia-modprobe* nvidia-persistenced* nvidia-settings* nvidia-utils-575* xserver-xorg-video-nvidia-575*
0 upgraded, 0 newly installed, 25 to remove and 62 not upgraded.
After this operation, 1,221 MB disk space will be freed.
Do you want to continue? [Y/n] Y
(Reading database ... 238561 files and directories currently installed.)
Removing nvidia-driver-575 (575.57.08-0ubuntu1) ...
Removing all DKMS Modules
Done.

$ sudo apt autoremove
11 collapsed lines
The following packages will be REMOVED:
  dkms libdrm-intel1:i386 libdrm-radeon1:i386 libgbm1:i386 libglapi-mesa:i386 libnvidia-compute-575:i386
  libnvidia-decode-575:i386 libnvidia-egl-gbm1:i386 libnvidia-egl-wayland1:i386 libnvidia-egl-xcb1:i386
  libnvidia-egl-xlib1:i386 libnvidia-encode-575:i386 libnvidia-fbc1-575:i386 libnvidia-gpucomp-575:i386
  libpciaccess0:i386 libsensors5:i386 libwayland-server0:i386 libxcb-dri2-0:i386 libxext6:i386
  mesa-libgallium:i386 screen-resolution-extra
0 upgraded, 0 newly installed, 21 to remove and 61 not upgraded.
After this operation, 350 MB disk space will be freed.
Do you want to continue? [Y/n] Y
Processing triggers for man-db (2.12.0-4build2) ...
Processing triggers for libc-bin (2.39-0ubuntu8.4) ...

Now let’s install the download and install the beta driver as instructed in the NVIDIA Dev Forum post.

NVIDIA Driver Installation

$ chmod u+x NVIDIA-Linux-x86_64-575.51.02.run
$ sudo ./NVIDIA-Linux-x86_64-575.51.02.run --glvnd-egl-config-path=/etc/glvnd/egl_vendor.d --compat32-libdir=/usr/lib32
$ sudo reboot

and voila!

$ nvidia-smi
Sun Jun 29 18:30:47 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 575.51.02              Driver Version: 575.51.02      CUDA Version: 12.9     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 3060 ...    Off |   00000000:01:00.0 Off |                  N/A |
| N/A   49C    P0             22W /   80W |       0MiB /   6144MiB |      3%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
|   1  NVIDIA GeForce RTX 5060 Ti     Off |   00000000:09:00.0 Off |                  N/A |
|  0%   34C    P0             25W /  180W |       0MiB /  16311MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

Both the eGPU and Laptop’s Discrete GPU are available for use.

Danger (Boot Failure)

If Ubuntu fails to boot or gets stuck, open a terminal via recovery mode and uninstall the NVIDIA driver and reboot.

$ sudo nvidia-uninstall

It would have loaded Nouveau drivers. Use the Additional Drivers app to install a normal NVIDIA Driver and then remove and blacklist Nouveau to prevent conflicts with the proprietary NVIDIA driver and prevents trouble from future Ubuntu updates. You can enable it with modeset=1 in the same file.

$ sudo sh -c "echo 'blacklist nouveau\\noptions nouveau modeset=0' > /etc/modprobe.d/blacklist-nouveau.conf"
$ sudo update-initramfs -u

Now reboot and follow the guide as usual.

CUDA Setup

Now let install CUDA using the official guide. I will use the Network Repository Installation method as shown here.

$ wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/cuda-keyring_1.1-1_all.deb
10 collapsed lines
--2025-06-29 18:41:17--  https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/cuda-keyring_1.1-1_all.deb
Resolving developer.download.nvidia.com (developer.download.nvidia.com)... 23.53.243.85, 23.53.243.68
Connecting to developer.download.nvidia.com (developer.download.nvidia.com)|23.53.243.85|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 4328 (4.2K) [application/x-deb]
Saving to: ‘cuda-keyring_1.1-1_all.deb’

cuda-keyring_1.1-1_all.deb   100%[=============================================>]   4.23K  --.-KB/s    in 0s

2025-06-29 18:41:21 (2.11 GB/s) - ‘cuda-keyring_1.1-1_all.deb’ saved [4328/4328]

$ sudo dpkg -i cuda-keyring_1.1-1_all.deb
5 collapsed lines
Selecting previously unselected package cuda-keyring.
(Reading database ... 237692 files and directories currently installed.)
Preparing to unpack cuda-keyring_1.1-1_all.deb ...
Unpacking cuda-keyring (1.1-1) ...
Setting up cuda-keyring (1.1-1) ...

$ sudo apt-get update
14 collapsed lines
Hit:1 https://download.docker.com/linux/ubuntu noble InRelease
Get:2 https://nvidia.github.io/libnvidia-container/stable/deb/amd64  InRelease [1,477 B]
Hit:3 https://packages.microsoft.com/repos/code stable InRelease
Hit:4 http://in.archive.ubuntu.com/ubuntu noble InRelease
Hit:5 http://security.ubuntu.com/ubuntu noble-security InRelease
Hit:6 http://apt.fruit.je/ubuntu noble InRelease
Hit:7 http://in.archive.ubuntu.com/ubuntu noble-updates InRelease
Hit:8 https://ppa.launchpadcontent.net/graphics-drivers/ppa/ubuntu noble InRelease
Hit:9 http://in.archive.ubuntu.com/ubuntu noble-backports InRelease
Hit:10 https://ppa.launchpadcontent.net/obsproject/obs-studio/ubuntu noble InRelease
Get:11 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64  InRelease [1,581 B]
Get:12 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64  Packages [715 kB]
Fetched 718 kB in 7s (110 kB/s)
Reading package lists... Done

$ sudo apt install cuda-toolkit
30 collapsed lines
The following NEW packages will be installed:
  ca-certificates-java cuda-cccl-12-9 cuda-command-line-tools-12-9 cuda-compiler-12-9 cuda-crt-12-9
  cuda-cudart-12-9 cuda-cudart-dev-12-9 cuda-cuobjdump-12-9 cuda-cupti-12-9 cuda-cupti-dev-12-9
  cuda-cuxxfilt-12-9 cuda-documentation-12-9 cuda-driver-dev-12-9 cuda-gdb-12-9 cuda-libraries-12-9
  cuda-libraries-dev-12-9 cuda-nsight-12-9 cuda-nsight-compute-12-9 cuda-nsight-systems-12-9 cuda-nvcc-12-9
  cuda-nvdisasm-12-9 cuda-nvml-dev-12-9 cuda-nvprof-12-9 cuda-nvprune-12-9 cuda-nvrtc-12-9 cuda-nvrtc-dev-12-9
  cuda-nvtx-12-9 cuda-nvvm-12-9 cuda-nvvp-12-9 cuda-opencl-12-9 cuda-profiler-api-12-9 cuda-sandbox-dev-12-9
  cuda-sanitizer-12-9 cuda-toolkit cuda-toolkit-12-9 cuda-toolkit-12-9-config-common
  cuda-toolkit-12-config-common cuda-toolkit-config-common cuda-tools-12-9 cuda-visual-tools-12-9 default-jre
  default-jre-headless fonts-dejavu-extra gds-tools-12-9 java-common libatk-wrapper-java libatk-wrapper-java-jni
  libcublas-12-9 libcublas-dev-12-9 libcufft-12-9 libcufft-dev-12-9 libcufile-12-9 libcufile-dev-12-9
  libcurand-12-9 libcurand-dev-12-9 libcusolver-12-9 libcusolver-dev-12-9 libcusparse-12-9 libcusparse-dev-12-9
  libnpp-12-9 libnpp-dev-12-9 libnvfatbin-12-9 libnvfatbin-dev-12-9 libnvjitlink-12-9 libnvjitlink-dev-12-9
  libnvjpeg-12-9 libnvjpeg-dev-12-9 libxcb-cursor0 libxcb-xinput0 nsight-compute-2025.2.1 nsight-systems-2025.1.3
  openjdk-21-jre openjdk-21-jre-headless
0 upgraded, 73 newly installed, 0 to remove and 61 not upgraded.
Need to get 3,933 MB of archives.
After this operation, 9,220 MB of additional disk space will be used.
Do you want to continue? [Y/n] Y
done.
Setting up default-jre-headless (2:1.21-75+exp1) ...
Setting up openjdk-21-jre:amd64 (21.0.7+6~us1-0ubuntu1~24.04) ...
Setting up default-jre (2:1.21-75+exp1) ...
Setting up cuda-nvvp-12-9 (12.9.79-1) ...
Setting up cuda-nsight-12-9 (12.9.79-1) ...
Setting up cuda-visual-tools-12-9 (12.9.1-1) ...
Setting up cuda-tools-12-9 (12.9.1-1) ...
Setting up cuda-toolkit-12-9 (12.9.1-1) ...
Setting up cuda-toolkit (12.9.1-1) ...
Setting alternatives

$ sudo reboot

Add export PATH=${PATH}:/usr/local/cuda-12.9/bin in your shell startup file.

And now we have CUDA Toolkit installed.

$ nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2025 NVIDIA Corporation
Built on Tue_May_27_02:21:03_PDT_2025
Cuda compilation tools, release 12.9, V12.9.86
Build cuda_12.9.r12.9/compiler.36037853_0

Some Testing

Warning (Performance)

A Thunderbolt 4 connection has a lower bandwidth than a native PCIe x16 slot and this will bottleneck the GPU’s performance in scenarios where lots of data has to be transferred (like gaming at high frame rates).

Using an external monitor (connected to the GPU) will also increase performance as it significantly reduces the amount of data being sent from the GPU.

On Windows, this testing was done without optimizations, such as disabling PCI Express > Link State Power Management. Subsequent runs showed significantly improved frametime stability with this feature disabled.

I have LM Studio installed already, so let’s check the inference performance for some medium-sized models. (Laptop on Balanced Mode)

Mistral-Small-3.2-24B-Instruct-2506-IQ4_XS.gguf (Vision)

Flash Attention, FP16 KV Cache, 16K Context Length
Starts with 28.5 tokens/s and 20 token/s when context is nearly full.

Qwen3-30B-A3B-Q4_K_M.gguf

Flash Attention, FP16 KV Cache, 16K Context Length
Starts with 71 tokens/s and 40 token/s when context is nearly full.

Some 3DMark scores (Laptop in Performance Mode)

With stock settings

Timespy - 11687
Steel Nomad - 3298

With +100 MHz core clock , +2000 MHz memory clock , 120% Power Limit

Timespy - 12063 (~23% performance loss from desktop - bandwidth bound)
Steel Nomad - 3518 (~1% performance loss from desktop - compute bound)

Core and Memory Junction temperatures look very good (54°C - 62°C).

Final Thoughts

I don’t expect to use this setup for more than 1.5-2 years, so I think this is fine until I settle down and build a PC. Overall this setup looks very promising and the additional VRAM is great. The next step is training/finetuning models.