Disclaimer & buying options for OCuLink eGPU adapters for reliable up to 64Gbps ePCIe 4.0 x4 connectivity is here
I am experiencing serious bandwidth limitations when using an nvme ssd on an eGPU case.
The nvme ssd, when connected to an internal PCI-e gives speeds close to 3000MB/s in reading and writing (in a PC with Ubuntu).
The same nvme ssd connected through an eGPU gives speeds of 800MB/s in writing and 1700MB/s in reading (on a 2018 Mac mini).
This is not the well-known limitation of the bandwidth to 10Gb/s due to the firmware bug a few years ago. Because: 1) the firmware version used no longer has that limitation and 2) with an AMD graphics card it gives a H2D bandwidth of 20Gb/s (measured with CL!ng).
I expected speeds close to 2500MB/s when using this ssd nvme in the eGPU (due to the known 20Gb/s limitation of the eGPU), but.... only 800MB/s! Why?
Any idea where the problem might be? Are there limitations I don't know about when using eGPU as storage units? THANK YOU!
Test conditions:
- eGPU tested: An Akitio node (which I own, firmware 25.1) and a Racer Core X (temporarily borrowed, firmware 33.1)
- OS: macOS Catalina and macOS Big Sur
- Thunderbolt Cable: The stock 0.5m one and an Apple Thunderbolt 3 Pro Cable (2m) one
- Benchmark software: AJA System Test and Blackmagic Disk Speed Test
Other proven things:
- Replace the nvme ssd by an ARECA card connected to a 10-disk SATA RAID 0, with the same result.
AKiTiO Node, firmware 25.1
Check the Thunderbolt connections: 40 Gbps x1, link width x2.
Check the PCIe connections: 8 GT/s x4.
Replace the gen 3 NVMe with a PCIe gen 4 NVMe. It should give 2800 MB/s read/write with AmorphousDiskMark sequential test.
Many gen 3 NVMe drives have limits with Thunderbolt. The Sabrent Rocket 4.0 2 TB I tested does not.
@joevt, you were right!
Connectivity was not the problem, as it was right (40 Gbps x1, link width x2 and 8 GT/s x4).
Replaced the NVMe with a Sabrent Rocket 4.0 512GB and both write/read maxed at 2700 MB/s!
But this brings me to my real goal: to make my RAID0 work.
In short, with the same eGPU (an Akitio Node):
- 3 NVMe SSD give over 800MB/s in writing, 3 NVMe!
- One RAID0 of 10 disks also over the 800MB/s in writing instead of the expected 1800MB/s. Tried with 2 RAID controllers (one from ARECA, and one from LSI), with same result.
But, in contrast, the NVMe Sabrent Rocket 4.0 512GB gives 2700MB/s!
Why? Why some PCIe devices don't give all the expected performance? Any incompatibility? Any help would be appreciated.
AKiTiO Node, firmware 25.1
@vicmarto, I don't know what the problem could be. Maybe lspci -vvv might show an error or something but I don't know what to look for. Maybe a comparison of the the gen 4 NVMe with a gen 3 NVMe might show something?
My guess is there's compatibility issues between the NVMe controller and Thunderbolt controller. I've found a few PCIe 3.0 drives (WD Black NS750 and Sandisk Extreme Pro) that work great inside TB3 enclosure [link].
LinkedIn | Youtube | Instagram
• external graphics card builds
• best laptops for external GPU
• eGPU enclosure buyer's guide
2017 21" iMac 4K (RP555) [7th,4C] + RX 6800 @ 32Gbps-TB3 (Netstor HL23T-Plus) + macOS 13 & Win11 22H2 [build link]
@itsage, You mention Sandisk Extreme Pro at https://egpu.io/forums/thunderbolt-enclosures/wavelink-ute02-nvme-m-2-thunderbolt-3-enclosure