Q: What do the eGPU port acronyms like 32Gbps-TB3 mean and what are their ranked & measured peak H2D/write bandwidths?

   Disclaimer & buying options for OCuLink eGPU adapters for reliable up to 64Gbps ePCIe 4.0 x4 connectivity is here

Bandwidth limitations when using eGPUs as storage units?  

  RSS

vicmarto
(@vicmarto)
Active Member
Joined: 6 years ago
 

I am experiencing serious bandwidth limitations when using an nvme ssd on an eGPU case.

The nvme ssd, when connected to an internal PCI-e gives speeds close to 3000MB/s in reading and writing (in a PC with Ubuntu).

The same nvme ssd connected through an eGPU gives speeds of 800MB/s in writing and 1700MB/s in reading (on a 2018 Mac mini).

This is not the well-known limitation of the bandwidth to 10Gb/s due to the firmware bug a few years ago. Because: 1) the firmware version used no longer has that limitation and 2) with an AMD graphics card it gives a H2D bandwidth of 20Gb/s (measured with CL!ng).

I expected speeds close to 2500MB/s when using this ssd nvme in the eGPU (due to the known 20Gb/s limitation of the eGPU), but.... only 800MB/s! Why? Question  

Any idea where the problem might be? Are there limitations I don't know about when using eGPU as storage units? THANK YOU!

 

 


 

Test conditions:

Other proven things:
- Replace the nvme ssd by an ARECA card connected to a 10-disk SATA RAID 0, with the same result.

AKiTiO Node, firmware 25.1


ReplyQuote
joevt
(@joevt)
Noble Member
Joined: 8 years ago
 

@vicmarto

Check the Thunderbolt connections: 40 Gbps x1, link width x2.

Check the PCIe connections: 8 GT/s x4.

Replace the gen 3 NVMe with a PCIe gen 4 NVMe. It should give 2800 MB/s read/write with AmorphousDiskMark sequential test.

Many gen 3 NVMe drives have limits with Thunderbolt. The Sabrent Rocket 4.0 2 TB I tested does not.

 #263 

 

 

Mac mini (2018), Mac Pro (Early 2008), GA-Z170X-Gaming 7, Sapphire Pulse Radeon RX 580 8GB GDDR5, Radeon Pro W5700, Sonnet Echo Express III-D, Trebleet Thunderbolt 3 to NVMe M.2 case


ReplyQuote
vicmarto
(@vicmarto)
Active Member
Joined: 6 years ago
 

@joevt, you were right!

Connectivity was not the problem, as it was right (40 Gbps x1, link width x2 and 8 GT/s x4).

Replaced the NVMe with a Sabrent Rocket 4.0 512GB and both write/read maxed at 2700 MB/s!

 

But this brings me to my real goal: to make my RAID0 work.

In short, with the same eGPU (an Akitio Node):

  • 3 NVMe SSD give over 800MB/s in writing, 3 NVMe!
  • One RAID0 of 10 disks also over the 800MB/s in writing instead of the expected 1800MB/s. Tried with 2 RAID controllers (one from ARECA, and one from LSI), with same result.

But, in contrast, the NVMe Sabrent Rocket 4.0 512GB gives 2700MB/s!

Why? Why some PCIe devices don't give all the expected performance? Any incompatibility? Any help would be appreciated.

 

AKiTiO Node, firmware 25.1


ReplyQuote
joevt
(@joevt)
Noble Member
Joined: 8 years ago
 

@vicmarto, I don't know what the problem could be. Maybe lspci -vvv might show an error or something but I don't know what to look for. Maybe a comparison of the the gen 4 NVMe with a gen 3 NVMe might show something?

 

Mac mini (2018), Mac Pro (Early 2008), GA-Z170X-Gaming 7, Sapphire Pulse Radeon RX 580 8GB GDDR5, Radeon Pro W5700, Sonnet Echo Express III-D, Trebleet Thunderbolt 3 to NVMe M.2 case


ReplyQuote
itsage
(@itsage)
Founder Admin
Joined: 8 years ago
 

My guess is there's compatibility issues between the NVMe controller and Thunderbolt controller. I've found a few PCIe 3.0 drives (WD Black NS750 and Sandisk Extreme Pro) that work great inside TB3 enclosure [link].

LinkedIn | Youtube | Instagram
 
external graphics card builds
best laptops for external GPU
eGPU enclosure buyer's guide

 
2017 21" iMac 4K (RP555) [7th,4C] + RX 6800 @ 32Gbps-TB3 (Netstor HL23T-Plus) + macOS 13 & Win11 22H2 [build link]  


ReplyQuote
joevt
(@joevt)
Noble Member
Joined: 8 years ago
 

@itsage, You mention Sandisk Extreme Pro at https://egpu.io/forums/thunderbolt-enclosures/wavelink-ute02-nvme-m-2-thunderbolt-3-enclosure

Mac mini (2018), Mac Pro (Early 2008), GA-Z170X-Gaming 7, Sapphire Pulse Radeon RX 580 8GB GDDR5, Radeon Pro W5700, Sonnet Echo Express III-D, Trebleet Thunderbolt 3 to NVMe M.2 case


ReplyQuote