Enclosure & Hardware Discussions
eGPU Enclosure for multiple GPUs
 

eGPU Enclosure for multiple GPUs  

  RSS

NorthVisualStudio
(@northvisualstudio)
New Member
Joined: 4 weeks ago
 

Hi

I use GPU's to render 3D scenes and was trying to make an external chassis to house additional GPU's. The idea is to replicate the Thunderbolt eGPUs which can be purchased but with more GPU's.

The idea is to have 6 or 8 GPUs per enclosure but I'm not sure how to go about it. I'm not sure if this too many GPUs. I have a good idea of the power requirements, enclosure etc.. but where my knowledge fails is on how the GPUs are connected to the Thunderbolt cards.

If anyone has any advise on this, it would be most appreciated.

Thanks

N

Pending: Add my system information and expected eGPU configuration to my signature to give context to my posts


ReplyQuote
ruslan
(@ruslan)
Active Member
Joined: 2 months ago
 
Posted by: @northvisualstudio

Hi

I use GPU's to render 3D scenes and was trying to make an external chassis to house additional GPU's. The idea is to replicate the Thunderbolt eGPUs which can be purchased but with more GPU's.

The idea is to have 6 or 8 GPUs per enclosure but I'm not sure how to go about it. I'm not sure if this too many GPUs. I have a good idea of the power requirements, enclosure etc.. but where my knowledge fails is on how the GPUs are connected to the Thunderbolt cards.

If anyone has any advise on this, it would be most appreciated.

Thanks

N

Maybe a PCI-Express splitter?

I was looking into buying one as well and it looks like the cheap bitcoin-mining splitters that use USB aren't supported by MacOS.
Your best bet is an expensive option - around $200. 
https://forums.macrumors.com/threads/amfeltec-pci-express-splitter.2027827/

(The setup would most likely not look pretty - you'd have to power on additional power supplies by hand and build a custom box for this.)

Pending to give context to my posts: Add my system information and expected eGPU configuration to my signature

ReplyQuote
NorthVisualStudio
(@northvisualstudio)
New Member
Joined: 4 weeks ago
 

Thanks for the link. How does the splitter connect to a Thunderbolt card? Do you know if mini PCIe bottlenecks the data transfer?

Would one of these backplates be an option?
https://www.onestopsystems.com/product/expansion-backplane-8-pcie-x8-slots-452

This post was modified 4 weeks ago

Pending: Add my system information and expected eGPU configuration to my signature to give context to my posts


ReplyQuote
joevt
(@joevt)
Reputable Member
Joined: 2 years ago
 
Posted by: @ruslan

it looks like the cheap bitcoin-mining splitters that use USB aren't supported by MacOS

If the bitcoin mining splitter has a PCIe switch then it should work. All the options on that Mac Rumors post also use a PCIe switch. Maybe there's power issues (turning things on in the right order)? If there is resource issues (memory, etc.) then any other solution will have the same resource issues.

Posted by: @northvisualstudio

How does the splitter connect to a Thunderbolt card? Do you know if mini PCIe bottlenecks the data transfer?

Would one of these backplates be an option?
https://www.onestopsystems.com/product/expansion-backplane-8-pcie-x8-slots-452

Bitcoin mining splitters are usually all 1 PCIe lane (7.877 Gbps) shared by all connected devices regardless of the slot you connect it to.

Any Thunderbolt 3 PCIe expansion box will have a PCIe slot that you can connect those solutions to. There's four PCIe 3.0 lanes but Thunderbolt is limited to 22 Gbps.

The onestopsystems backplanes have a PCIe switch (if they provide more than one slot).

A PCIe switch has multiple downstream ports (one for each slot that it provides) and an upstream port to connect to a PCIe slot of your computer (or Thunderbolt 3 PCIe expansion box).

There are 3 items between the upstream port of the PCIe switch on the backplane and the computer you want to connect it to:
1) A target PCIe card on the backplane
2) A host PCIe card in your computer or Thunderbolt 3 expansion box
3) One or two cables to connect the host and target cards. The cables can provide up to 16 PCIe lanes but you only need 4 lanes if you're host card is in a Thunderbolt expansion box.

I have a Netstor NA255A which is similar to the backplane you linked but has only 4 slots (but the slots support PCIe 3.0). It includes the case, power supply, fans, etc., target card, host card, and two x8 cables. I haven't tried connecting it to a Thunderbolt 3 PCIe expansion box but there isn't any reason why it shouldn't work. For example, I have a Sonnet Echo Express III-D which has 3 slots (uses a PCIe switch to provide 3 PCIe slots). I also have an Amfeltec x16 PCI Express Gen 3 SQUID Carrier Board for 4 M.2 (uses a PCIe switch to provide 4 M.2 slots) which I've used in the III-D (which means you can chain PCIe switches).

You didn't say what kind of system you want to connect the graphics cards to. You said Thunderbolt card - does that mean you have PCIe slots? If so, then don't use Thunderbolt. If you really want to try Thunderbolt, then use the solution mentioned above or:

 - Find a target PCIe card that has a Thunderbolt port instead of external PCIe cable connector. I think One Stop Systems might have such a card because they mention a Thunderbolt option for their CUBE products such as the nanoCUBE. Maybe it's Thunderbolt 1 (10 Gbps) or Thunderbolt 2 (16 Gbps) instead of Thunderbolt 3? If such a card is available, I wonder if it will work in any of the backplanes? Sonnet's Thunderbolt 3 PCIe expansion boxes appear to use a Thunderbolt 3 target PCIe card. And they sell it separately as an upgrade for their Thunderbolt 2 PCIe expansion boxes. Maybe it could work? I have one but haven't tried it with the Netstor.

I don't know how many graphics cards can be supported. It might be different between MacOS and Windows. Thunderbolt drivers might add limitations.

Pending: Add my system information and expected eGPU configuration to my signature to give context to my posts


ReplyQuote