Setup & Software Discussions
Dual eGPU setup working on nMP + FCPX fix + Pictures via Instagram
 

Dual eGPU setup working on nMP + FCPX fix + Pictures via Instagram  

 of  2
  RSS

imcalledandy
(@imcalledandy)
Estimable Member
Joined: 3 years ago
 

I now have 2 x 980Ti running on my 2013 Mac Pro 🙂

I followed the setup using @goalque's script to get the first one up and running. That is plugged into Bus-0, lower left port.

Then I built a 2nd using the exact same parts as the first - main components are an EVGA 980Ti and the Akitio Thunder2. Powered by BeQuiet PSUs using the molex to barrel method to power the Akitio.

The second eGPU is plugged into Bus-1 top left port.

I'm using the eGPUs for rendering in C4D (Octane / Cycles) The results are 50% faster than CPU rendering (in Cycles) Octane is just much faster with more cards 🙂

To get FCPX to work, it's a bit odd - I have a Thunderbolt display plugged into Bus-0 port 2 (so lower right hand side). I didn't make the connection until I tried to open FCPX without the Thunderbolt display connected. FCPX would crash on launch every time. With the TB display plugged in it opens normally.

I can provide more information and hopefully I'll get some photos of the builds once the second one is complete.

multiCUDA

Here are some images of the build:  https://www.instagram.com/p/BRbmex7htEL

2013 MacPro 2xD700 [email protected] (AKiTiO Thunder2) + macOS 10.13.3
2017 TB MBP 2xGTX1080Ti-TB3 (Akitio Node) + macOS 10.13.4


nycjunglist, visua, deplomb and 3 people liked
ReplyQuote
itsage
(@itsage)
Famed Member Admin
Joined: 3 years ago
 

That’s a lot of GPUs! I’m glad you finally got it going.

Best ultrabooks for eGPU use

eGPU enclosure buying guide


ReplyQuote
imcalledandy
(@imcalledandy)
Estimable Member
Joined: 3 years ago
 
Posted by: itsage

 

That's a lot of GPUs! I'm glad you finally got it going.

   

Thanks! Me too!

2013 MacPro 2xD700 [email protected] (AKiTiO Thunder2) + macOS 10.13.3
2017 TB MBP 2xGTX1080Ti-TB3 (Akitio Node) + macOS 10.13.4


ReplyQuote
Flint Ironstag
(@flint-ironstag)
Estimable Member
Joined: 3 years ago
 

Congratulations on your setup!  Looking forward to seeing the photos.   I find it odd that more nMP owners aren't looking into this.  Maybe it's because of the nosebleed pricing and unnecessarily complex installation.  Or the fact that our GPU choices are a generation behind.

MP 6,1 | 4c | d700
MP 6,1 | 6c | d500


ReplyQuote
Anchoricex
(@anchoricex)
Trusted Member
Joined: 3 years ago
 

I have been wondering this for a while. With Dual AMD's and Dual 980TI's, you now have a setup that'll do both optimized FCPX work (D700's), and will play hard ball with Adobe applications / Maxon C4D that work better with Cuda thru your nVidia cards. That's pretty beast man! Are there any issues with applications selecting between which cards are used?

Pending: Add my system information and expected eGPU configuration to my signature to give context to my posts


ikir and imcalledandy liked
ReplyQuote
nickmilitello
(@nickmilitello)
Eminent Member
Joined: 3 years ago
 

What OS are you running to get FCPX and dual 980ti to work?  Are you on Seirra and FCPX 10.3.2?

Pending: Add my system information and expected eGPU configuration to my signature to give context to my posts


ReplyQuote
nickmilitello
(@nickmilitello)
Eminent Member
Joined: 3 years ago
 

What OS are you running to get FCPX and dual 980ti to work?  Are you on Seirra and FCPX 10.3.2?

Pending: Add my system information and expected eGPU configuration to my signature to give context to my posts


ReplyQuote
nickmilitello
(@nickmilitello)
Eminent Member
Joined: 3 years ago
 

Quick update.  Tried plugging in a HMDI cable to Titan Z to my 40" 4k Samsung tv (use a primary display((usually use Mini DisplayPort to HDMI 4k at 60htx). no output, just black screen.

Tried getting an old 1080 HDMI monitor only plugging in Titan Z - HDMI - 1080 monitor FCPX 10.3.2 works fine.  

Plugged in 4k monitor to Mini DisplayPort while 1080 was plugged into Titan Z while FCPX was running and FCPX was stable.  However restart FCPX while both plugged in, crash. 

Ordering a display port to HDMI 4k 60htx adapter to come out of the Titan to see if that works.... Will report back.  Seems to be something with if the d700 are in use FCPX doesn't know what to do.....

Pending: Add my system information and expected eGPU configuration to my signature to give context to my posts


ReplyQuote
Menneisyys
(@menneisyys)
Eminent Member
Joined: 3 years ago
 

"Ordering a display port to HDMI 4k 60htx adapter"

I'd be interested in such a thing too. All I could find so far was 4k30.

Pending: Add my system information and expected eGPU configuration to my signature to give context to my posts


ReplyQuote
imcalledandy
(@imcalledandy)
Estimable Member
Joined: 3 years ago
 

@Flint Ironstag - Thanks - I'll get some photos up soon - have been in and out of the office, but I'll aim to do it this week.

@Anchoricex - Cheers! No issues with switching applications. I used to have some startup problems where I would have to unplug the Thunderbolt monitor to get macOS to chime but once you've run the setup once and shutdown the issues go away on the next boot.

@nickmilitello - Yes on Sierra (10.12.2) and latest FCPX. As I mentioned, I think it's down to the Thunderbolt display using one of the D700 cards and so tricking FCPX into ignoring the GTX 980Tis. I think you're coming to a similar conclusion! Hope the adapter works for you.

 

2013 MacPro 2xD700 [email protected] (AKiTiO Thunder2) + macOS 10.13.3
2017 TB MBP 2xGTX1080Ti-TB3 (Akitio Node) + macOS 10.13.4


ReplyQuote
imcalledandy
(@imcalledandy)
Estimable Member
Joined: 3 years ago
 

Is anyone running macOS 10.12.3 and not having any issues with their Mac Pro + eGPU setup? Just want to check before I update.

2013 MacPro 2xD700 [email protected] (AKiTiO Thunder2) + macOS 10.13.3
2017 TB MBP 2xGTX1080Ti-TB3 (Akitio Node) + macOS 10.13.4


ReplyQuote
nickmilitello
(@nickmilitello)
Eminent Member
Joined: 3 years ago
 

Sooooo..... Finally got some success with eGPU and FCPX 10.3.2.  Had to use a Displayport to HDMI 4k 60hz adapter.  Found it on Amazon.   Came straight from Titan Z to my 4k display.  Thankfully I am only using a single monitor.  If I plug in a HDMI to the Titan or the Mac Pro to get a second monitor in directly and launch FCPX, it crashes again.  

Also I am still only able to get a single Titan Z or 980ti to boot in Sierra 10.12.2.  However it is nice to have FCPX and Octane working without having to restart constantly.  thanks all for the help.

Pending: Add my system information and expected eGPU configuration to my signature to give context to my posts


ReplyQuote
imcalledandy
(@imcalledandy)
Estimable Member
Joined: 3 years ago
 

@nickmilitello Glad you got it working!

2013 MacPro 2xD700 [email protected] (AKiTiO Thunder2) + macOS 10.13.3
2017 TB MBP 2xGTX1080Ti-TB3 (Akitio Node) + macOS 10.13.4


ReplyQuote
laudisi
(@laudisi)
New Member
Joined: 3 years ago
 

I am running 10.12.3 on nMP.... just not successfully. I'll have a more detailed post later today to hopefully suss it out...

Pending: Add my system information and expected eGPU configuration to my signature to give context to my posts


ReplyQuote
imcalledandy
(@imcalledandy)
Estimable Member
Joined: 3 years ago
 

@laudisi - looking forward to hearing your progress on this.

2013 MacPro 2xD700 [email protected] (AKiTiO Thunder2) + macOS 10.13.3
2017 TB MBP 2xGTX1080Ti-TB3 (Akitio Node) + macOS 10.13.4


ReplyQuote
imcalledandy
(@imcalledandy)
Estimable Member
Joined: 3 years ago
 

For those interested, I have posted the build of my eGPUs over on my Instagram

https://www.instagram.com/p/BRbmex7htEL

2013 MacPro 2xD700 [email protected] (AKiTiO Thunder2) + macOS 10.13.3
2017 TB MBP 2xGTX1080Ti-TB3 (Akitio Node) + macOS 10.13.4


ReplyQuote
benr
 benr
(@benr)
Eminent Member
Joined: 3 years ago
 

Quick question: have you tried daisy-chaining the two eGPUs (assuming your Akitio cards have two thunderbolt ports)?

I have a build very similar using a 980Ti with the cards from a HighPoint RocketStor 6361A in a Silverstone SG13B case. 

I've been debating adding a 2nd identical setup for the Octane/Cycles speed gains (and hopefully DaVinci Resolve also). Which got me wondering:

There's a lot of empty space in these cases, and SFX-L power supplies exist up to 800W which would comfortably drive both 980Ti cards. 

If the two Thunderbolt cards work daisy-chained, I'm trying to figure out if the entire two-card setup could be fitted in one (perhaps custom) case with a short internal cable connecting them?

(Or if anyone who knows more than I do could chime in, is there any kind of daughter card or riser that could put two GPUs on one thunderbolt controller card?)

Thanks

Pending: Add my system information and expected eGPU configuration to my signature to give context to my posts


ReplyQuote
Yukikaze
(@yukikaze)
Prominent Member Moderator
Joined: 3 years ago
 

PCIe switch-capable risers exist. They can be used to connect two PCIe devices to a single physical slot by switching PCIe traffic between the two devices, each in their own downstream slot. They do not tend to be cheap, though (except of single-lane ones that can be found for relatively cheap on ebay).

Want to output [email protected] out of an old system on the cheap? Read here.
Give your Node Pro a second Thunderbolt3 controller for reliable peripherals by re-using a TB3 dock (~50$).

"Always listen to experts. They'll tell you what can't be done, and why. Then do it."- Robert A. Heinlein, "Time Enough for Love."


ReplyQuote
Flint Ironstag
(@flint-ironstag)
Estimable Member
Joined: 3 years ago
 

Maxexpansion has a solution:

 

http://www.maxexpansion.com/cube2-metal-expansion-enclosure-5-pcie-slots

 

MP 6,1 | 4c | d700
MP 6,1 | 6c | d500


ReplyQuote
imcalledandy
(@imcalledandy)
Estimable Member
Joined: 3 years ago
 

Good to know there are other options... it's quite expensive and would that power supply be adequate? 

2013 MacPro 2xD700 [email protected] (AKiTiO Thunder2) + macOS 10.13.3
2017 TB MBP 2xGTX1080Ti-TB3 (Akitio Node) + macOS 10.13.4


ReplyQuote
imcalledandy
(@imcalledandy)
Estimable Member
Joined: 3 years ago
 

@benr - I think they can be daisy chained but I haven't tried it. 

I can say that a second card works great with Octane. 

2013 MacPro 2xD700 [email protected] (AKiTiO Thunder2) + macOS 10.13.3
2017 TB MBP 2xGTX1080Ti-TB3 (Akitio Node) + macOS 10.13.4


ReplyQuote
Misaka19999
(@misaka19999)
New Member
Joined: 3 years ago
 

Cool, just wondering if you has enabled SLI?

Pending: Add my system information and expected eGPU configuration to my signature to give context to my posts


ReplyQuote
imcalledandy
(@imcalledandy)
Estimable Member
Joined: 3 years ago
 

No, I don't believe that is possible.

2013 MacPro 2xD700 [email protected] (AKiTiO Thunder2) + macOS 10.13.3
2017 TB MBP 2xGTX1080Ti-TB3 (Akitio Node) + macOS 10.13.4


ReplyQuote
exam23
(@exam23)
Trusted Member
Joined: 3 years ago
 

I have one single GTX980ti / egpu.  I plugged it into the lower right have corner on the nMP thunderbolt port.  And still when rendering, I am not seeing a difference.  Can someone provide guidance? 

Pending: Add my system information and expected eGPU configuration to my signature to give context to my posts


ReplyQuote
exam23
(@exam23)
Trusted Member
Joined: 3 years ago
 

I followed the instructions to ensure the egpu is at BusO, no luck in forcing FCPX into the egpu.  I am currently using 10.2.1   Can someone shed more light into the matter?  The egpu is running great on 3d for sure.  Just trying to get FCPX to use the egpu has been painful...

Pending: Add my system information and expected eGPU configuration to my signature to give context to my posts


ReplyQuote
imcalledandy
(@imcalledandy)
Estimable Member
Joined: 3 years ago
 

FCPX is not going to really benefit from an NVIDIA card / CUDA as it uses Open CL. Granted, the 980Ti does have Open CL but I believe the architecture of FCPX is more suited to the internal cards that ship with the Mac Pro.

FYI - I have my first eGPU connected to Bus-0 Lower left

2013 MacPro 2xD700 [email protected] (AKiTiO Thunder2) + macOS 10.13.3
2017 TB MBP 2xGTX1080Ti-TB3 (Akitio Node) + macOS 10.13.4


ReplyQuote
benr
 benr
(@benr)
Eminent Member
Joined: 3 years ago
 

Hey Andy, any chance you can try daisy chaining the two cards at some point, see if you get the same CUDA and Octane/Cycles performance? Also, do you use DaVinci Resolve? Would be curious to know about that too if you do.

Pending: Add my system information and expected eGPU configuration to my signature to give context to my posts


ReplyQuote
exam23
(@exam23)
Trusted Member
Joined: 3 years ago
 

 Thanks, imcalledandy...

I gave up on the GTX 980ti. You are absolutely correct with regards to FCPX not really going to utilize the GTX.  In fact, when I ran the gfx software to monitor the activities with rendering and exporting, GTX didn't even move/register.  Since I am not a gamer and really focus on 3D graphics, I gave up on the GTX.

Then, I threw in the RX480. The install process was pretty much the same as the GTX.  Except there were a couple of lines where I had to change to force OSX to see the RX480. To my surprise, it was a very easy process. Then the disappointment came. First, with Cinebench, ZERO difference what so ever and no improvements. Then with the benchmark with Heaven, FPS was nowhere even close to the GTX. Finally, then Xmas came. Remembering the purpose of my project is to create more efficiency and to add more processing power to FCPX. First, I exported the bruceX for benchmarked. With the dual D300, it was in the area of 31-32secs. To my surprise, now it was coming back at 20-21secs. I was shocked! Then I opened up XRG to monitor the activities of the GPUs and reran BruceX. I thought the improvement came because of OSX finally using the RX480.  Yes and no, it did see the RX480, but it also utilized the dual D300 along with the Rx480. I know what some of you are going to say. Thaz it, what a small improvement. In reality, that is about a 30% improvement. I for one am pretty happy about it. To be able to utilize all three of my GPSs. Then I continued with my testing and monitor the gpu activities. Overall, it's a huge improvement for me with FCPX, which I am very happy about.  I'm not going to game on the Mac, so I could really care less about the 3D.  I really appreciate everyone's help cause I know I have posted a lot.

 

New question:

Can I get a PCIe Raiser and put two rx480 into the Akitio box?  I'm pretty sure I can run two RX480 off the same power supply.  But what about two GPUs off one Akitio box? I'm getting greedy now for sure... 

Pending: Add my system information and expected eGPU configuration to my signature to give context to my posts


ReplyQuote
Yukikaze
(@yukikaze)
Prominent Member Moderator
Joined: 3 years ago
 

In theory, you can get an active PCIe riser with a PCIe switch and it should work. In practice, I don't think anyone actually tried.

Want to output [email protected] out of an old system on the cheap? Read here.
Give your Node Pro a second Thunderbolt3 controller for reliable peripherals by re-using a TB3 dock (~50$).

"Always listen to experts. They'll tell you what can't be done, and why. Then do it."- Robert A. Heinlein, "Time Enough for Love."


ReplyQuote
benr
 benr
(@benr)
Eminent Member
Joined: 3 years ago
 

Hey Yukikaze,

Do you have any examples of the PCIe switch/riser you're mentioning? I'm interested in trying this but not 100% what products you're referring to.

Thanks!

Pending: Add my system information and expected eGPU configuration to my signature to give context to my posts


ReplyQuote
Yukikaze
(@yukikaze)
Prominent Member Moderator
Joined: 3 years ago
 

They aren't easy to find. I've encountered them at work before, typically around prototype hardware, and also in servers. Rack-mounted servers use there to re-orient a single vertical PCIe slot on the motherboard into multiple slots horizontal, in parallel with the motherboard, to reduce the server height.

I tried to take a look at ebay, and saw exactly one (aside of cheap Chinese ones with PCIe x1 slots, like this). This is from a Supermicro server. The chip under the green heatsink is the PCIe switch that multiplexes traffic downstream to the three PCIe slots. No idea what the blue slot is. No idea if that will work for you in any way. It should work with any PCIe slot, as PCIe defines a switch standard, but that is in theory, as I have no idea if there's any Supermicro magic going on behind the scenes.

The ones we use at work are pretty damn expensive, and usually manufactured on order, so I am not sure they are "consumer" devices in the sense that you can buy them off the shelf.

Want to output [email protected] out of an old system on the cheap? Read here.
Give your Node Pro a second Thunderbolt3 controller for reliable peripherals by re-using a TB3 dock (~50$).

"Always listen to experts. They'll tell you what can't be done, and why. Then do it."- Robert A. Heinlein, "Time Enough for Love."


ReplyQuote
imcalledandy
(@imcalledandy)
Estimable Member
Joined: 3 years ago
 
Posted by: benr

Hey Andy, any chance you can try daisy chaining the two cards at some point, see if you get the same CUDA and Octane/Cycles performance? Also, do you use DaVinci Resolve? Would be curious to know about that too if you do.

Hey @benr I don't use DaVinci Resolve I'm afraid but I can confirm that daisy chaining the eGPUs works!

Performance in the IPR seems to be pretty good too.

However, when I gave Cycles a really heavily displaced scene, rendering at 2000 x 2000 it appeared to fall back to the CPU rather than Multi-CUDA. I wonder if it couldn't see all the available VRAM?

Does anyone know how many eGPUs we could potentially hook up then? I'm guessing at least 2 more...

2013 MacPro 2xD700 [email protected] (AKiTiO Thunder2) + macOS 10.13.3
2017 TB MBP 2xGTX1080Ti-TB3 (Akitio Node) + macOS 10.13.4


ReplyQuote
imcalledandy
(@imcalledandy)
Estimable Member
Joined: 3 years ago
 

It definitely appears the eGPUs aren't using all the available VRAM for rendering when daisy chained, so I'll revert to my previous setup.

 

cuda error
cuda out of memory

2013 MacPro 2xD700 [email protected] (AKiTiO Thunder2) + macOS 10.13.3
2017 TB MBP 2xGTX1080Ti-TB3 (Akitio Node) + macOS 10.13.4


ReplyQuote
benr
 benr
(@benr)
Eminent Member
Joined: 3 years ago
 

Brilliant, many thanks for testing, Andy.

Pending: Add my system information and expected eGPU configuration to my signature to give context to my posts


ReplyQuote
imcalledandy
(@imcalledandy)
Estimable Member
Joined: 3 years ago
 

@benr - I now believe that I'd set the level of displacement far too high because Cycles switched to CPU rendering when I had the same scene in my usual configuration. I'll test again with lower settings next week and let you know. 

2013 MacPro 2xD700 [email protected] (AKiTiO Thunder2) + macOS 10.13.3
2017 TB MBP 2xGTX1080Ti-TB3 (Akitio Node) + macOS 10.13.4


ReplyQuote
barefeats
(@barefeats)
Active Member
Joined: 3 years ago
 

I have dual eGPUs running on the 2016 MacBook Pro. However, when I hook the same two eGPUs to the 2013 Mac Pro (TB2 Bus 0 and 1), it won't boot up. It boots fine if I connect one eGPU at a time. 

What am I missing here?

BTW, I'm running the 1080 Ti in one eGPU and the 1080 in the other eGPU.

Pending: Add my system information and expected eGPU configuration to my signature to give context to my posts


ReplyQuote
 of  2