A Call For Measurements: Isolating the Thunderbolt Effect.  

Page 1 / 6 Next
  RSS

Yukikaze
(@yukikaze)
Prominent Member Moderator
Joined: 2 years  ago
Posts: 808
April 24, 2017 10:22 pm  

I was thinking about this, and I think I came up with a way to test the effect of Thunderbolt 2 and/or Thunderbolt 3 on GPU performance. I have no means to carry out these measurements because I do not own a desktop, but I was wondering if someone would be willing to pick up this glove. We might need to raise a little money for this effort as it requires some hardware most people won't have, but I am sure the community can get together to figure this out.

Here is my proposal:

Requirements:

  1. A modern, Windows-based gaming tower: Something like a i7-6700K, 16GB of RAM and a GTX1080, or another high-end gaming card.
  2. A TB3 eGPU enclosure with updated firmware.
  3. A PCIe to Thunderbolt3 add-on card. This is the part that people that own both an eGPU+Laptop and a gaming tower likely do not have. Alternatively, someone who has a desktop with a Thunderbolt3 port on the motherboard. These motherboards do exist, but they are very rare.
  4. A 4K monitor (to test a wide selection of resolutions from 1080p, through 1440p/1600p and up to the full 2160p).

The idea:

Measure the same system, with the same CPU, the same memory, the same everything. The only variable will be the connection of the GPU to the system. In the first case, it will be connected directly to the PCIe x16 slot. In the second case, it will be placed in the eGPU enclosure that will connect to the TB3 add-on card placed in the same slot (a bonus measurement would be connecting the TB add-on card to a slot that goes to the PCH, as opposed to the CPU directly).

The end result should be as good of an isolation of the Thunderbolt3 performance reduction factor as we can hope for. Bonus points if the same can be repeated with a TB2 setup to see the difference between the two. This is better than comparing a desktop vs a laptop, because it keeps more variables the same, and it is better than comparing a dGPU with an eGPU because we remove potential CPU bottlenecks from the result (they are a valuable question by their own right, but that is not what I want to measure here).

Thoughts?

My eGPU Zoo - Link to my Implementations.
Want to output [email protected] out of an old system on the cheap? Read here.
"Always listen to experts. They'll tell you what can't be done, and why. Then do it."- Robert A. Heinlein, "Time Enough for Love."


theitsage, Fred., 3RYL and 1 people liked
ReplyQuote
sicily428
(@sicily428)
Trusted Member
Joined: 2 years  ago
Posts: 96
April 24, 2017 10:28 pm  
Posted by: Yukikaze

 

I was thinking about this, and I think I came up with a way to test the effect of Thunderbolt 2 and/or Thunderbolt 3 on GPU performance. I have no means to carry out these measurements because I do not own a desktop, but I was wondering if someone would be willing to pick up this glove. We might need to raise a little money for this effort as it requires some hardware most people won't have, but I am sure the community can get together to figure this out.

Here is my proposal:

Requirements:

  1. A modern, Windows-based gaming tower: Something like a i7-6700K, 16GB of RAM and a GTX1080, or another high-end gaming card.
  2. A TB3 eGPU enclosure with updated firmware.
  3. A PCIe to Thunderbolt3 add-on card. This is the part that people that own both an eGPU+Laptop and a gaming tower likely do not have.

The idea:

Measure the same system, with the same CPU, the same memory, the same everything. The only variable will be the connection of the GPU to the system. In the first case, it will be connected directly to the PCIe x16 slot. In the second case, it will be placed in the eGPU enclosure that will connect to the TB3 add-on card placed in the host.

The end result should be as good of an isolation of the Thunderbolt3 performance reduction factor as we can hope for. Bonus points if the same can be repeated with a TB2 setup to see the difference between the two. This is better than comparing a desktop vs a laptop, because it keeps more variables the same, and it is better than comparing a dGPU with an eGPU because we remove potential CPU bottlenecks from the result (they are a valuable question by their own right, but that is not what I want to measure here).

Thoughts?

   

i think that also a clevo p750dm2-g/p775dm3-g or an eurocom tornado f5/msi 16L13 with a cpu i7-6700k/i7-7700k and tb3 could make this test. or 6820hk/7820hk overclocked?


ReplyQuote
Yukikaze
(@yukikaze)
Prominent Member Moderator
Joined: 2 years  ago
Posts: 808
April 24, 2017 10:30 pm  

As long as the dGPU and the eGPU use the same exact chip at the same exact clocks (or can be over/under clocked to match), then this can also be done with one of the "desktop replacement" laptops, yup.

My eGPU Zoo - Link to my Implementations.
Want to output [email protected] out of an old system on the cheap? Read here.
"Always listen to experts. They'll tell you what can't be done, and why. Then do it."- Robert A. Heinlein, "Time Enough for Love."


ReplyQuote
enjoy
(@enjoy)
Reputable Member
Joined: 2 years  ago
Posts: 373
April 24, 2017 10:39 pm  

This is the same as @theitsage done with Alienware R3 13, he added 1060 as eGPU to laptop with dGPU 1060 so maybe it is waste of time and money, my thread about PCI Express vs. eGPU is also the other ways to find how the performance drop is, its not accurate, but users must know that with External Display the performance drop is 20% for TB1, TB2 and TB3 and that the Internal Display performance drop is based of the GPU, the Thunderbolt number because the bandwidth goes in two directions not in one like with External Display (Monitor)....

I'am sure that the test you want will show the same results + or - 5% less or more 🙂

ϟ AKiTiO Thunder2 + EVGA GTX 1060 6GB SC Gaming (macOS Sierra 10.12.4 and Windows 10)
 MacBook Pro (Retina, 15-inch, Later 2013) 3.2GHz Quad Core Intel i7-4750HQ / 8 GB 1600 MHz DDR3 / 256GB SSD + 1TB
mini eGPUPCI Express vs. ThunderboltMac CAN gameGaming Laptops vs. MacBook Pro with eGPU


ReplyQuote
Yukikaze
(@yukikaze)
Prominent Member Moderator
Joined: 2 years  ago
Posts: 808
April 24, 2017 10:50 pm  

My reason of thinking about this is twofold: One is purely academic. I want to see this measured on a gaming desktop, where the CPU is of no issue. Sort of a fresh look at the same question.

The second is the notebookcheck review of the Core, where the GTX1070 (or higher) eGPU is barely of any effect over a dGPU GTX1060. That doesn't seem to line up with a 20% drop in performance, as the GTX1070 itself is more than 20% faster than the 1060, and the 1080 and 1080Ti are far, far more powerful.

In addition, with such a benchmarking setup, you can test ANY card to establish the bottleneck effect, whereas comparing a dGPU to an eGPU is limited exactly to the dGPU chip that you have. That in and of itself is valuable information.

My eGPU Zoo - Link to my Implementations.
Want to output [email protected] out of an old system on the cheap? Read here.
"Always listen to experts. They'll tell you what can't be done, and why. Then do it."- Robert A. Heinlein, "Time Enough for Love."


ReplyQuote
nando4
(@nando4)
Noble Member Admin
Joined: 2 years  ago
Posts: 1802
April 24, 2017 10:57 pm  

I'd nominate benchmarks comparing TB3-CPU (15" MBP) architecture versus TB3-PCH (every other notebook).  That's the last holdout that may explain TB3 underperformance.  There we'd need back-to-back TB3 gaming benchmarks on two CPU+eGPU alike systems but one would be a 15" MBP.   There we'd isolate the effect of the device shared PCH and possible DMI bottlenecking/latency on TB3-PCH Intel mobile architecture.

If TB3-CPU benchmarks are not significantly better than TB3-PCH then it would point to the TB3 controllers having performance issues. TB3-CPU architecture matches desktop systems' where the PCIe bus is attached on the CPU, example from here:

Skylake-S needs a new socket, new chipset and new motherboard. The socket is called LGA 1151 and has a single pin more than Haswell, the company’s current generation Core architecture. The new desktop processor supports both DDR4 1.2V and DDR3L 1.35V and it will be up to motherboard manufacturers to support one or the other.

The chipset that will let you overclock Skylake-S is called Z170 and it comes with 1 x 16 plus 2x8 or 1x8 plus 2x4 PCIe express 3.0 combination...

SkylakeS

The difference between TB3-CPU and  TB3-PCH notebook/mobile architecture is shown at https://egpu.io/macos-pascal-drivers-gtx-1080-ti-2016-macbook-pro/ , with only the former matching favorably compared to a desktop system.

eGPU Setup 1.35    •    eGPU Port Bandwidth Reference Table    •    Several builds
2015 15" Dell Precision 7510 M1000M + macOS 10.14 & Win10


De5tr0yer and Fred. liked
ReplyQuote
enjoy
(@enjoy)
Reputable Member
Joined: 2 years  ago
Posts: 373
April 24, 2017 11:20 pm  

http://www.ultrabookreview.com/10761-razer-core-review/#a5

Performance and daily experience

For the purpose of my testing, I compared the results of my current desktop with the Razer Blade 2016 attached to the Core.  Both used the same video card, an Nvidia GTX 970.  The desktop has a Core i5-3570K (with stock clock speeds) processor and 16GB of RAM, while the Blade gets an Intel Core i7-6700HQ processor with 16 GB of RAM as well.  I also tested with the XPS 15 which has an Nvidia GTX 960m paired with the i5-6300HQ.

Some time after I initially wrote this article, I was able to repeat the Razer Blade and XPS 15 testing using a reference GTX 1070 and then again with the Asus Strix GTX 1080.  My desktop is dismantled now so I couldn’t repeat the tests for that one.

All benchmarks used a QNIX 2560 x 1440 px monitor, and the laptop’s display was disabled when attached to the Core.   I was able to overclock my monitor to 96Hz pretty easily, so we could see actual framerates above 60(except for Fallout below native resolution).

  Desktop w/970 Blade + Core+970 Blade + Core+1070 Blade + Core+1080 Blade XPS 15 + Core+970 XPS 15 + Core+1070 XPS 15 + Core+1080 XPS 15
3DMark – Fire Strike 9168 8742 11554  12797 6513 7433* 8266*  11607 3561
3DMark – Sky Diver 21371 22692 26401  27315 18741 17746* 18599*  22909 9949
Fallout 4 – Outside the Corvega plant entrance with a battle
  Desktop w/970 Blade + Core+970 Blade + Core+1070 Blade + Core+1080 Blade XPS 15 + Core+970 XPS 15 + Core+1070 XPS 15 + Core+1080 XPS 15
Ultra, 2560 x 1440 px 43-53 fps 38-48 fps 48-63 fps  55-65 fps N/A 32-41 fps* 44-60 fps*  49-60 fps N/A
Ultra, 1920 x 1080 px 51-62 fps 52-60 fps 54-60+ fps  54-60+ fps 42-52 fps 43-60 fps* 48-60 fps*  54-60 fps 23-32
Witcher 3 – Walking around the grounds on the first tutorial 
  Desktop w/970 Blade + Core+970 Blade + Core+1070 Blade + Core+1080 Blade XPS 15 + Core+970 XPS 15 + Core+1070 XPS 15 + Core+1080 XPS 15
Ultra, 2560 x 1440 px 34-38 fps 24-26 fps 34-38 fps  40-48 fps N/A 20-22 fps* 33-36 fps*  40-48 fps N/A
High, 2560 x 1440 px 42-47 fps 34-38 fps 48-50 fps  57-60 fps N/A 29-31 fps* 48-50 fps*  57-60 fps N/A
Ultra, 1920 x 1080 px 45-53 fps 28-31 fps 41-46 fps  48-54 fps 28-31 fps 22-25 fps* 42-46 fps*  46-56 fps 17-20
High, 1920 x 1080 px 60 fps 45-53 fps 60 fps  60 fps 39-41 fps 38-40 fps* 60 fps*  60 fps 23-25
Dragon Age Inquisition – Battle nearby a camp at the beginning 
  Desktop w/970 Blade + Core+970 Blade + Core+1070 Blade + Core+1080 Blade XPS 15 + Core+970 XPS 15 + Core+1070 XPS 15 + Core+1080 XPS 15
Ultra, 2560 x 1440 px 32-35 fps 22-28 fps 32-40 fps  45-48 fps N/A 23-28 fps* 22-28 fps*  43-48 fps N/A
High, 2560 x 1440 px 53-61 fps 55-60 fps 57-64 fps  67-75 fps N/A 42-47 fps* 40-48 fps*  63-73 fps N/A
Ultra, 1920 x 1080 px 46-54 fps 35-42 fps 48-56 fps  58-63 fps 27-31 fps 35-40 fps* 35-42 fps*  52-64 fps 17-20
High, 1920 x 1080 px 75-89 fps 60-68 fps 73-90 fps  80-98 fps 48-52 fps 55-64 fps* 55-65 fps*  76-88 fps 26-33
Crysis 3 – Opening mission 
  Desktop w/970 Blade + Core+970 Blade + Core+1070 Blade + Core+1080 Blade XPS 15 + Core+970 XPS 15 + Core+1070 XPS 15 + Core+1080 XPS 15
Very High, 2560 x 1440 px 31-45 fps 25-45 fps 35-65 fps  40-75 fps N/A 25-40 fps* 20-50 fps*  40-65 fps N/A
High, 2560 x 1440 px 43-65 fps 34-65 fps 49-96 fps  57-110 fps N/A 35-60 fps* 33-75 fps*  57-95 fps N/A
Very High, 1920 x 1080 px 46-67 fps 40-65 fps 46-85 fps  52-100 fps 29-42 fps 37-60 fps* 29-65 fps*  50-85 fps 17-27
High, 1920 x 1080 px 65-96 fps 53-85 fps 59-96 fps  71-120 fps 44-60 fps 46-82 fps* 45-80 fps*  65-103 fps 25-40

*readings on the XPS were taken with bios version 1.2, which was found to(sometimes) hinder performance of the GPU slightly.  I can’t go back and remeasure them since I no longer have that GPU.  The GTX 1080 readings were taken on bios 1.1.19, which is much more consistent.

I also added a few logs showing frequencies and temperatures of the Razer Core connected to the Razer Blade and my 1440p monitor.

As you can clearly see, the Razer Core certainly does a good job boosting the performance over the Razer Blade alone.  If you have the new Nvidia GTX 1070 or 1080, it improves  those graphics settings even more to get some really decent fps on AAA titles.  Even the 970 shows some significant signs of improvement over the 970m, if you don’t want to spend that kind of money.

I’ve been doing a lot of research since originally writing this section and there have been a number of things I found out over the past few weeks.  The first thing to mention is the performance hit the Core takes as opposed to using a desktop.  If you compare my desktop+970 readings to the Razer Blade+Core+970, you’ll notice a significant fps drop.  The CPUs in both machines offer very similar performance scores in benchmark tests, so the only thing I can chalk off the performance hit to is losses through the thunderbolt connections/driver issues.  So if you’re planning on using a particular graphics card and are comparing it to desktop benchmarks, expect to see a 10-15% performance drop right off the bat.

Another thing I want to mention is the performance with the newest Nvidia GTX 10xx series cards.  As you can see, my Firestrike scores are much higher than the 970, which is what you would expect.  But if you compare those results with Firestrike benchmarks taken in people’s desktops with the 1070 or 1080, you’ll see a much larger performance drop(nearly 30%).  On top of that, the Firestrike performance of the 1080 is only 10% higher than the 1070.  At first I thought maybe the 1080 is being bottlenecked by the Core somehow.  I ran another test though(which I’ll get to shortly) and determined it wasn’t the bandwidth limit being reached.  My only conclusion is that the CPU in the Razer Blade is skewing the Firestrike scores to be really low.  Here’s why.

The score is actually calculated off of the weighted average of three scores: Graphics(GPU only), Physics(CPU) and combined.  Since the CPU is the same in all tests, the Physics score is unchanged.  And since the Graphics score has such a large increase, but the CPU score stays the same, the average is going to be lower when compared to someone testing the card in a machine with a top of the line Skylake desktop CPU.  In other words, a 20% increase in graphics performance equates to roughly a 10% increase in the Firestrike(standard) score.  This would be a good place to use the Firestrike Extreme score as a comparison since it relies more on graphics performance at high resolutions.

The good news is this isn’t really a factor with many titles since they are more driven by the GPU than the CPU.  You can definitely see a more proportional performance jump between the 1070 and the 1080, when looking at the fps measurements I got in the games I tested.  It’s certainly better than the 970 readings and much much better than using the 970m in the Razer Blade.  To the point, I’m happy with the performance of both the 1070 and 1080 in the Core.

So how did I determine that I wasn’t hitting the bandwidth limit of the Core?  Well since I got the Asus Strix 1080, I decided to overclock it and see if I could improve things.  My original Firestrike score was 12797(Graphics: 16764, Physics: 9603).  I overclokced the GPU boost clock to 1973 Mhz, the memory to 11.2Ghz and let the voltage alone.  My Firestrike score jumped to 13294(Graphics: 17785 Physics: 9610), a 6% gain from a ~10% overclock – not bad!  I didn’t go crazy testing games with the overclock but I did gain 3-4 fps on the Witcher on Ultra QHD settings.

In regards to the XPS 15, you’ll notice my benchmarks are a little all over the place between GPUs.  It’s because the system is a little sensitive to a number of factors(which I describe in more detail below).  In short, if you use the wrong bios your performance will differ.  For the 970 and 1070 tests I was on bios 1.2 but for the 1080 I was on bios version 1.1.19.  The main thing it allows is for more consistently higher GPU usage, whereas before it was erratic and sometimes throttled for whatever reason.  I left the XPS 15 + 970 and 1070 benchmarks on the table but take them with a grain of salt.

You might have noticed that even with the 1080 and the better bios, the XPS 15 still has a lower Firestrike score and some of the gaming benchmarks are a little lower than with the Razer Blade.  I looked into it and it’s definitely because my XPS 15 is the i5 version.  I’ve been collaborating with Doug who also recently got a Core.  He has the i7 version of the XPS 15 and got a Firestrike score more similar to what I got with the Razer Blade.  The physics portion of his score is 9973, which is more consistent with the Razer Blade.  The physics portion of my score was only 6165.  So if you’re looking for the bext performmance and want an XPS 15, aim for the i7 version.  It’s still a buggy connection though, which I describe more in the section below.

ϟ AKiTiO Thunder2 + EVGA GTX 1060 6GB SC Gaming (macOS Sierra 10.12.4 and Windows 10)
 MacBook Pro (Retina, 15-inch, Later 2013) 3.2GHz Quad Core Intel i7-4750HQ / 8 GB 1600 MHz DDR3 / 256GB SSD + 1TB
mini eGPUPCI Express vs. ThunderboltMac CAN gameGaming Laptops vs. MacBook Pro with eGPU


ReplyQuote
sicily428
(@sicily428)
Trusted Member
Joined: 2 years  ago
Posts: 96
April 25, 2017 10:28 am  

clevo p750dm works with razer core

http://forum.notebookreview.com/threads/clevo-egpu-n850hk1-with-tb3.801852/

 

is there tb3-cpu or a tb3-pch in a hp zbook g3 (2xtb3) ?


ReplyQuote
ed_co
(@ed_co)
Reputable Member
Joined: 2 years  ago
Posts: 350
April 25, 2017 1:19 pm  
Posted by: sicily428

is there tb3-cpu or a tb3-pch in a hp zbook g3 (2xtb3) ?

   

It looks nando4 says, just the mbp pro 15" 2016 has the tb3-cpu configuration

mid-2017 Macbook Pro RP560 + MSI GTX1080Ti GAMING [email protected] (Mantiz Venus) + macOS10.13.5 & Win10


ReplyQuote
theitsage
(@itsage)
Famed Member Admin
Joined: 2 years  ago
Posts: 3075
April 25, 2017 1:35 pm  

Would this ASUS ThunderboltEX3 card work on an Alienware Area 51 R2? If so, I can get this project underway. I read the AW Area 51 R2 uses a custom X99 motherboard. At one point, the BIOS even had Thunderbolt support but it was beta and later removed. Here's the specs of this PC:

  • CPU: i7-5820K (overclocked to 4.2GHz)
  • RAM: 4 x 8GB DDR4 2400MHz
  • GPU: 2 x GTX 980 Ti Hybrid SLI
  • Boot Drive: 2 x 575GB SSD in RAID-0

Best ultrabooks for eGPU use

eGPU enclosure buying guide

82 external GPU build guides


Yukikaze and nando4 liked
ReplyQuote
Yukikaze
(@yukikaze)
Prominent Member Moderator
Joined: 2 years  ago
Posts: 808
April 25, 2017 2:16 pm  

It should work in any desktop These expansion cards require an additional motherboard header to work. It would be an awesome benchmarking project! We'd need actual game benchmarks too, however, not just synthetics (these seems to perform better than games in some cases, so it won't be accurate that way). I assume you could put different video cards through that test? R9 Fury (X)? GTX980Ti? Do you have a GTX10XX card?

EDIT: I am getting greedy here (excitement is getting the better of me, sorry), but could you also compare the difference between slots? One time the TB3 card should be in the main PCIe slot (connected to the CPU). In a second scenario, the card should be in a PCIe slot that is connected to the PCH. I assume the X99 board should have one of those too.

My eGPU Zoo - Link to my Implementations.
Want to output [email protected] out of an old system on the cheap? Read here.
"Always listen to experts. They'll tell you what can't be done, and why. Then do it."- Robert A. Heinlein, "Time Enough for Love."


ReplyQuote
theitsage
(@itsage)
Famed Member Admin
Joined: 2 years  ago
Posts: 3075
April 25, 2017 2:54 pm  

ASUS ThunderboltEX 3 PCIe card is enroute. I dug around for more information on the AW Area 51 R2 motherboard and can confirm it has the Thunderbolt header connector (#12) to connect to this expansion card.

@wimpzilla gifted me some games through Steam so I will be able to run those in-game benchmarks for this project. The two GPUs I'm planning to use is R9 Fury X and GTX 1080 Ti. I don't have a 4K display however.

Best ultrabooks for eGPU use

eGPU enclosure buying guide

82 external GPU build guides


ikir and nando4 liked
ReplyQuote
Yukikaze
(@yukikaze)
Prominent Member Moderator
Joined: 2 years  ago
Posts: 808
April 25, 2017 3:04 pm  

The PCIe x4 slot is probably connected to the PCH. That would allow to test direct-to-CPU and via-PCH configurations too.

This is going to result in high quality data. The scientist in me is extra-happy 🙂

EDIT: I forgot about that extra TB header thing. Good to know that the Alienware motherboard has the header, but does it have one that will work with the expansion card? The Asus site indicates a 9-pin connector. On the motherboard,  I am seeing a 5-pin one. What am I missing here?

My eGPU Zoo - Link to my Implementations.
Want to output [email protected] out of an old system on the cheap? Read here.
"Always listen to experts. They'll tell you what can't be done, and why. Then do it."- Robert A. Heinlein, "Time Enough for Love."


theitsage liked
ReplyQuote
Yukikaze
(@yukikaze)
Prominent Member Moderator
Joined: 2 years  ago
Posts: 808
April 25, 2017 3:12 pm  

That extra connector is weird. I wonder if it is actually required for PCIe-over-Thunderbolt to work, or is it used to provide USB connectivity to the TB chip by using one or two USB headers off the board (as the expansion card itself may not have a USB controller on it). I guess we'll find out, but it would suck if you were to buy that card and it won't work (although if it is off Amazon, you could return it without a hassle, so worth a shot, I guess).

EDIT #MAX_INT: I tried to look up people getting it to work on non-Asus motherboards and I am seeing a lot of conflicting results: Some people report it non-functional even on Asus boards, and I've seen people who claim to have it working on non-Asus boards. No idea at this point. It does look like you need the BIOS to be capable of selecting (via a menu) which PCIe slot has the Thunderbolt board, and it might not work in all slots.

EDIT #-1: It looks like the extra connector is a "GPIO" header. This isn't USB. This is likely not gonna work, then. We'd need someone with a motherboard off the supported list to be sure that it can work.

EDIT #0 (overflow jokes, heh): It looks like an MSI Thunderbolt expansion may work, but that thing (MSI Thunderbolt Plus) is a mythical creature that may have never been sold...

My eGPU Zoo - Link to my Implementations.
Want to output [email protected] out of an old system on the cheap? Read here.
"Always listen to experts. They'll tell you what can't be done, and why. Then do it."- Robert A. Heinlein, "Time Enough for Love."


ReplyQuote
theitsage
(@itsage)
Famed Member Admin
Joined: 2 years  ago
Posts: 3075
April 25, 2017 3:21 pm  

I think there's a Thunderbolt controller on the motherboard which communicates to the Thunderbolt PCIe card via this TB header cable. There's not much information on this particular setup. Here's one of the few pictures I found which show one end of the TB header cable has a 5-pin connector.

The i7-5820K has 28 PCIe lanes. When I tried triple RX 480 CrossFire setup, Slot 1 is 8x, Slot 3 is 16x, and Slot 5 is 4x. I'm excited to see what comes out of this project. Thank you @Yukikaze for the fun idea. I welcome all suggestions to get the best data out of this project as possible.

Best ultrabooks for eGPU use

eGPU enclosure buying guide

82 external GPU build guides


ReplyQuote
Yukikaze
(@yukikaze)
Prominent Member Moderator
Joined: 2 years  ago
Posts: 808
April 25, 2017 3:26 pm  

Hmm. That wiring setup might actually work, as it seems to show how the wiring should be done from the 9-pin to the 5-pin connector.

But I don't think there is a TB controller on the board. That wiring is definitely not good enough for high-rate signaling. Seeing my link above to the "GPIO header" requirement, I suspect that this is a GPIO control side-channel, used to talk to and configure the TB controller on the expansion card.

I hope this works!

My eGPU Zoo - Link to my Implementations.
Want to output [email protected] out of an old system on the cheap? Read here.
"Always listen to experts. They'll tell you what can't be done, and why. Then do it."- Robert A. Heinlein, "Time Enough for Love."


ReplyQuote
nando4
(@nando4)
Noble Member Admin
Joined: 2 years  ago
Posts: 1802
April 25, 2017 3:44 pm  
Posted by: theitsage

 

I think there's a Thunderbolt controller on the motherboard which communicates to the Thunderbolt PCIe card via this TB header cable. There's not much information on this particular setup. Here's one of the few pictures I found which show one end of the TB header cable has a 5-pin connector.

The i7-5820K has 28 PCIe lanes. When I tried triple RX 480 CrossFire setup, Slot 1 is 8x, Slot 3 is 16x, and Slot 5 is 4x. I'm excited to see what comes out of this project. Thank you @Yukikaze for the fun idea. I welcome all suggestions to get the best data out of this project as possible.

   

Awesome test bed to give vital TB3 performance data.  With your motherboard,  can use hwinfo64 to identify the TB3 PCIe slot attachment type:

  • TB3-CPU attachment will show up under 0:1.0, 0:1.1 or 0:1.2 PCIe slots . The motherboard manual will specify these high performance slots to be used for video cards.
  • TB3-PCH attachment will show up under 0:1b.x, 0:1c.x or 0:1d.x PCIe slots. They are hex so might also be shown as 0:27.x, 0:28.x, 0:29.x. x can be any number.

Also, the motherboard's x4 slot will come in handy to get x4 32Gbps PCIe CUDA-Z bandwidth vs 22Gbps TB3 CUDA-Z bandwidth.

eGPU Setup 1.35    •    eGPU Port Bandwidth Reference Table    •    Several builds
2015 15" Dell Precision 7510 M1000M + macOS 10.14 & Win10


theitsage liked
ReplyQuote
SaturnV
(@zach)
Eminent Member
Joined: 2 years  ago
Posts: 23
April 25, 2017 4:30 pm  

BareFeats did a test comparing this very thing (loosely), in terms of IF bandwidth affects the outcome of using professional apps. I know this particular thread/discussion/scenario is more exacting, but it does give insight into TB2 vs TB3 and internal PCIe performances. 

Does Bandwidth matter?

As of right now, bandwidth doesn't make much of a difference, if at all, for the professional applications that were tested. 


ReplyQuote
Yukikaze
(@yukikaze)
Prominent Member Moderator
Joined: 2 years  ago
Posts: 808
April 25, 2017 7:12 pm  

Yeah, our current "working assumption" is that for professional offloads where there is little data input/output to the device (In other words, in "send data out, crunch it, copy results" kind of workloads), there should be little to no effect between the host-to-device (and back) bandwidth and the device performance. It is good to see some data corroborating that as fact.

My eGPU Zoo - Link to my Implementations.
Want to output [email protected] out of an old system on the cheap? Read here.
"Always listen to experts. They'll tell you what can't be done, and why. Then do it."- Robert A. Heinlein, "Time Enough for Love."


ReplyQuote
theitsage
(@itsage)
Famed Member Admin
Joined: 2 years  ago
Posts: 3075
April 28, 2017 2:06 pm  

It was close but did not work. All components plugged in where they should and the AKiTiO Node even powered on when connected to the Thunderbolt 3 port. Intel Thunderbolt Software shows no communication however.

I found older BIOS (A02, A04) which had support for Thunderbolt. This Alienware Area 51 R2 is currently running A10 and can't be downgraded through Windows. I'm not familiar with flashing BIOS through DOS.

Best ultrabooks for eGPU use

eGPU enclosure buying guide

82 external GPU build guides


gtosi and nando4 liked
ReplyQuote
nando4
(@nando4)
Noble Member Admin
Joined: 2 years  ago
Posts: 1802
April 28, 2017 2:22 pm  
Posted by: theitsage

 I found older BIOS (A02, A04) which had support for Thunderbolt. This Alienware Area 51 R2 is currently running A10 and can't be downgraded through Windows. I'm not familiar with flashing BIOS through DOS.

 

1. Use Rufus to create a FreeDOS USB stick. Set Rufus for MBR boot and use a FAT32 format.

2. Copy https://downloads.dell.com/FOLDER02794314M/1/Alienware_Area_51_R2_A02.exe to the USB stick.

3. Reboot and hold F12 or whatever the key is to manually select boot drive. Point to USB stick.

4. At C:\> DOS prompt type Alienware_Area_51_R2_A02 . If nothing happens or 'bad command or filename' then type 'dir' to identify the filename, then type it.

If have no luck downgrading to A.02 as some vendors prevent rollbaack, then consider fishing through the UEFI variables of your A.10 BIOS to see if can re-enable Thunderbolt. Details of how to do that are here.

eGPU Setup 1.35    •    eGPU Port Bandwidth Reference Table    •    Several builds
2015 15" Dell Precision 7510 M1000M + macOS 10.14 & Win10


ReplyQuote
theitsage
(@itsage)
Famed Member Admin
Joined: 2 years  ago
Posts: 3075
April 28, 2017 2:49 pm  

Thank you for instructions to flash the BIOS in DOS. I will give this a shot. I'll probably flash it back to A08 first using this approach because it's safer that way. 😀

Best ultrabooks for eGPU use

eGPU enclosure buying guide

82 external GPU build guides


ReplyQuote
VisViva
(@visviva)
Eminent Member
Joined: 2 years  ago
Posts: 21
April 30, 2017 4:08 am  

I want to share a benchmark that I recently came up with, which hopefully can reflex the access time difference between native PCIe and PCH-to-TB-to-PCIe connection. As I mentioned in this post, I used the a software to estimate host to device (H2D) and device to host (D2H) performance. This software is part of the samples in AMD APP SDK3.0 for OpenCL.

After reading its help and playing with it a bit, I ran it against my TB3 external GPU with Akitio Node (RX470) and internal dedicate GPU (Quadro M1000M). I ran it using the smallest size possible, so the time for this transfer may somewhat reflect its latency. Then I compare results from different cases. These cases are combinations of: <eGPU or dGPU>, <active or idle>     

(active/idle means weather the GPU is outputting desktop to a monitor. active is outputting; idle is not)

 

This detailed settings specified are:

  • --testType 3 (clEnqueue[Read,Write], prepinned)
  •  --numBytes 2048 (the min value for this option)
  • --numLoops 2000 (perform this 2000 times)
  • --printLog (show detailed results for each loop, instead of just average)
  • --deviceId <x> (specify which GPU do I want it to run)

In addition to this, I changed one figure in its source so it outputs 10 figures after decimal point (instead of the default value of 3).  

Then I convert the speed in GBPS to time for the 2KiB transmission in microseconds, and I found their medians as well as scatter plots.

 

                 name                  h2d_med    d2h_med
    _______________________________    _______    _______
    'AT_2KiB_type3_eGPU_active_out'    48.198     45.432 
    'AT_2KiB_type3_eGPU_idle_out'      49.383     46.222 

                 name                  h2d_med    d2h_med
    _______________________________    _______    _______

    'AT_2KiB_type3_dGPU_active_out'    37.136     38.716 
    'AT_2KiB_type3_dGPU_idle_out'      40.691     42.272 

 

 

It appears to me that on my system, the eGPU has longer access time, roughly 10 microseconds longer. (This article cites about 1.5 microsecond of latency per hop with thunderbolt, for reference)

However, this experiment is not well controlled in terms of variables, so everything here is strictly for reference only, and I cannot promise this method's confidence. Apparently there's the difference of NVidia and AMD; also the dGPU has x16 PCIe lanes, while the eGPU has only x4 PCIe lanes for TB controller. Furthermore, I do not know and I cannot guarantee if there are other source of uncertainties in terms of timing.

 

 

 

 

 

 

 


ReplyQuote
wimpzilla
(@wimpzilla)
Reputable Member
Joined: 2 years  ago
Posts: 392
April 30, 2017 7:58 am  

Hello,

You can simply confirm that using MSI afterburner and record the frametime while gaming, between both setups.

The lag itself is perceived as variation between the frames, example if 2 frames are at 10ms and the next at 30ms, its still ok, if the 1st are 10ms and the next is 70ms there is a problem and then you feel the lag.

If the 10ms latency added on each frame compare to the pci-e are kept stable, i would not see any issues, but when under stress, the bus will show it's limits so even if the card could achieve 30fps displayed, the frametime output will shows a real 25/28 fps feeling.

If you don't saw any difference into the frametime, then dunno where this latency apply and impact.

2012 13-inch Dell Latitude E6320 + R9 [email protected] (EXP GDC 8.4) + Win10
E=Mc²


ReplyQuote
theitsage
(@itsage)
Famed Member Admin
Joined: 2 years  ago
Posts: 3075
May 1, 2017 12:32 pm  

I tried downgrading the Alienware Area 51 R2's BIOS last night in DOS. It was a no-go.

Best ultrabooks for eGPU use

eGPU enclosure buying guide

82 external GPU build guides


ReplyQuote
nando4
(@nando4)
Noble Member Admin
Joined: 2 years  ago
Posts: 1802
May 1, 2017 12:38 pm  
Posted by: theitsage

 

I tried downgrading the Alienware Area 51 R2's BIOS last night in DOS. It was a no-go.  

Hmm.. worth scouting for a modded BIOS. In parallel, pls save a BIOS dump using Universal BIOS Backup and PM me a link to it (it will have DMI/system tatooing details you might not want public). I'll see if we can get extract a IFR file from it with hidden bios Thunderbolt UEFI variables that can be altered manually with a bootdisk.

eGPU Setup 1.35    •    eGPU Port Bandwidth Reference Table    •    Several builds
2015 15" Dell Precision 7510 M1000M + macOS 10.14 & Win10


theitsage liked
ReplyQuote
nando4
(@nando4)
Noble Member Admin
Joined: 2 years  ago
Posts: 1802
May 1, 2017 12:48 pm  

As an aside, an inkling suggesting TB3-CPU outperforms  TB3-PCH on similar CPU-specced systems. @theitsage's proposed testing would quantify this. From here:

 

Posted by: votehart407

 

I am using a Titan X Pascal and the results have been much better [late-2016 15" MBP] VS a Razer Blade and MSI GS 40. I cannot say for certain what is causing the increased performance but it has salvaged the EGPU set up for me. 

 

and here

 

Posted by: votehart407

 

MBP 2016 3dMark Ultra - 6190

GS40 3DMark Ultra - 5989

 10-15 fps improvements in most games.  

eGPU Setup 1.35    •    eGPU Port Bandwidth Reference Table    •    Several builds
2015 15" Dell Precision 7510 M1000M + macOS 10.14 & Win10


theitsage liked
ReplyQuote
Yukikaze
(@yukikaze)
Prominent Member Moderator
Joined: 2 years  ago
Posts: 808
May 1, 2017 3:47 pm  
That is an interesting result, nando. It may also mean that our famous "Thunderbolt performance hit" might actually be a "Route everything through the PCH" performance hit.

My eGPU Zoo - Link to my Implementations.
Want to output [email protected] out of an old system on the cheap? Read here.
"Always listen to experts. They'll tell you what can't be done, and why. Then do it."- Robert A. Heinlein, "Time Enough for Love."


ikir and nando4 liked
ReplyQuote
ed_co
(@ed_co)
Reputable Member
Joined: 2 years  ago
Posts: 350
May 1, 2017 4:49 pm  
Posted by: Yukikaze

 

That is an interesting result, nando. It may also mean that our famous "Thunderbolt performance hit" might actually be a "Route everything through the PCH" performance hit.

   

The courious thing is that there is no other computer out there that has TB3-CPU?? Who find that info, nando4?? Seriously, all the vendors are not taking advantage of the tb3 except apple?? Weird!!

mid-2017 Macbook Pro RP560 + MSI GTX1080Ti GAMING [email protected] (Mantiz Venus) + macOS10.13.5 & Win10


ReplyQuote
nando4
(@nando4)
Noble Member Admin
Joined: 2 years  ago
Posts: 1802
May 1, 2017 5:02 pm  
Posted by: ed_co

 

The courious thing is that there is no other computer out there that has TB3-CPU?? Who find that info, nando4?? Seriously, all the vendors are not taking advantage of the tb3 except apple?? Weird!!

   

Apple co-developed Thunderbolt with Intel. PC vendors have and continue to play catchup to Apple's lead.  We now have an anecdotal morsel to drive a quantitative analysis of TB3-CPU vs TB3-PCH performance.  theitsage's proposed rig the perfect testbed to do it on.

Here's hoping the CPUs are performance comparable,leaving it to be TB3 PCHvsCPU architecture differences. That's certainly wasn't the case in 2013-2015 15" MBPs whose second tier i7-quad (i7-48xxHQ/i7-49xxHQ) CPUs would run unlocked turbo multipliers with TDP ceilings well above the 47W of say 1st tier PC competitors (i7-47xxQM). 

3dmark11 physics results for both the PC (TB3-PCH) and 15" MBP (TB3-CPU)  would give a clearer picture on the CPU performance variable.

eGPU Setup 1.35    •    eGPU Port Bandwidth Reference Table    •    Several builds
2015 15" Dell Precision 7510 M1000M + macOS 10.14 & Win10


ReplyQuote
theitsage
(@itsage)
Famed Member Admin
Joined: 2 years  ago
Posts: 3075
May 1, 2017 5:13 pm  

If we can't find a workaround to enable Thunderbolt in my Alienware Area R2 BIOS, I may look into getting a Thunderbolt ready X99 motherboard.

Best ultrabooks for eGPU use

eGPU enclosure buying guide

82 external GPU build guides


ReplyQuote
enjoy
(@enjoy)
Reputable Member
Joined: 2 years  ago
Posts: 373
May 1, 2017 6:10 pm  

You must buy Gigabyte ATX DDR4 Intel LGA 1151 SATA E (6Gbit/s) Motherboards (GA-Z170X-Gaming 7)  https://www.amazon.com/dp/B012IBPJ6I/

It is off 32% and i watched Hackintosh builds with it, so you can try also macOS performance hit 🙂

ϟ AKiTiO Thunder2 + EVGA GTX 1060 6GB SC Gaming (macOS Sierra 10.12.4 and Windows 10)
 MacBook Pro (Retina, 15-inch, Later 2013) 3.2GHz Quad Core Intel i7-4750HQ / 8 GB 1600 MHz DDR3 / 256GB SSD + 1TB
mini eGPUPCI Express vs. ThunderboltMac CAN gameGaming Laptops vs. MacBook Pro with eGPU


ReplyQuote
theitsage
(@itsage)
Famed Member Admin
Joined: 2 years  ago
Posts: 3075
May 3, 2017 4:15 pm  

I'm going to swap the motherboard to an ASUS Rampage V Extreme. There are things with the original Alienware MB that I'm not very thrilled so this is a good reason to replace it. 😀

Best ultrabooks for eGPU use

eGPU enclosure buying guide

82 external GPU build guides


ReplyQuote
ed_co
(@ed_co)
Reputable Member
Joined: 2 years  ago
Posts: 350
May 3, 2017 4:28 pm  
Posted by: theitsage

 

I'm going to swap the motherboard to an ASUS Rampage V Extreme. There are things with the original Alienware MB that I'm not very thrilled so this is a good reason to replace it. 😀

   

I am sure you know, but just in case, be aware about the huge form factor, is not ATX!!

mid-2017 Macbook Pro RP560 + MSI GTX1080Ti GAMING [email protected] (Mantiz Venus) + macOS10.13.5 & Win10


ReplyQuote
theitsage
(@itsage)
Famed Member Admin
Joined: 2 years  ago
Posts: 3075
May 3, 2017 4:34 pm  

I checked and people with AW Area 51 R2 have been able to mount E-ATX boards without issues. The mounting screw layout matches and the case has more than enough room. There are a couple of cable clips in the way but they can be relocated easily.

Best ultrabooks for eGPU use

eGPU enclosure buying guide

82 external GPU build guides


ed_co liked
ReplyQuote
ed_co
(@ed_co)
Reputable Member
Joined: 2 years  ago
Posts: 350
May 3, 2017 4:46 pm  
Posted by: theitsage

 

I checked and people with AW Area 51 R2 have been able to mount E-ATX boards without issues. The mounting screw layout matches and the case has more than enough room. There are a couple of cable clips in the way but they can be relocated easily.

   

Beautiful, sounds like a plan!! Please, post what you do with photos!! We are DIY junkies!! 😉

mid-2017 Macbook Pro RP560 + MSI GTX1080Ti GAMING [email protected] (Mantiz Venus) + macOS10.13.5 & Win10


theitsage liked
ReplyQuote
Page 1 / 6 Next