10-Nov-2017: Powercolor Devil Box has a H2D firmware fix available here.

25-Aug-2017: The Aorus Gaming Box has a H2D firmware fix available here.

24-April-2017: Asus ROG XG Station 2 has a H2D firmware fix available here.

19-April-2017: Mantiz Venus will be shipped with H2D fixed firmware fix here, updated here.

6-April-2017: AKiTiO Node has released a firmware fix for this issue available here.

As of 30-March-2017, all the TI83-based TB3 enclosures in the buyer's guide, listed below, have been tested and found to be delivering half the critical host-to-device (H2D) bandwidth of their 22Gbps Intel's TB3 specification , 1100MiB/s or 9.22Gbps.

• AKiTiO Node
• Mantiz Venus
• Power Color Devil Box (new releases are TI83, older ones are TI82)
• Asus ROG XG Station 2

 This brings their real-world performance down to less than TB2 levels (1100MiB/s here for TB3 vs 1300MiB/s for TB2). It has a direct impact on bandwidth-dependent gaming FPS when using an external LCD attached to your eGPU.  Performance analysis is in the appendix.

What sort of gaming FPS improvement could be had if this was half-H2D bandwidth issue was fixed?

Consider a H2D of 1100MiB/s is 9.22GBps, which is close to x4 1.1 8Gbps.  Below is a summary of the games seeing significant FPS increase with a x4 1.1 -> x8 1.1 bandwidth increase (approx. equivalent to the H2D issue being fixed). As found at techpowerup's GTX1080 PCIe Scaling test @ FHD:

* Hitman: +46% (63.7->93.1)  here

* Far Cry Primal: 43% (51.4->73.9) here

* Just Cause 3: +42% (73.7->104.7)  here

* Total War Warhammer : +39% (43.2->58.6)  here

* Assassin's Creed Syndicate: +29%  (55.1->70.9) here

* BF1: +26% (97.8->123.4) here

How to solve this half-H2D bandwidth performance problem?

1. Purchase a TI82-based enclosure

 Do note that while the TI82-based AKITiO Thunder3 gets a better H2D of 1700MiB/s here and even 2200MiB/s here it's still less than the TI82-based Razer Core that so far hasn't given us a H2D benchmark less than 2200MiB/s.

2. Wait for the enclosure vendors to fix the problem with new firmware

 The affected TI83-based TB enclosures have a better feature set than the TI82-based enclosures so may be worth the wait to get this performance issue fixed. At eGPU.io we've notified the following eGPU.io reps of this problem:

• AKiTiO eGPU.io rep @DanKnight
• Mantiz eGPU.io rep @Mymantiz_John

As Intel FW is likely the culprit, the vendors should aim for at least for 22Gbps H2D performance specced by Intel for TB3.

FYI: Intel have FW throttled TB3 down to 22Gbps. It is 32Gbps capable.

While at it, I've asked the vendor reps to ask Intel to unleash TB3 from 22Gbps to it's full  32Gbps PCIe traffic capacity. The underlying Intel system architecture can support it as discussed here. It would then help counter the large outting of TB3 eGPU underperformance based on, quite likely, this half-H2D performance issue tainting the results in the following  video:

https://www.youtube.com/watch?v=SeGWwziCrpw

 Appendix: benchmark results identifying the problem

The tested comparison systems and enclosures being:

1. A Dell Precision M7510 - TI82 Razer Core vs TI83 AKiTiO Node - H2D of 2124MiB/s versus 1126MiB/s

2. ASUS UX501VW, TI82 AKITiO Thunder3 vs TI83 AKiTiO Node: H2D 2081MiB/s vs 1144 MiB/s

3. 2016 13" Macbook from @Goalque email - TI82 Devil Box vs TI83 AKiTiO Node - H2D 1625MiB/s vs 1108MiB/s

@Goalque verifying these results using Matlab which confirms CUDA-Z is working properly:

The 2016 13" MBP +  Powercolor Devil Box enclosure gets H2D 1625MiB/s, compared to the Node's 1108MiB/s. Not quite the top end 2200MiB/s H2D we've seen which suggests Apple may have throttled the TB3 notebook firmware OR the Devil Box is also not delivering max performance. To confirm the latter would require PCIe TB3 SSD comparative benchmarking to see if it can hit 2200MiB/s.

 4. MSI GS63VR. Another TI83-based AKiTiO Node half-H2D result here.

5. TI83-based  Mantiz Venus is half-H2D affected here .

6. TI83-based  Asus ROG XG Station 2 is half-H2D affected here.

7. TI83-based AKiTiO Node is half-H2D affected with an AMD RX470 here as tested with OpenCL benchmarking.

HI Nando4,

Is my result normal?

Akitio node, late 2016 macbook pro 13' touch bar, GTX 1050TI

 Thunderbolt Bus 1:

      Vendor Name: Apple Inc.

      Device Name: MacBook Pro

      UID: 0x0001533671212601

      Route String: 0

      Firmware Version: 15.14

      Domain UUID: E783E973-F7B6-2F66-BBCB-1BEE04A41622

      Port:

          Status: No device connected

          Link Status: 0x101

          Speed: Up to 40 Gb/s x1

          Current Link Width: 0x1

          Receptacle: 1

          Link Controller Firmware Version: 0.17.0

      Port:

          Status: No device connected

          Link Status: 0x101

          Speed: Up to 40 Gb/s x1

          Current Link Width: 0x1

          Receptacle: 20

          Link Controller Firmware Version: 0.17.0

question is:

1. this can also be tested in Heaven or Valley?

2. this can be felt during gaming?

3. is there a second way to know the thunderbolt speed beside CUDA?

4. is it related to the AKITIO node cable? I would try the Belkin cable

5. do we have the right to ask Apple to correct that, if it's Apple restriction, because it's not as what they promised

Posted by: ddqp

 

HI Nando4,

Is my result normal?

Akitio node, late 2016 macbook pro 13' touch bar, GTX 1050TI

  

You too are seeing underperforming results.  Confirmed to be an enclosure issues by @Goalque's results (new addition) to the opening post showing a Powercolor Devil Box delivering 1625MiB/s H2D on a 2016 13" MBP whereas a Node delivers 1108MiB/s.

AKiTiO Engineers are investigating. If you have the issue, please post your system reports and benchmarks, as much detail as you can. I've sent this thread to engineers.

It's worth noting the TI82 equipped enclosures so far have yielded higher Host to Device numbers. Could this be a firmware restriction on the TI83 controller?

Deleted 

This has nothing to do with macOS. The results are the same on Windows. The firmware version of my TI82 equipped Devil Box might explain the ~500MiB/s difference.

My results on Akitio Node to tb2 MBP mid 2015 m370x

CUDA-Z Report

=============

Version: 0.10.251 64 bit  http://cuda-z.sf.net/ 
OS Version: Windows x86 6.2.9200 
Driver Version: 378.92
Driver Dll Version: 8.0 (6.14.13.7892)
Runtime Dll Version: 6.50


Core Information
----------------
	Name: GeForce GTX 980 Ti
	Compute Capability: 5.2
	Clock Rate: 1240.5 MHz
	PCI Location: 0:11:0
	Multiprocessors: 22 (2816 Cores)
	Threads Per Multiproc.: 2048
	Warp Size: 32
	Regs Per Block: 65536
	Threads Per Block: 1024
	Threads Dimensions: 1024 x 1024 x 64
	Grid Dimensions: 2147483647 x 65535 x 65535
	Watchdog Enabled: Yes
	Integrated GPU: No
	Concurrent Kernels: Yes
	Compute Mode: Default
	Stream Priorities: Yes


Memory Information
------------------
	Total Global: 6144 MiB
	Bus Width: 384 bits
	Clock Rate: 3505 MHz
	Error Correction: No
	L2 Cache Size: 48 KiB
	Shared Per Block: 48 KiB
	Pitch: 2048 MiB
	Total Constant: 64 KiB
	Texture Alignment: 512 B
	Texture 1D Size: 65536
	Texture 2D Size: 65536 x 65536
	Texture 3D Size: 4096 x 4096 x 4096
	GPU Overlap: Yes
	Map Host Memory: Yes
	Unified Addressing: Yes
	Async Engine: Yes, Bidirectional


Performance Information
-----------------------
Memory Copy
	Host Pinned to Device: 1259.09 MiB/s
	Host Pageable to Device: 1159.32 MiB/s
	Device to Host Pinned: 1341.19 MiB/s
	Device to Host Pageable: 1193.47 MiB/s
	Device to Device: 107.563 GiB/s
GPU Core Performance
	Single-precision Float: 6869.32 Gflop/s
	Double-precision Float: 225.315 Gflop/s
	64-bit Integer: 359.719 Giop/s
	32-bit Integer: 2020.52 Giop/s
	24-bit Integer: 1443.23 Giop/s


Generated: Mon Mar 27 22:00:55 2017

This is my result with a Node + Titan X on a 2014 iMac (TB2).

Thanks for the info guys. Our team is currently investigating and running tests.

Oh man, good to know... and good to see that the Akitio team is on it, and I love their humbleness 😉

I can confirm this on mine too! MSI GS63VR Skylake 6700HQ

Akitio team, are you trying NVM16 vs NVM23?

TI83-based Mantiz Venus is affected as well:

https://egpu.io/forums/thunderbolt-enclosures/mantis-venus/paged/5/#post-5788

What's about the AKiTiO Thunder3? Any reason to suspect the same problem?

When I try to research this issue the only results that I get come back to this forum. I keep checking this and Akitio's Twitter every hour waiting for a solution. Until then, my Node and GTX 1080 sit on the floor unused 🙁

Nando4, how did you come to the conclusion that it's an Intel FW issue? Just trying to learn from you lol

Posted by: PFCBarefoot

Nando4, how did you come to the conclusion that it's an Intel FW issue? Just trying to learn from you lol   

The vendor advised they've made progress in recouping some of the missing H2D bandwidth by way of a different enclosure Intel FW.  Now they'll engage Intel with the aim to recoup all the missing H2D bandwidth. This thread being referenced for the problem at hand.

Posted by: PFCBarefoot

 

I keep checking this and Akitio's Twitter every hour waiting for a solution. Until then, my Node and GTX 1080 sit on the floor unused 🙁

 

   Same for me - waiting for updates and bombarding ASUS with phone calls asking for NVM update of laptop's TB3.

Is it possible to fix issue without updated firmware on laptop?

Posted by: Jaye

 

Is it possible to fix issue without updated firmware on laptop?

   

We're not sure yet. It will likely require a FW update from either Intel or Akitio. Most likely it will be an Intel FW update as several companies (not just Akitio) that are being effected.

Would this issue impact non gaming applications like After effects or Davinci resolve?

Since After Effects uses GPU acceleration and CUDA cores - I think answer would be yes.

Update March 29, 2017:

Intel has responded to us with a beta firmware. We're currently running tests for stability and seeing if it comes with any other issues. Intel has also reached out to the other companies as well (according to them).

That's wonderful news! Thank you for your help @DanKnight.

Posted by: nando4

 

Posted by: greg

 

What's about the AKiTiO Thunder3? Any reason to suspect the same problem?

 

I don't have full 4-lane Thunder3 CUDA-Z results, but the 2-lane TB3 (16Gbps) XPS 9350 showed higher H2D numbers at https://egpu.io/nvidia-optimus-dell-xps-13-akitio-thunder3-gtx-980/ which suggests it isn't affected. The Thunder3 is also TI82-based like the unaffected Razer Core and Devil Box shown on the opening post.

   

This alert has been upgraded to cover ALL TI83-based TB3 enclosures in the buyer's guide,  as they have been tested and found to be deliver half the critical host-to-device (H2D) bandwidth of their Intel TB3 specification of 22Gbps. Affected are:

• AKiTiO Node
• Mantiz Venus
• Power Color Devil Box (new releases are TI83, older ones are TI82)
• Asus ROG XG Station 2

HI all:

 My name is John, I am a developer of Thunderbolt 3 related products.

Here's my study result & opinion.

1. I did tons of tests in Games ( Far Cry Primal , Witchers, COD,) & Benchmark Utility ( Heaven, Valley, 3Dmark & Compubench  ), I see no FPS & Performance Gap between the what so called 2200MB and 1100MB. If there's anyway or method that you guys can provide to proof, let me know, I'll do the test again. and In this case , in the real world, w/o the gap, what will be the purpose to provide the solution as mentioned above?  A solution is to fix the problems and I found there's actually no problems no gaps between these C2D Bandwidth.

2. TI82 were the old design which had been caused ghost key, USB devices lost & not comply with the latest power delivery 2.0, 3.0 requirement. According to this, TI 65983 were suggested by Intel and mainly to solve those issues. I strongly do not suggest to buy the TI82 model. I am not the only one, Intel request all players to adapt the 65983 and more over Apple simply Block 65982 by their MBP Late 2016 model. To cause the 2200mb /1100 mb is FW. while Launching TI82 Model was attached the Old FW, now since 83 is current model, it attached the latest FW, Old/NEW FW caused this difference. ( ** but for our real tests, just Numbers , again, No performance & FPS gap )

3. Intel 22GB Bandwidth, it is accurate Bandwidth,  32GB is the electrical bandwidth. Please don't mix it.  Thunderbolt 3 provide the 40GB bandwidth with  DATA 22GB ( Max ) + DP signal remained  or DP signal ( 32GB ) + Remained Data ( 8G ). in this case, you may say, PCIe x 4 32GB ( Electrical ) Go through the Thunderbolt 3 switch & Thunderbolt 3 Cables, the accurate real  Data will be 22GB. I can cook the FW, Even I setup to PCI-e Bandwidth to 40G, but actually the alpine Ridge & Cable width is only afford 22G in the end. 

4. Intel may update the FW, I had no idea after updated what the CUDA Z # is going to be. But again, I tried the 2200mb/1100MB and I see no difference. If I got the FW updated, I'll let everybody know here and I'll do the tests again to compared with the recent data. or Intel will not update the FW in the end.

The reaction.... when you have Thunderbolt 2  eGPU:

Posted by: Mymantiz_John

HI all:

 My name is John, I am a developer of Thunderbolt 3 related products.

Here's my study result & opinion.

You should repeat again your test focusing on  demanding benchmark/games scenes. You will notice the difference in the daily usage if you constantly ask your setup to run full capability. Check again the minimum FPS in these scenes and compare the results. Also you should focus on benchmarking the FrameTime instead only max/min/average fps!

After that TBE is under constant upgrade pressure, it did not surprise me to Intel and manufacturer force the upgrade blocking the firmware. As same as Win10 forced people to upgrade, not only the os but also the hardware!!!

Posted by: wimpzilla

 You should repeat again your test focusing on  demanding benchmark/games scenes. You will notice the difference in the daily usage if you constantly ask your setup to run full capability. Check again the minimum FPS in these scenes and compare the results. Also you should focus on benchmarking the FrameTime instead only max/min/average fps!

After that TBE is under constant upgrade pressure, it did not surprise me to Intel and manufacturer force the upgrade blocking the firmware. As same as Win10 forced people to upgrade, not only the os but also the hardware!!!

   

HI:

1.  Thanks for your valuable suggestion, guide me to have a deeper test or any test value  you would like to see. I have many Hosts & Cards may support to clarify this issues.

2. FW divided to 2 types, one is eGPU and  the other is PCI-E. I have both, I can make Ti83 Board to be an PCI-E box and running CUDAZ to be 2200mb. But the side effect is the "Unstable Hotplug & Play ". and once during the application running, suddenly remove the connection will cause the system Hang or reboot request.  I do not see any real world performance drop and enjoy the stable hot plug & play.

3. Intel Thunderbolt 3 certification is like this. If you submit the device is an eGPU, you have to use the eGPU FW, and during the certification process, Lab will exam the design, circuit & VGA compatibility. If I submit my device as an PCI-e Box and I have to choose the PCI-e FW and the device will be attached with an PCI-e Card ( Not GFX ). Certificaiton protocol is different than the eGPU's. 

4. Considering the stability & no performance gap ( or not apparently gap , at least not 50% drop ) and more over we are actually running a GFX externally, I have to set the device to be a eGPU FW.

Posted by: enjoy

 

The reaction.... when you have Thunderbolt 2  eGPU:

   

I had tested Razer Core 2200mb which is similar result with the 1100mb one, don't understand why I need to have Thunderbolt 2 eGPU? even I have thunderbolt2 eGPU, how will I have the same test condition to compared with TB2 & TB3 performance by using different Hosts? TB2 has to be link with a TB2 Hosts, such as MBP2015. compared with MBP2016 different CPU, UEFI, is it a meaningful comparison? If I lost something or concept goes wrong, kindly teach me.

you have TB3 enclosure with limited bandwidth caused by intel, connected to TB3 computer, and your measurement show no difference with TB3 no limited enclosure;

what about if you plug TB2 enclosure on TB3 computer, then the restriction will be TB2 enclosure, if it shows the same result(game fps, benchmark, etc) with TB3 enclosure, then it means, TB3 enclosure is no meanings.

Posted by: ddqp

 

you have TB3 enclosure with limited bandwidth caused by intel, connected to TB3 computer, and your measurement show no different with TB3 no limited enclosure;

what about if you plug TB2 enclosure on TB3 computer, then the restriction will be TB2 enclosure, if it show the same result(game fps, benchmark, etc) with TB3, then it means, TB3 enclosure is no meanings.

   

I hope I could have one TB2 box  in my county. is there anybody who had tested MBP2016 late + NV1080 in 3Dmark + TB2 Box? 

I completely agree  with your engineering point of view about the firmware choice. But like @ddqp said, picking the heart of the issue, whats is the meaning to provide a potential 4x pci express bandwidth, but restrict it depending witch hardware you would use, witch weight it have among the pro/customer market.

I'm sure where it is really needed, worth, custom firmware and maybe expensive TBE driver could perform hotplug and high bandwidth without any problem.

That said, the main issue and advert for an adapter like these, it is to be plug and play like you said, but also be capable of reliable performance with high end video adapters, witch mean in any way a good pci-e link, bandwidth, response time.

You can try yourself to play on 1080p/1440p/4K and record the frametime along the usual fps count. If you didn't see any substantial drops, stutters, you are good to go. 

Posted by: wimpzilla

 

I completely agree  with your engineering point of view about the firmware choice. But like @ddqp said, picking the heart of the issue, whats is the meaning to provide a potential 4x pci express bandwidth, but restrict it depending witch hardware you would use, witch weight it have among the pro/customer market.

I'm sure where it is really needed, worth, custom firmware and maybe expensive TBE driver could perform hotplug and high bandwidth without any problem.

That said, the main issue and advert for an adapter like these, it is to be plug and play like you said, but also be capable of reliable performance with high end video adapters, witch mean in any way a good pci-e link, bandwidth, response time.

You can try yourself to play on 1080p/1440p/4K and record the frametime along the usual fps count. If you didn't see any substantial drops, stutters, you are good to go. 

   

Thanks for your advises, I'll prepare those material to have deeper test.

HOw many min playback & record by your suggestion? and what will be the difference if I use the Unigine Heaven or Valley to run the test? They are mainly for the FPS right? Teach me more about this, thanks.

If you want to build up a nice review, data, you could use further, i suggest you to use the basic video adapters review guides lines, used to review the hardware in top tech sites.

Obviously your aim is not to do such big review with a lot of games. Only a few heavy games that have built in benchmark, usually repeating the test 2/3 time minimum to get consistent data to work with. Recording frametime and focusing on minimum fps reflect a smooth and enjoyable video experience, since here you are benchmarking the box, not different cards, so only the TBE2 or TBE3 should be the limiting factor.

You can include benchmarks, but remember if your aim is to gathers consistent, reliable data, to get a conclusion, you should primary focus on game experience, benchmark not always reflect the ingame optimizations and load, especially in latest games.

Posted by: wimpzilla

 

If you want to build up a nice review, data, you could use further, i suggest you to use the basic video adapters review guides lines, used to review the hardware in top tech sites.

Obviously your aim is not to do such big review with a lot of games. Only a few heavy games that have built in benchmark, usually repeating the test 2/3 time minimum to get consistent data to work with. Recording frametime and focusing on minimum fps reflect a smooth and enjoyable video experience, since here you are benchmarking the box, not different cards, so only the TBE2 or TBE3 should be the limiting factor.

You can include benchmarks, but remember if your aim is to gathers consistent, reliable data, to get a conclusion, you should primary focus on game experience, benchmark not always reflect the ingame optimizations and load, especially in latest games.

 

 

   

unfortunately I could have TB3 box but with two versions FW 2200mb & 1100mb, it is hard to get the TB2 box here. I'll try to get one, once it is done, I'll post again. 

but basically, impossible the performance is poor than the tb2 box, and more over, tb3 support much more than the tb2. even the worst case TB3( No matter 2200mb or 1100 mb ) is so similar with TB2 performance , but still to purchase TB3 box is much worthy than tB2's according to the larger  bandwidth ( PCIe  22gb data + others ) , Power delivery & Hot Plug & Play. and also need to think about the latest Laptop design changed to the tb3 type C port and no more Mini DP for TB2. 

this should be no argument  in purchase guideline point of view.