Is it feasible to make a DIY Thunderbolt 3 setup?  

  RSS

rycao18stu
(@rycao18stu)
New Member
Joined: 11 months  ago
Posts: 3
November 7, 2017 3:28 pm  

I'm talking about a full Thunderbolt 3 setup, not some sort of expressport - Thunderbolt adapter or something. Is it possible to DIY this for a cheaper price than $250 (or lowest price of a premade eGPU box). I've looked for boards with PCIe slots and thunderbolt support, but I haven't found anything. 

If it is possible, what parts would I need?

If not, why not? Is it because it's cheaper to buy an Akitio Node or HP Accelerator?

 

Thanks,

rycao18stu


ReplyQuote
Yukikaze
(@yukikaze)
Prominent Member Moderator
Joined: 2 years  ago
Posts: 789
November 7, 2017 4:19 pm  

It is not possible because you can't buy the controller boards required to make one. And when you could (inXtron sold TB3 DevKits for 280$, but no longer do) they were as expensive as entire enclosures.

My eGPU Zoo - Link to my Implementations.
Want to output [email protected] out of an old system on the cheap? Read here.
"Always listen to experts. They'll tell you what can't be done, and why. Then do it."- Robert A. Heinlein, "Time Enough for Love."


3RYL, nando4 and theitsage liked
ReplyQuote
rycao18stu
(@rycao18stu)
New Member
Joined: 11 months  ago
Posts: 3
November 7, 2017 6:54 pm  

Thanks for the response! Do you have any suggestions for enclosures? I see a lot of people use the Akitio Node, and I just saw the HP Omen Accelerator open back up for preorders? Are there anymore that I don't know about? 


ReplyQuote
nando4
(@nando4)
Noble Member Admin
Joined: 2 years  ago
Posts: 1781
November 7, 2017 9:34 pm  
Posted by: rycao18stu

Do you have any suggestions for enclosures? I see a lot of people use the Akitio Node, and I just saw the HP Omen Accelerator open back up for preorders? Are there anymore that I don't know about? 

Click the Buyers Guide header link to see the currently available eGPU enclosures. There you'll see enclosures by AKiTiO, Sonnet, HP, Mantiz, Netstor, Razer and Aorus.

Upcoming royalty-free Thunderbolt will give manufacturers more component mix&match flexibility:

https://egpu.io/thunderbolt-3-news-for-egpus/

eGPU Setup 1.35    •    eGPU Port Bandwidth Reference Table    •    Several builds
2015 15" Dell Precision 7510 M1000M + macOS 10.14 & Win10


Yukikaze liked
ReplyQuote
rycao18stu
(@rycao18stu)
New Member
Joined: 11 months  ago
Posts: 3
November 8, 2017 12:04 am  

Is there any set date that Thunderbolt will become royalty free? Has intel filed a patent or something that will expire next year? How do people know it will become royalty free next year? 


ReplyQuote
karatekid430
(@karatekid430)
Estimable Member
Joined: 12 months  ago
Posts: 118
November 8, 2017 3:25 am  

I bought the Aorus Gaming Box and sold the GTX 1070 from it. I paid AUD $1000 and sold the GPU for $530, although the setup has since dropped by AUD $100.

It doubles as a 100W USB-C power delivery supply. It is pretty good value, and does not take much space from your desk. I am using the AMD Radeon R9 Nano, and there is room to spare inside.

The only thing that is needed to make it perfect is that I want the same thing but with a second Thunderbolt 3 port for daisy-chaining. My Dell 9365 only has one TBT3 port.
I have found that PCIe 3.0 x4 is more than enough for modern GPUs, and I prefer the convenience of daisy chaining over a few percent higher performance.
For example, I would use the Sonnet Echo Express SEL with a Intel X550 T2 for high performance networking - but it only offers bus power of 15W, and the eGPU should really be the first in the chain. There are people who have published benchmarks of performance losses as things move down the daisy chain, away from the computer. This is fine for 10GbE, but the GPU would prefer the lower latency.
I know that Intel disallows the daisy-chaining for eGPU, but my Dell 9365 says "External GPUs Supported: No", and it still works, so surely it is just Intel-powered bulls--- and just trying to milk royalties and certification fees. They could just say it's an external PCIe enclosure, which is not an eGPU, which just happens to have a power supply big enough to take a GPU.

But in terms of DIY.... that depends how you define it. You will likely never be able to make your own PCB, because at those data rates, the tiniest of things have effects on the signal.

You could buy the Sonnet Thunderbolt 3 NVMe flash drive, remove the SSD and use a M.2 to PCIe adapter assembly (unless there are firmware restrictions). But this is kind of messy.

I am going to be looking into flashing an Asrock Thunderbolt 3 AIC with firmware to change the controller role from host to downstream, and from microsatacables.com use the SFF-8643 to PCIe x4 slot, and the SFF-8639 to PCIe x4 slot, with an Intel 750 SSD U.2 cable, to connect it to a GPU or other PCIe device. Hopefully the role is set in firmware for the controller, in contrast to being hardwired in the hardware.

I remember some article published by white hat hackers about Thunderbolt 3 and deadly DMA attacks. They tore apart an Apple Thunderbolt - 1GBASE-T network adapter and said that the controller is in the Thunderbolt port, and the small cord carried PCIe. It probably is not PCIe 3.0 x4, but somebody could look at whether the Ethernet end could be replaced with a MXM GPU, or even a PCIe slot. That would make the most compact solution (and also probably the worst). But I suppose you could just put a low-powered GPU there with 6x DisplayPort and drive 6x 3840x2160p60 displays - which is way more than the 2x you can get out of a standard adapter. Most GPUs only have 4x outputs, but back in 2009/2010 there was the AMD Radeon HD 5870 Eyefinity 6 Edition, which I absolutely adored. I have been waiting for a flagship ever since with this (a few happened by board manufacturers, but they were like the 7870, and not the flagship). But there should be digital signage versions somewhere in 2017 that you could use.

The most basic eGPU setup you can get is attaching your GPU to the M.2 slot of a computer. People have done it for NUCs. I would have tried with my Surface Pro 4 and booted off a USB had I had the equipment when I upgraded the SSD (worth noting that despite them advertising PCIe 3.0 x4 SSD, the M.2 slot is only wired at x2). The M.2 PCIe 3.0 x4 will be faster than Thunderbolt 3, with no controller overhead and other side-effects. I hotplugged and hot unplugged a GPU to my desktop successfully. It would have worked without rebooting had the resource allocation not failed. Although I am fairly sure that this is not a problem in Linux, because it is not ACPI's b---h and it throws all the BIOS allocation away and does it itself. I love the kernel. After getting the GPU running in Windows, you can hot unplug it, but you will get a watchdog timeout BSOD if you do not disable the driver first in devmgmt.msc.

I love Thunderbolt for PCIe, but ideally, the PCI-SIG needs to re-define PCIe, breaking backwards compatibility if required, to work natively and securely with hot-plugging. It would just be simpler. There is the OCULINK-2 with PCIe 4.0 - but the connector looks too bulky for phones and tablets, and it will not have the reversible USB-C magic, nor will it likely have power delivery and alternate modes.

The reason we need something like this is that manufacturers do not make what we want with USB. It should be theoretically possible to make a USB to PCIe adapter to attach a real GPU via USB. Of course, the performance will be shocking, but what if we just want to connect it, without the expectation of performance? Because you could make a huge adapter which a USB cable, and inside a USB to Ethernet, and then connect the Ethernet to an Intel NUC or similar, and then attach that to the GPU via M.2 PCIe, and then write software and kernel mode drivers to send Vulkan API calls over TCP/IP. It would be useless in performance, but it could work. So surely the intelligent people who make these things could come up with a way to go from USB to PCIe with less fuss than that.

At least USB 3.2 defines 20Gbps operation by using both of the tx/rx pairs in the USB-C connector. This is just going to make the situation worse where vendors do not allocate enough PCIe lanes to actually feed the USB controllers, but maybe we will hopefully finally see a 10Gbps or at least NBASE-T Ethernet adapter for USB, and this is the biggest we need Thunderbolt right now, aside from the external graphics.

I go somewhat off topic, but I hope my ramblings can be useful to somebody. It is all related and technically relevant.


ReplyQuote
karatekid430
(@karatekid430)
Estimable Member
Joined: 12 months  ago
Posts: 118
November 8, 2017 3:32 am  

My biggest worry with eGPUs is that by having them approved, somebody could remove the GPU from them and install a PCILeech and DMA your encrypted computer, and steal everything.

With the M.2 to PCIe x4 adapter, I was able to hot plug a lot of things successfully. While GPUs had resource allocation issues, I installed the Sunix UPD 2018 (Asmedia controller) and had it actually working without a reboot. Does that mean that with Bitlocker, somebody can open your desktop and hot plug a PCILeech? I always assumed that it would not work, and the idea was a reboot would be required, and the TPM would dump its keys when it realised something had changed. But I am starting to think otherwise.

Does anybody know if normal PCIe slots are bound by IOMMUs? The IOMMU protects against DMA by allowing the kernel to allocate memory regions for each device, just like it does for processes. That way, if the PCIe devices try to access something not allocated to them, they are denied access. Presumably like a segmentation fault for a process.

Just something to consider, security-wise.


theitsage liked
ReplyQuote
Yukikaze
(@yukikaze)
Prominent Member Moderator
Joined: 2 years  ago
Posts: 789
November 8, 2017 5:31 am  
Posted by: rycao18stu

How do people know it will become royalty free next year? 

That's how (May 24, 2017).

Specifically:

"In addition to Intel’s Thunderbolt silicon, next year Intel plans to make the Thunderbolt protocol specification available to the industry under a nonexclusive, royalty-free license. Releasing the Thunderbolt protocol specification in this manner is expected to greatly increase Thunderbolt adoption by encouraging third-party chip makers to build Thunderbolt-compatible chips. We expect industry chip development to accelerate a wide range of new devices and user experiences."

My eGPU Zoo - Link to my Implementations.
Want to output [email protected] out of an old system on the cheap? Read here.
"Always listen to experts. They'll tell you what can't be done, and why. Then do it."- Robert A. Heinlein, "Time Enough for Love."


ReplyQuote
paulrich
(@paulrich)
New Member
Joined: 8 months  ago
Posts: 1
February 14, 2018 1:26 am  

I would have a brand new board from a HP Accelerator Shell 
card I could sell ? 

 


ReplyQuote
mois
 mois
(@mois)
Trusted Member
Joined: 7 months  ago
Posts: 70
July 28, 2018 10:07 am  

@karatekid430 , do you have any numbers for the overhead of daisy chaining in Thunderbolt 3?

https://egpu.io/forums/pc-setup/thunderbolt-egpu-daisy-chain/ quotes https://developer.apple.com/library/content/documentation/HardwareDrivers/Conceptual/ThunderboltDevGuide/Basics01/Basics01.html saying "Devices on a Thunderbolt bus exhibit higher latency than devices on internal slots—about 1.5 microseconds of round-trip latency per hop. This means that putting a device at the end of a Thunderbolt chain can add up to 9 microseconds of round-trip latency.", this is the only number I find. I'm not even clear if it's for Thunderbolt 3.

Back in ~2012 / Thunderbolt 1 or 2, daisy chaining would cause massive performance losses on downstream devices, but I get the impression that this was fixed in Thunderbolt 3 and daisy chaining is now insignificant.

The reason to use your second Thunderbolt 3 connector for peripherals and keep the eGPU as exclusive device on your first connector, instead, is that your TB3 controller in your laptop has 32gbps bandwidth while each TB3 connector has 22gbps bandwidth, so this way your eGPU has 22gbps and your peripherals get another 10gbps separate bandwidth. This is why you bother to use more than one TB3 connector at all today, instead of just daisy-chaining all of them on one TB3 connector (if all your TB3 devices have daisy chain connectors of course).


ReplyQuote