PCIe Ethernet in Thunderbolt docks  

@khronokernel, welcome to the forums! I have tried the Razer TB4 dock and the ThinkPad Universal Thunderbolt 4 dock. Am returning the latter simply because it doesn’t work appropriately in macOS (DisplayPort MST). Otherwise latter would be a better dock. Not a fan of the Realtek NIC myself. In my experience it has generally worked ok on the Razer. I did notice once that Ethernet didn’t work on hot-plugging on my 2019 16-inch MBP. Re-plugging fixed it.

On the 2021 16” MBP, haven’t seen Ethernet issues yet but there’s one issue that plagues the USB ports (also happens on 2019 Mac) - on hot-plugging to the dock I need to power cycle my DAC/AMP setup to get it to show up. Minor inconvenience as it’s right on my desk but could be an issue for things that are bus-powered and your dock is tucked away (like mine). Perhaps I should test a USB drive.

Since I switch laptops (work vs. personal) hot-plug behavior is important to me. However, once everything is working, it’s been a stable setup without issues.

I shared some of my experience here: https://egpu.io/forums/laptop-computing/my-experience-switching-to-an-egpu-less-world/

BTW, quick suggestion: adding power delivery as separate column in the table would make it much more useful.

Posted by: @mac_editor

BTW, quick suggestion: adding power delivery as separate column in the table would make it much more useful.

Ah great idea! I mostly care for 60w~ so didn't really account for it in my research but looking back I bet that's pretty important to a lot of people ;p

@khronokernel, Thank you very much for sharing this excellent resource. Would you mind if we cross-post it to the front end and top-nav for more visibility? This Thunderbolt dock table can help users greatly during their search.

Posted by: @itsage

Would you mind if we cross-post it to the front end and top-nav for more visibility?

Oh ofc, no issue at all! 

@khronokernel, This is really awesome! I looked up OWC Thunderbolt 3 Pro Dock on Amazon - it's pretty cool but has some very funny negative comments. I would love to find something like this but with maybe just the Ethernet, 1-2 USB 3.1 and maybe sd card reader. I would just love to off load Ethernet to PCIe controller and see if this helps with performance. I am using Belkin USB-C adapter at the moment. 

Wow that same dock by OWC is also sold under different company - Akitio. Okie Dokie

@khronokernel,  super interested in this topic and subbed here. This would've been awesome to test by like NexusGamming or Linus to see if there is real world difference in performance. Judging by some comments from shoppers, it looks like PCIe route has a lot of computability issues. This needs some testing to really understand what the real difference is and if it is worth the trouble. I am thinking Razer Thunderbolt 4 dock personally. It looks sweet, a bit limited on the networking side but already on Amazon Warehouse Deals $100 cheaper. I am going to give it sometime and see what other people think/experience before making decision.

Posted by: @dima_novosad

This needs some testing to really understand what the real difference is and if it is worth the trouble.

The main area where you'll see a difference between USB and PCIe would be:

CPU Overhead

If you're hammering the network while also using the CPU for other tasks, you'll notice the USB driver stack using a noticable amount of CPU. Generally this varies system to system, but it's generally quite noticeable (sometimes pegging a core on download via USB NICs)

However unless you're constantly maxing out a 1Gbps link, it will be fairly difficult to peg a core with just USB ethernet. Unless you know your workload is going to be quite CPU and Network intensive, this shouldn't be too much of an issue.

USB Bandwidth

Assuming the Thunderbolt dock you purchased only has 1 USB 3.0 controller, all peripherals connect will be sharing that 5Gbps link.

Therefore if you have:

• Table ignores additional overhead from USB conversion

With these 4 devices hooked up and active, you've already exceeded the sum total 5Gbps a USB 3.0 controller can provide to its devices. So in situations where you'll be maxing out your network and the other USB devices connected on the same controller, you can notice a degrade in performance.

• USB 3.1 can partially alliviate this issue but due to some weird bandwidth hacks we'll stick with stock 3.0

Some higher end docks will have additional USB 3.0 controllers so front side ports won't take bandwidth away from rear ones, however docks like these generally already have a PCIe Ethernet controller anyways.

• CalDigit TS3 Plus is a notable example where they include 3 USB controllers (x2 3.0 controller, x1 3,1 controller) and a PCIe Intel i210 NIC

@khronokernel, wow this great explanation. thank you for this. I feel like I am picking it up. You mentioned USB 3.1 could solve a lot of the things. Is it because the bandwidth is 10gb instead of 5? Also, I was just replying yesterday to another thread where someone was having similar setup to mine with 6800XT and Razer Chroma but when they hooked up external dock via USB they got error code 12. Does this mean that the ports on MacBookPro share this 10GB bandwidth or are they independent? From I what understand the total bandwidth is 40Gb so if to allocate 10GB for a dock would the gpu only have 30 available? Would this be not enough for egpu?

Posted by: @dima_novosad

You mentioned USB 3.1 could solve a lot of the things. Is it because the bandwidth is 10gb instead of 5?

So as with most things USB, bandwidth sharing is complicated. So while yes USB 3.1 does support 10Gbps shared bandwidth for devices on the same controller, weird issues come into play when mixing and matching 3.1 and 3.0 devices:

• https://semiengineering.com/using-usb-3-1s-multiple-ins-to-reach-10-gbps-data-rates/

This mainly applies to users of USB hubs which I assume it quite a few as many docks (especially TB4) are limiting the number of USB ports present (to make room for the additional TB4 ports for daisy chaining). However if you're using the direct ports off the USB Controller, this shouldn't be much of a concern.

Posted by: @dima_novosad

Does this mean that the ports on MacBookPro share this 10GB bandwidth or are they independent? From I what understand the total bandwidth is 40Gb so if to allocate 10GB for a dock would the gpu only have 30 available? Would this be not enough for egpu?

Unfortunately selectively allocating bandwidth to individual devices is not possible on the same port. Additionally Thunderbolt 3's effective bandwidth is not 40Gbps, instead it's closer to 32Gbps~ as this is the amount 4 lane PCIe 3.0 will provide. The remaining 8Gbps is allocated for dual DisplayPort 1.4 as well overhead.

• DisplayPort is limited to 1.2 on pre-Titan Ridge TB3 controllers, Coffee Lake and newer Macs should all be equipped with Titan Ridge or newer controllers.

So assuming you have 2 thunderbolt devices in a chain (ie. dock and eGPU), you have 2 options:

• Share the same port and daisy chain
  • Your TB3 dock will allocate 2 lanes of PCIe 3.0 to itself, as well as your eGPU
  • Downside is that you're limiting the overall bandwidth of the eGPU
• Connect to individual ports/busses
  • Both the TB3 dock and eGPU will get a full PCIe 3.0 link
  • Note that per-port is referring to individual busses. Intel MacBook Pros (2016+, 4 port model) will have 2 busses, one for each side and thus bandwidth is shared if you use both ports on the same side
    • Connecting a eGPU on one side and TB3 dock on the other is the ideal setup. 
    • Apple Silicon Macs don't have this limitation, each port has a dedicated bus. This applies to all models including the M1, M1 Pro and M1 Max. However macOS does not support eGPUs on Apple Silicon at this time