eGPU in Linux - has anyone here gotten it to work?  

 

mcarver316
(@mcarver316)
Estimable Member
Joined:1 year  ago
Posts: 133
May 22, 2017 1:25 pm  

Any help is appreciated- thanks!

---

I was able install the nvidia drivers and my mobo's thunderbolt is listed (Intel Corporation DSL6340 Thunderbolt 3 NHI [Alpine Ridge 2C 2015]).

Confirmed NVIDIA driver is installed (these are the internal GPUs in the GT80s):

Screenshot from 2017 05 22 09 18 53

Here is the thunderbolt info (Aki Node is not detected unfortunately)

Screenshot from 2017 05 22 09 20 48

Edited: 1 year  ago

2015 17" MSI GT73VR-6RF 2xGTX1070 + 4xGTX1080Ti@32Gbps-TB3 (AKiTiO Node driving Cubix Chassis) + Win10 > 365Gigahashes per second


ReplyQuote
Sky11
(@sky11)
Reputable Member
Joined:1 year  ago
Posts: 301
May 22, 2017 2:43 pm  

try to disable Thunderbolt security in the Device Manager - it should help with detection


theitsage liked
ReplyQuote
Yukikaze
(@yukikaze)
Honorable Member Moderator
Joined:1 year  ago
Posts: 729
May 22, 2017 2:58 pm  
Posted by: Sky11

 

try to disable Thunderbolt security in the Device Manager - it should help with detection

   

Did you mean BIOS?

My eGPU Zoo - Link to my Implementations.

"Always listen to experts. They'll tell you what can't be done, and why. Then do it."- Robert A. Heinlein, "Time Enough for Love."


ReplyQuote
Sky11
(@sky11)
Reputable Member
Joined:1 year  ago
Posts: 301
May 23, 2017 2:00 pm  

My apologies. Must have been a shortage of caffeine in my blood. Yes, of course the system BIOS. But unfortunately not all systems have such setting.

One of these days I will give it a try and will let you folks know.


ReplyQuote
mcarver316
(@mcarver316)
Estimable Member
Joined:1 year  ago
Posts: 133
May 25, 2017 11:38 am  
Posted by: Sky11

try to disable Thunderbolt security in the Device Manager - it should help with detection

Yes. Secure Boot is automatically disabled in the BIOS when configured to boot Legacy, as opposed to UEFI or UEFI with CSM. There are no other settings in the MSI BIOS for security specific to Thunderbolt.

2015 17" MSI GT73VR-6RF 2xGTX1070 + 4xGTX1080Ti@32Gbps-TB3 (AKiTiO Node driving Cubix Chassis) + Win10 > 365Gigahashes per second


ReplyQuote
Sky11
(@sky11)
Reputable Member
Joined:1 year  ago
Posts: 301
May 25, 2017 2:28 pm  

Secure boot and Thunderbolt security are unrelated. Disabling the secure boot will not automatically disable TBT security.

Very likely by default TBT security is enabled in the BIOS, that is why you need to have Thunderbolt drivers installed in Windows - remember the pop-up first time you plugged the eGPU box?

Need to look for Intel Thunderbolt drivers for Linux to enable Thunderbolt device enumeration

 

Edited: 12 months  ago

theitsage liked
ReplyQuote
mcarver316
(@mcarver316)
Estimable Member
Joined:1 year  ago
Posts: 133
May 25, 2017 3:11 pm  
Posted by: Sky11

Secure boot and Thunderbolt security are unrelated. Disabling the secure boot will not automatically disable TBT security.

Very likely by default TBT security is enabled in the BIOS, that is why you need to have Thunderbolt drivers installed in Windows - remember the pop-up first time you plugged the eGPU box?

Need to look for Intel Thunderbolt drivers for Linux to enable Thunderbolt device enumeration

 

This is the same laptop used in Windows and has worked fine for the eGPU.

The OEM MSI BIOS for the GT80S is very limited. The only setting it has for Thunderbolt is enabling/disabling the automatic firmware update.

There is a section in the MSI forum for users requesting custom BIOS for more granular control (mostly related to overclocking), but such BIOS is unsupported by  MSI.

I'll look further into the Intel Thunderbolt Linux drivers- eGPU information for Linux is almost nonexistent.

Edited: 12 months  ago

2015 17" MSI GT73VR-6RF 2xGTX1070 + 4xGTX1080Ti@32Gbps-TB3 (AKiTiO Node driving Cubix Chassis) + Win10 > 365Gigahashes per second


ReplyQuote
nando4
(@nando4)
Noble Member Admin
Joined:1 year  ago
Posts: 1678
May 25, 2017 3:18 pm  

@mcarver316, have you tried to extract the UEFI variables from a BIOS dump?  I did this on a Dell E5470, extracting an IFR file , finding  hidden Thunderbolt settings I could toggle even though the laptop had no TB3 port.

eGPU Port Bandwidth Reference TableeGPU Setup 1.35


ReplyQuote
mcarver316
(@mcarver316)
Estimable Member
Joined:1 year  ago
Posts: 133
May 25, 2017 3:25 pm  
Posted by: nando4

@mcarver316, have you tried to extract the UEFI variables from a BIOS dump?  I did this on a Dell E5470, extracting an IFR file , finding  hidden Thunderbolt settings I could toggle even though the laptop had no TB3 port.

Thanks nando4!

I will investigate the linked information and check this weekend.

2015 17" MSI GT73VR-6RF 2xGTX1070 + 4xGTX1080Ti@32Gbps-TB3 (AKiTiO Node driving Cubix Chassis) + Win10 > 365Gigahashes per second


nando4 liked
ReplyQuote
mcarver316
(@mcarver316)
Estimable Member
Joined:1 year  ago
Posts: 133
May 28, 2017 5:39 am  
Posted by: Sky11

Secure boot and Thunderbolt security are unrelated. Disabling the secure boot will not automatically disable TBT security.

Very likely by default TBT security is enabled in the BIOS, that is why you need to have Thunderbolt drivers installed in Windows - remember the pop-up first time you plugged the eGPU box?

Need to look for Intel Thunderbolt drivers for Linux to enable Thunderbolt device enumeration

 

NOTE: this fixes error code 12- I have yet to test Linux connectivity. The unlocked BIOS offers an option to be active at boot time, rather than wait until Windows is active.

No DSDT fix needed!!! I asked an MSI BIOS genius to unlock the MSI BIOS, switched Thunderbolt security from unique ID to legacy, and boom!!! Both 1080 Ti are active without error code 12!!!

FINALLY!!! Thanks Sky11!!!

FINALLY ALL4GPUACTIVE

Edited: 12 months  ago

2015 17" MSI GT73VR-6RF 2xGTX1070 + 4xGTX1080Ti@32Gbps-TB3 (AKiTiO Node driving Cubix Chassis) + Win10 > 365Gigahashes per second


ReplyQuote
Sky11
(@sky11)
Reputable Member
Joined:1 year  ago
Posts: 301
May 28, 2017 1:36 pm  

you are welcome, Mcarver.

It seems like it is possible to enable TBT device detection in Linux, stay tuned for more details.


ReplyQuote
Sky11
(@sky11)
Reputable Member
Joined:1 year  ago
Posts: 301
May 28, 2017 1:49 pm  

Folks who is experimenting with Linux eGPU - please have a look at this:   https://lkml.org/lkml/2017/5/26/432

Quote:

This patch series adds support for Thunderbolt security levels, which were
first introduced in Intel Falcon Ridge Thunderbolt controller, to prevent
DMA attacks when PCIe is tunneled over Thunderbolt fabric. This is needed
if there is no IOMMU available (or it is not enabled because of various
reasons).

Most PCs out there having Falcon Ridge or newer have security level set to
"user" which means that user authorization is needed before PCIe tunnel is
creaded (the PCIe device appears). This effectively means that without
driver support the user needs to configure security level from BIOS to
"none" to get Thunderbolt devices connected. With these patches the user
can authorize devices using sysfs attributes

hogehh and VisViva liked
ReplyQuote
hogehh
(@hogehh)
New Member
Joined:11 months  ago
Posts: 3
July 30, 2017 11:34 am  

With the 4.13rc2 kernel, I am able to authorize TB3 connection to my eGPU (Gigabyte Aorus GTX 1070 Gaming Box). (Thanks Sky11 for the info!!!!)

$ uname -a

Linux taichi-N501VW 4.13.0-041300rc2-generic #201707232231 SMP Mon Jul 24 02:32:52 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

$ cat /sys/bus/thunderbolt/devices/0-1/device_name 

GV-N1070IXEB-8GD

$sudo sh -c 'echo 1 > /sys/bus/thunderbolt/devices/0-1/authorized'

the eGPU fan starts to spin by this authorization!!!

$ lspci | grep NVIDIA

01:00.0 3D controller: NVIDIA Corporation GM107M [GeForce GTX 960M] (rev a2) <- dGPU

09:00.0 VGA compatible controller: NVIDIA Corporation Device 1b81 (rev a1) <-09:00.1 Audio device: NVIDIA Corporation Device 10f0 (rev a1)

 

After waiting for a while, nvidia-smi shows both dGPU(960M) and eGPU(1070) !!! 


ReplyQuote
LD_PRELOAD
(@ld_preload)
New Member
Joined:10 months  ago
Posts: 2
August 4, 2017 9:43 pm  

(Originally posted to r/Thinkpad last week.)

 

The basics, so far: Had tried Arch, Antergos, Manjaro with Gnome Shell, KDE Plasma, XFCE4, i3 on Thinkpad P50 i7 -7600HQ, using EVGA GeForce GTX SSC 1060 with only partial success, but no actual functioning: External card being recognized under lspci and Nvidia Settings, no external monitor detected by the DE's via Akitio Node 3 but Nvidia Settings did see it. Could not enable external monitor in the DE's; tried various xorg configs and related configs.

 

So in all the DE's and Distros Thunderbolt 3 was recognized and Nvidia Settings saw the internal M1000M in the P50 and the external 1060. But could not enable 2nd X screen or the external monitor in the DE's although the external monitor and card were still detected in Nvidia Settings.

 

Today tested Ubuntu 17.04 Unity with more partial success. External monitor had 2nd X screen via Node 3 but not fully functioning. Cursor of Broken X, movable large black white-outlined cursor, like under twm session. But cannot open programs or move any windows to X screen 1.

 

Having been researching getting this working in my spare time for a few weeks and trying various Xorg configurations, I came upon a bug report for Ubuntu somewhat related to this situation earlier this morning. The upshot in the reports was that Nvidia under Unity and Gnome would not work in the conditions that generated the reports (not directly related to this scenario but it was helpful) but XFCE4 would.

 

I had just set up Ubuntu 17.04 Unity last night and had about the same success as with the other DE's/Distros trying to get the external card and monitor to work. After reading through the bug reports I installed XFCE4 on Ubuntu 17.04 and.....It worked! It. Worked. !. GTX1060 in use on X Screen 1 external monitor. GTX M1000M in use on P50 Screen 0 - laptop lcd. Ran some Unigine benchmarks and was pretty pleased with the results. So, as I said earlier, if there's interest, I'll try to add more info/answer questions, as time allows.

 

ALSO: My setup also has a Windows 10 disk. And an odd thing consistently occurs, I have to boot into Windows 10 first or the GTX 1060 is not detected in linux. I do not know what, why or how this is in effect but I edited this post to note this.

 

(edited for some spelling/grammar, and to add note.)

 

Follow-up to note: Windows 10 Fast Boot is the problem. Disabling FB made the Linux ssd's unbootable, grub> prompt.  Re-enabled FB for now until i can do more research. These are seperate drives.

Edited: 10 months  ago

nando4 liked
ReplyQuote
kravemir
(@kravemir)
New Member
Joined:9 months  ago
Posts: 1
August 12, 2017 3:29 pm  
Posted by: LD_PRELOAD

.... So, as I said earlier, if there's interest, I'll try to add more info/answer questions, as time allows. ...

There is an interest. Though, I'm considering AMD Polaris/Vega GPU. Do you think it would work, or does it require specific drivers for nvidia only?


ReplyQuote
LD_PRELOAD
(@ld_preload)
New Member
Joined:10 months  ago
Posts: 2
August 15, 2017 3:11 am  

For cards and drivers, have you looked at Akitio's compatability list?  The P50 has Intel and Nvidia graphics, so I went with Nvidia because I have more experience with it.

The Ubuntu 17.04 XFCE4 setup is still working fine through two kernel updates. One thing I was wrong about is I cannot move windows between external and internal screens, just the cursor. It's like they're, in effect, two seperate X sessions. I haven't changed anything, so far, as it's working well enough otherwise.

Arch, Antergos, Manjaro are using kernel 4.12.6-1 and 4.13.rc1 with the lastest and stable Nvidia drivers available in those distros, 375, 382,386, under Gnome Shell, XFCE4 and i3. What's changed is after boot I'm at two blank/black tty's. The external monitor "blinks" at the tty, a good sign. I found I can enter text and login to a DE. I don't use a display manager, just startx. In the desktop, it's as before. The external monitor is not active or detected. The egpu, GTX1060 is detected in Nvidia Settings Panel as is the external monitor. And that is disabled. I've regenerated an xorg.config and logged back out, to the blank tty, and back in with no change in result.

The blank tty's is, to me, a good sign, it's pointing to a driver configuration problem. When I have more free time I'll be back at it.

HTH


ReplyQuote
(@schultzern)
New Member
Joined:9 months  ago
Posts: 2
September 13, 2017 12:44 pm  

Hi all,

I too have been playing around with an external GPU in Linux for the past weeks. I have a 2017 Razer Blade Stealth and a Razer Core with an AMD Radeon RX 480. I've had no success in being able to use the external graphics card over the internal Intel HD 620. I've gotten the system to recognize the AMD card in lspci, but no success in using the external monitors connected to it. I've tested both Fedora 25 with open source drivers amdgpu, as well as Ubuntu 16.04 with amdgpu-pro. I've tested both KDE Plasma with X11, and GNOME with wayland/x11. 

Further I decided to try using a NVIDIA card and borrowed a Quadro K2200 from work. I was able to use it on Fedora 25 with KDE Plasma under X11 after installing nvidias proprietary drivers (think it was 384.xx something). However, the laptop screen didn't work and my internal graphics card (Intel HD 620) was not working anymore. If I booted up without the eGPU my laptop screen defaulted to lowest resolution (800x600). So only partial success so far...

Anyways, I was wondering how you configured X11 to use two graphics card simultaneously? And have you experienced that your external GPU have crashed your system? Recently my KDE Plasma session under fedora 25 have crashed when I plug in my eGPU, to the point that I have to do a forced shutdown... 

Any tips or ideas are much appreciated. 

- Razer Blade Stealth (2017) & Macbook Pro 13 (2017)
- Razer Core + AMD Radeon RX480 8GB


ReplyQuote
Da_Blitz
(@da_blitz)
Active Member
Joined:10 months  ago
Posts: 5
September 15, 2017 7:54 am  

I have some info at http://pocketnix.org/posts/eGPUs%20under%20Linux%3A%20an%20advanced%20guide that may be helpful, while i am using the auorus, it should apply to any eGPU box

amd needs a slightly different setup but at this time i don't have an amd card to test and write up info on

The TL;DR is you run 2 X11 servers, one for internal and one for eGPU and load some libs to have programs render on the second X11 server but display on the first

the NVidia supplied driver is missing a lot of features that would make eGPUs work seamlessly (DRI3) due to licensing issues


theitsage liked
ReplyQuote
(@schultzern)
New Member
Joined:9 months  ago
Posts: 2
September 15, 2017 10:52 am  

Thanks Da_Blitz. I read your advanced guide and its much appreciated. I compiled the 4.13 kernel and have had success with authorizing the eGPU over Thunderbolt with user security level. Progress!! I also noticed that somewhere someone was working on a daemon to handle thunderbolt connectivity, authorizing and firmware update, which is great news, as I'm currently using scripts to do this. 

I will try your method for the secure bios setting this weekend. I will also try with 2 X11 servers as suggested. It seems that my system is not to happy about hybrid graphics with AMD compared to how NVIDIA is working with bumblebee. 

- Razer Blade Stealth (2017) & Macbook Pro 13 (2017)
- Razer Core + AMD Radeon RX480 8GB


ReplyQuote
Johnny
(@johnny)
New Member
Joined:7 months  ago
Posts: 3
November 5, 2017 11:00 am  

Hi, I want to use an old SCSI card with my Sonnet Echo Express SEL TB3 on Ubuntu 16.04 LTS (4.4.0-98-generic). It seems that the SEL TB3 is recognized out of the box. Unfortunately, the SCSI card isn't recognized, but it works when connected directly to the mainboard (Z170N-Gaming 5, BOS F23a). Can anyone help me with this?


ReplyQuote
Johnny
(@johnny)
New Member
Joined:7 months  ago
Posts: 3
November 5, 2017 12:34 pm  

Some BIOS setting:

Security Option -> Setup (System)

Fast Boot -> Disabled (Enabled; Ultra Fast)

Storage Boot Option Control -> Legacy (UEFI; Do not launch)

Other PCI devices -> Legacy (UEFI; Do not launch)

TBT USB 3.1 Force Power -> Enabled (Disabled)

Security Device Support -> Disable (Enable)


ReplyQuote
Johnny
(@johnny)
New Member
Joined:7 months  ago
Posts: 3
November 5, 2017 12:39 pm  

[img] [/img]


ReplyQuote
(@aritax)
New Member
Joined:7 months  ago
Posts: 1
November 8, 2017 6:33 pm  

Hey,

I've played around with an eGPU under Linux now quite a bit and I've come pretty far.

My setup is the following:

-HP Spectre x360 2017

-Asus ROG XG Station 

Under Windows, all this is a piece of cake, you just need the TB3 driver for Windows installed.

Under Linux or specifically Ubuntu it got a lot easier since Ubuntu 17.10 came out. With the new Linux Kernel 4.13 the main step is done.

After reading the work of Da_Blitz, many things are pretty clear. Just authorizing the graphics dock with 

echo 1 > /sys/bus/thunderbolt/devices/0-0/0-1/authorized

connects the dock and everything works fine, except the graphics card.

The driver for my graphics card are allready in the Kernel out of the box, good so far, but it still didn't work, the graphics card is recognized in the system as video output, but is not listed as candidate by xrandr.

After A LOT of time and I am really ashamed how long it did take me to realize this, I came to the point to question wether or not Wayland even supports multiple Graphics cards.

I just started Gnome with xorg and tried all again and out of the blue, there was my graphics card.

I could even select it in the monitor settings.

Sadly, on the last step towards working, my monitor doesn't recognize any input from my graphics card.

Edit: I forgot to mention, I am using a AMD Radeon RX 580

Edited: 7 months  ago

ReplyQuote
Da_Blitz
(@da_blitz)
Active Member
Joined:10 months  ago
Posts: 5
November 9, 2017 10:08 am  

Wayland rendering works very differently to x11 in that wayland only implements surfaces and applications decide how they want to render to them. Ideally an application will use DRM render nodes which allow you to run things such as openGL and openCL workloads without having a screen attached (think of it as splitting u p a GPU into a rendering part and a display part and being able to use them separately)

I wish i would help out more on the AMD front, it is my understanding that the OSS drivers should work fairly well for eGPUs where as with nvidia only the Official non OSS drivers work

are you able to paste the output of xrandr and indicate which inputs belong to your machine and which to the GPU, i should be interested in seeing what the layout looks like for AMD and may be able to provide some feedback based on that


ReplyQuote
demch
(@demch)
New Member
Joined:6 months  ago
Posts: 1
November 25, 2017 12:42 am  

This is my howto for Ubuntu eGPU,  

Ubuntu 17.10, Intel Nuc7i7, AORUS GTX 1070 Gaming Box
Considerations:  We use X session no wayland

 

#Attach your device and check if you can see your Thunderbolt Controller
=========================================================================
fwupdmgr get-devices

#NUC7i5BN Thunderbolt Controller

#Install tbtadm
=========================================================================
sudo apt install cmake dkms net-tools libboost-filesystem-dev libboost-program-options-dev txt2tags build-essential pkg-config git glmark2
# https://github.com/01org/thunderbolt-software-user-space
git clone https://github.com/01org/thunderbolt-software-user-space.git

cd thunderbolt-software-user-space
mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr
sudo cmake --build . --target install
=========================================================================
sudo tbtadm devices
#0-1 GIGABYTE GV-N1070IXEB-8GD non-authorized not in ACL
sudo tbtadm approve-all
=========================================================================
sudo tbtadm devices
#0-1 GIGABYTE GV-N1070IXEB-8GD authorized in ACL
=========================================================================
lspci | grep VGA
#00:02.0 VGA compatible controller: Intel Corporation Device 5927 (rev 06)
#06:00.0 VGA compatible controller: NVIDIA Corporation GP104 [GeForce GTX 1070] (rev a1)
=========================================================================
sudo tbtadm topology
#Controller 0
#+- Details:
#| +- Name: NUC7i5BN, Intel Corporation
#| +- Security level: SL1 (user)
#|
#+- GV-N1070IXEB-8GD, GIGABYTE
#+- Details:
#+- Route-string: 0-1
#+- Authorized: Yes
#+- In ACL: Yes
#+- UUID: 00892ac8-305f-ed00-ffff-ffffffffffff

#Install drivers
=========================================================================
sudo ubuntu-drivers devices
#== /sys/devices/pci0000:00/0000:00:1c.0/0000:01:00.0/0000:02:01.0/0000:04:00.0/0000:05:01.0/0000:06:00.0 ==
#modalias : pci:v000010DEd00001B81sv00001458sd000036FCbc03sc00i00
#vendor : NVIDIA Corporation
#model : GP104 [GeForce GTX 1070]
#driver : nvidia-384 - distro non-free recommended
#driver : xserver-xorg-video-nouveau - distro free builtin

#== cpu-microcode.py ==
#driver : intel-microcode - distro non-free

sudo apt-get install nvidia-384

#Define a X device BusID must corresponse with the number reported by lspci
=========================================================================

lspci | grep VGA
#00:02.0 VGA compatible controller: Intel Corporation Device 5927 (rev 06)
#06:00.0 VGA compatible controller: NVIDIA Corporation GP104 [GeForce GTX 1070] (rev a1)

sudo vi /etc/X11/xorg.conf
=======================================

Section "Module"
Load "modesetting"
EndSection

Section "Device"
Identifier "nvidia"
Driver "nvidia"
BusID "PCI:6:0:0"
Option "AllowEmptyInitialConfiguration"
EndSection

===========================

#Flashing the firmware
=======================================================================
#Download and unzip the firmware Unpack your firmware, I used AORUS_N1070IXEB_8GD_VER10H2D.bin from
# https://www.gigabyte.com/Graphics-Card/GV-N1070IXEB-8GD#support-dl then update the firmware with:

sudo su
cat AORUS_N1070IXEB_8GD_VER10H2D.bin > /sys/bus/thunderbolt/devices/0-0/0-1/nvm_non_active1/nvmem
echo 1 > /sys/bus/thunderbolt/devices/0-0/0-1/nvm_authenticate

#Enable console text login in order to wait for the thunderbolt egpu initialization before starting x
=======================================================================
sudo vi /etc/default/grub
#GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"
#GRUB_CMDLINE_LINUX=""
GRUB_CMDLINE_LINUX="text"
GRUB_TERMINAL=console
=======================================================================
sudo update-grub
#sudo systemctl set-default graphical.target
sudo systemctl set-default multi-user.target

===================================================================
sudo reboot
login
wait for device initialization use lspci | grep VGA to verify
startx
# you can us glmark2 for test your set up
===================================================================

Edited: 6 months  ago

ReplyQuote
striges1111
(@striges1111)
New Member
Joined:1 year  ago
Posts: 1
November 29, 2017 3:31 am  

It would be simpler to add a udev rule to authorize TB devices, accorind to  https://01.org/linuxgraphics/gfx-docs/drm/admin-guide/thunderbolt.html  (Verified on Thinkpad P50,  RHEL 7.4 with 4.14 kernel). 

Those users who just want to connect any device without any sort of manual work, can add following line to/etc/udev/rules.d/99-local.rules:

ACTION=="add", SUBSYSTEM=="thunderbolt", ATTR{authorized}=="0", ATTR{authorized}="1"



ReplyQuote
(@jerry_normandin)
Active Member
Joined:8 months  ago
Posts: 10
December 9, 2017 4:21 am  

I have the eGPU working in MacOS on my MacPro 2013 black can.. but I built the eGPU for Linux.  I'm dumping MacOS.

This is the issue I have when attempting to load the nvidia module:

9485.653635] NVRM: This PCI I/O region assigned to your NVIDIA device is invalid:
NVRM: BAR1 is 0M @ 0x0 (PCI:0000:19:00.0)
[ 9485.653636] NVRM: The system BIOS may have misconfigured your GPU.
[ 9485.653644] nvidia: probe of 0000:19:00.0 failed with error -1
[ 9485.653705] NVRM: The NVIDIA probe routine failed for 1 device(s).
[ 9485.653706] NVRM: None of the NVIDIA graphics adapters were initialized!

 

I have the 2 onboard AMD D700 cards and the 1 Nvidia GTX970 in an Akitio Node

fwupdmgr get-devices

AKiTiO Node
DeviceID: d6010000-0092-7518-a21a-f4d4d6e1ca18
Guid: c6da27ea-e8f8-5b21-9c80-5bd3a310b4da
Plugin: thunderbolt
Flags: updatable|registered
DeviceVendor: inXtron
DeviceVendorId: TBT:0x0041
Version: 25.0
Created: 2017-12-09

GM204 [GeForce GTX 970]
DeviceID: ro__sys_devices_pci0000_00_0000_00_01_0_0000_10_00_0_0000_11_02_0_0000_14_00_0_0000_15_03_0_0000_17_00_0_0000_18_01_0_0000_19_00_0
Guid: 750d6e9a-997a-5414-858f-37e1d3315e15
Plugin: udev
Flags: internal|registered
DeviceVendor: NVIDIA Corporation
DeviceVendorId: PCI:0x10DE
Created: 2017-12-09

tbtadm approve-all
Found domain "/sys/bus/thunderbolt/devices/domain2"
Approval not relevant in SL0

root@bart-linux:~# tbtadm devices
2-101 Sonnet Technologies, Inc. USB 3.0 & eSATA Thunderbolt Adapter authorized not in ACL
2-3 DataTale RS-M4T authorized not in ACL
2-1 Other World Computing ThunderBay 4 authorized not in ACL
1-3 Other World Computing ThunderBay 4 authorized not in ACL
0-1 inXtron AKiTiO Node authorized not in ACL

lspci | grep VGA
02:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Tahiti XT [Radeon HD 7970/8970 OEM / R9 280X]
06:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Tahiti XT [Radeon HD 7970/8970 OEM / R9 280X]
19:00.0 VGA compatible controller: NVIDIA Corporation GM204 [GeForce GTX 970] (rev a1)

 

PCI BAR for address 19.00.0 isn't initializing properly.  There's no io memory assigned, BAR doesn't initialize.

I think it's because I already have 2 GPUs. and lots of other PCI devices on thunderbolt (my ThunderBay's for JBOD).

I need to modify the kernel to allocate more memory for PCIe devices.. I believe that's why BAR isn't initializing for PCIe 19.00.0..

 

At least this is the limitation I am running into with Linux running bare metal on my MacPro 2013 desktop tin can.

MacOS 10.11.6 is running with the eGPU.    I've been attempting to reverse engineer what automate_egpu.sh  does.

On a mac the IOPCITunnel hack gets around the issue I am attemtpting to resolve on Linux. It appears to be a root bridge work around.   Has anyone seen an  equivalent kernel module for Linux  ? Apparently the 2013 MacPro desktop needs it if both AMD D700 gpus are installed.

 

 

 


ReplyQuote
(@eugene_rogach)
New Member
Joined:1 month  ago
Posts: 1
April 14, 2018 10:33 am  

@demch please help me!
how should I edit /etc/X11/xorg.conf ? 
I have a problem when I do 'startx', server is stopped, no screens.

ubuntu 17.10 kernel 4.13


ReplyQuote
(@matt_deboer)
New Member
Joined:7 days  ago
Posts: 1
May 17, 2018 9:00 pm  

@jerry_normandin did you ever figure out your issue?

I'm at a similar place with a macbook pro 14,3 running ubuntu 18.04 (

"This PCI I/O region assigned to your NVIDIA device is invalid:"

)
Also got:

$> tbtadm approve-all
Found domain "/sys/bus/thunderbolt/devices/domain0"
Approval not relevant in SL0

Note: I also tried with kernel param "pci=nocrs" to no avail :/

did you ever resolve?


ReplyQuote
karatekid430
(@karatekid430)
Eminent Member
Joined:7 months  ago
Posts: 43
May 22, 2018 5:53 am  
Posted by: Sky11

try to disable Thunderbolt security in the Device Manager - it should help with detection

Especially coming from AMD, it is a bad suggestion to do this. Thunderbolt without security allows for DMA attacks which can break into a fully encrypted computer.

Instead, you just authorise the device "echo 1 | sudo tee /sys/bus/thunderbolt/devices/X-Y/authorized"

A properly functioning IOMMU will defend against these attacks, but supposedly Intel could not guarantee its presence always. I hope in the future they drop security levels and enforce IOMMU usage. Or add a security level "SL5: no security if IOMMU is present".

Otherwise, AMD's amdgpu.ko driver in Linux is still giving ATOM BIOS errors when the eGPU is approved under Thunderbolt bus (AMD R9 Nano / Fury = Fiji). I tried in 4.17-rc6 kernel and still no fix. Also if using amdgpu.dc=1 (default for all GPUs since 4.17), it has nasty crashes the moment a display is attached to the external graphics.

When you do get it working (it usually works after failing on one Thunderbolt port and moving it to the other), my main issue is the xorg.conf - I need a conditional thing where "if amdgpu detected, use it, else carry on and ignore that it is missing". I can get it working *perfectly* by specifying it to use xorg on the amdgpu and disabling the laptop display. But then when I boot next without the eGPU attached, the graphical interface fails and you need to use vt switching to login, remove the conf file and restart the display manager.

With no conf, sometimes I can get both displays working - but if I put it on the display on the GPU that is not running the X server, with the main one disabled, it goes incredibly laggy. For some reason, with both enabled (mirrored or extended), it works pretty well. With only the display on the GPU running the X server, it is perfectly smooth.

The surprising part here is that the eGPU loopback (rendering on external Radeon and passing back to internal display with no external monitor) is the part that works almost perfectly (after you sort out ATOM BIOS). You can either put DRI_PRIME=1 in /etc/environment - or just pass it manually to applications. I was running Unigine benchmarks. But AMD needs to put some more options in driver /sys/class/dri/.... to change fan profile. Of late, it has been favouring quietness over coolness, and even cut out from over temp a couple of times. Anyways, it works well until you try to use external monitors, and then it all falls apart (or lags your bum off).

TL;DR = if anybody can find out how to make xorg.conf continue when a specified device is not present, and fall back to the next device, that would help progress a lot.

Thanks people!


ReplyQuote