Tag: ROCM

Posted on
by

So didn’t show my GPU on when started via . I have an and all the (#ROCr / ) stuff installed. It only listed the iGPU by Intel on startup:

[---] OpenCL: Intel GPU 0: Intel(R) UHD Graphics 630 (driver version 23.35.27191.9, device version OpenCL 3.0 NEO, 25561MB, 2556>
[---] libc:  version 2.37

This works however fine when I run boinc manually as user (or clinfo for the matter), and not via systemctl start boinc-client, so I guessed it’s some permission issue. journalctl had the context I was looking for and threw this in the middle of the boinc-client startup:

audit[305157]: AVC avc:  denied  { read write } for  pid=305157 comm="boinc" name="kfd" dev="devtmpfs" ino=532 scontext=system_u:system_r:boinc_t:s0 tcontext=system_u:object_r:hsa_device_t:s0 tclass=chr_file permissive=0

This is SELinux’s charming way of telling me that it blocked read and write access to /dev/kfd (the main compute interface shared by all GPUs, according to the ROCm manual) for the boinc process. Nice. So what most users do now is grumble and disable SELinux, which is kinda a bad idea. The more advanced user does this and calls it a day:

sudo ausearch -c 'boinc' --raw | audit2allow -M boinc
sudo semodule -i boinc.pp

This basically prepares an override policy based on any rejected boinc activity that looks in my case like this:

module boinc 1.0;

require {
	type hsa_device_t;
	type random_device_t;
	type boinc_t;
	class chr_file { ioctl map open read write };
}

#============= boinc_t ==============
allow boinc_t hsa_device_t:chr_file { ioctl map open read write };
allow boinc_t random_device_t:chr_file write;

Not today though. It left me befuddled with the following output:

libsemanage.semanage_direct_install_info: Overriding boinc module at lower priority 100 with module at priority 400.
Failed to resolve typeattributeset statement at /var/lib/selinux/targeted/tmp/modules/400/boinc/cil:3
Failed to resolve AST
semodule:  Failed!

…and I have no idea why. I also found nothing on Google Search. So to not be DenverCoder9 (https://xkcd.com/979/) in the future here is what I found out so far:

sudo cat /var/lib/selinux/targeted/tmp/modules/400/boinc/cil | bunzip2 
(typeattributeset cil_gen_require hsa_device_t)
(typeattributeset cil_gen_require random_device_t)
(typeattributeset cil_gen_require boinc_t)
(allow boinc_t hsa_device_t (chr_file (ioctl map open read write)))
(allow boinc_t random_device_t (chr_file (write)))

Apparently it can’t resolve the required typeattributeset boinc_t – which is kinda odd as it exists (see sudo semodule -X 100 --cil -E boinc and the resulting cil file). Frankly this is where SELinux lost me too. I found the man page for boinc_selinux, which is not really known on my Fedora system here, so I may be missing something. It suggests to enable permissive mode for boinc_t (instead of dropping SELinux altogether):

Note: semanage permissive -a boinc_t

can be used to make the process type boinc_t permissive. Permissive process types are not denied access by SELinux. AVC messages will still be generated.

https://linux.die.net/man/8/boinc_selinux

And sure enough on the next restart my AMD GPU became available:

[---] OpenCL: AMD/ATI GPU 0: AMD Radeon RX 6700 XT (driver version 3558.0 (HSA1.1,LC), device version OpenCL 2.0, 12272MB, 12272>
[---] OpenCL: Intel GPU 0: Intel(R) UHD Graphics 630 (driver version 23.35.27191.9, device version OpenCL 3.0 NEO, 25561MB, 2556>
[---] libc:  version 2.37

Happy numbers crunching. Mebbe some fix for SELinux crosses my path in the future so I can update this with the proper solution.