$ \newcommand{\sysusb}{\textsf{sys-usb}} \newcommand{\DomZ}{\textsf{Dom0}} $I have been using Qubes as my primary OS¹ for a while now, and I am extremely happy with the level of control it provides me over my apps. Recently someone raised an interesting question on the Qubes OS forum:

Hello,

Is there a way to attach a particular USB device (by Vendor ID/Product ID) to a Qube automatically on connect? I have a sys-usb to manage USB devices.

The USB device I am using frequently becomes disconnected (and reconnected) during use and it only needs to be connected to one particular Qube.

Thanks,
N

https://forum.qubes-os.org/t/usb-device-auto-attach-to-qube/5977

I have wondered about this sort of automation before. I have a bunch of USB devices: external disks, crypto wallets, password managers, etc. which I attach to specific VMs immediately after plugging in. It would be nice to be able to attach trusted devices to certain VMs automatically.

On a “regular” OS, such as Debian or Fedora, triggering actions on connecting / disconnecting USBs devices is typically achieved using udev rules. We could match on the Vendor ID / Model ID etc. to detect when a particular USB device is added or removed, and may RUN a desired script on that event. However, this isn’t as straightforward with $\sysusb$ on Qubes OS, since it typically would have no access to other running VMs and cannot attach devices to them directly.

I proposed an initial idea on how this could be done, and this weekend I finally had some time to try it out 🙂 The high-level plan is outlined below:

1. Since $\sysusb$ cannot escape virtualization and directly attach devices to other VMs, we must somehow communicate via the AdminVM (i.e., $\DomZ$), which has access to all VMs.
2. Communication in Qubes typically happens using an elegant remote procedure call (RPC) framework, called Qrexec, so we must write some new Qrexec service that receives a device attachment request and fulfills it.
3. Our Qrexec service will be listening in $\DomZ$, and a $\sysusb$ client must pass the target VM name and device name to this service from a udev trigger.
4. Finally, we must also setup an appropriate Qrexec policy for this service in $\DomZ$.

If you are unsure of what RPC / Qrexec / udev are, then I would strongly suggest familiarizing yourself with those tools first before making any changes to your system.

Changes in $\DomZ$

There are two main changes necessary in $\DomZ$:
(a) a new Qrexec service to listen to device attachment requests from $\sysusb$,
(b) a Qrexec policy to restrict the source and destination VMs for calls to this service.

The Qrexec Service

First, let’s start with the new Qrexec service that would handle requests from $\sysusb$. I am only going to describe how to automatically attach block devices, but attaching full USB devices is very similar (see comment at the end of this section).

There are a few different ways of implementing this service. One important choice to make is regarding the mode of communication with $\sysusb$. Only stdin/stdout is passed between the Qrexec server and client — in particular, there is no support for command-line arguments in Qrexec. However, instead of communicating over stdin/stdout, I chose to use a service argument, which is actually passed as a single command-line argument to the service. A service argument, unlike stdin/stdout communication, is visible on the $\DomZ$ prompt, so the communication is a bit more transparent. Moreover the service argument is also sanitized by Qrexec, before being supplied to the script.

However, since Qrexec only allows a single service argument currently, we must pack our target VM name and device name together. An example prompt from my $\sysusb$ should be displayed below. custom.USBDeviceAttach is the service that we are going to create, and work+__+sda is the service argument. In this case, I pack the target VM name (work) and block device name (sda within $\sysusb$) together by concatenating both using +__+.

The service script itself is pretty simple, as I show below.

#!/usr/bin/env bash

# Exit immediately if any of the following commands fail
set -e

# Expect exactly one argument -- the service argument
[ "$#" -eq 1 ]

# Unpack service argument and expect exactly 2 strings
ARG=($(sed 's;+__+; ;g' <<< $1))
[ "${#ARG[@]}" -eq 2 ]

# Make sure we have a remote VM name from Qrexec
[ -n "$QREXEC_REMOTE_DOMAIN" ]

# Start the target VM, if needed
qvm-start --skip-if-running "${ARG[0]}"

# Attach the target device from remote VM to target VM
qvm-block attach "${ARG[0]}" "${QREXEC_REMOTE_DOMAIN}:${ARG[1]}"

To make it available to Qrexec as a new service, this script must exist under /etc/qubes-rpc and must be marked as executable (chmod +x). I used /etc/qubes-rpc/custom.USBDeviceAttach, for instance.

As mentioned before, tweaking this script to work for full USB passthrough is easy — the qvm-block in line 20 could be changed to qvm-usb. However, full USB passthrough is less secure.

The Qrexec Policy

For each Qrexec service, we must also specify a Qrexec policy to whitelist calls from specific VMs. In this case, we want to allow $\sysusb \rightarrow \DomZ$ calls, and deny all other calls:

sys-usb dom0 ask,default_target=dom0

@anyvm @anyvm deny

If you have multiple USB qubes, perhaps with different USB controllers assigned to each one, you might create copies of the first line and modify accordingly. The last line prevents all other (non-USB) qubes from triggering this Qrexec call.

This file should exist under /etc/qubes-rpc/policy, and must have the same name as the service file. I used /etc/qubes-rpc/policy/custom.USBDeviceAttach, for instance.

The ask in the first line could be changed to allow to skip the $\DomZ$ prompt, and allow all requests from $\sysusb$, but beware that this is a significant security risk. A compromised $\sysusb$ may request $\DomZ$ to attach potentially harmful devices to arbitrary VMs, and these requests would be fulfilled without without any user approval!

Changes in $\sysusb$

Finally, in $\sysusb$ we need to trigger our new Qrexec service when USB devices are connected to automatically attach them to desired VMs. But, before automating this process with udev, we should first manually test the RPC by running:

qrexec-client-vm dom0 custom.USBDeviceAttach+my-vm+__+sda

where $\textsf{sda}$ should be a block device (e.g., /dev/sda) in $\sysusb$, and $\textsf{my-vm}$ should be some target VM. If everything has been setup correctly, $\DomZ$ should display a prompt, similar to one in my screenshot above. On approving this request in $\DomZ$, the device should be correctly attached to the target VM, which can be checked by running qvm-block in $\DomZ$.

If everything works correctly, then this Qrexec call could be automated via a udev rule in $\sysusb$. As an example, I have the following rule in a new file /etc/udev/rules.d/50-auto-attach.rules:

ACTION=="add", SUBSYSTEM=="block", KERNEL=="sd[a-z]", \
RUN+="/bin/sh -c 'qrexec-client-vm dom0 custom.USBDeviceAttach+my-vm+__+%k &'"

This rule will match new block devices with kernel names matching sd[a-z] regex, so partitions such as sda1 etc. wouldn’t match this rule. Upon a successful match, the RUN command is executed, and $\DomZ$ is requested to attach the device to $\textsf{my-vm}$. Beware that this rule is not very secure and must be constrained with additional ATTRS or ENV values of the particular device to match on. An introduction to Udev is an excellent quick tutorial on this topic.

Unblocking udev

Observe that the RUN command has an & at the end, and thus runs in a detached shell. This is not a performance optimization, but is necessary to resolve a dependency cycle:

1. udev stays blocked on RUN scripts — a USB device is connected and available in $\sysusb$ only after this script terminates
2. However, the Qrexec call triggered by the run script expects the device to already be available in $\sysusb$, and attempts to attach it to $\textsf{my-vm}$!

To resolve this dependency cycle, we run the Qrexec call in a detached shell and unblock udev. By the time $\DomZ$ processes the RPC, receives user approval (via the prompt), and attempts to attach the USB device to another VM, udev should have finished connecting the device (usually instantaneous).

This (using &) is not the correct way to detach scripts from udev. There are multiple threads online regarding this, e.g., see: this StackOverflow thread, this AskUbuntu thread, and this blogpost. However, this approach suffices for our case, because Qrexec does not require the client to stay alive after making the RPC. So although udev would kill the detached script after connecting the USB devices, $\DomZ$ would have received the call and would carry out the requested operation. $\DomZ$ might show a warning popup if the client is killed before $\DomZ$ finishes attaching the device and returns a success code to client:

However, this warning is completely harmless. Any of the solutions proposed in the above threads could be implemented to properly detach the script and get rid of the warning.

Persistent udev Rules

Typically $\sysusb$ is an application VM (called AppVM in Qubes). Since the root filesystem for AppVMs is discarded on shutdown and reset to the corresponding TemplateVM root filesystem, the new rules saved under /etc/udev/rules.d/ would not persist across reboots. Therefore, the 50-auto-attach.rules file should be saved in the AppVM’s persistent storage (within /rw), and must be loaded into udev during boot. For instance, I have saved it at /rw/config/50-auto-attach.rules and have the following in my /rw/config/rc.local init script:

cp /rw/config/*.rules /etc/udev/rules.d/

udevadm control --reload-rules && udevadm trigger

Furthermore, for persistent DispVM $\sysusb$ configuration, the changes above must be made in the DispVM template, and not $\sysusb$ directly.