No matter how you are using your computer, anyone would love to have an easy way to get their computer to do a little magic when they plug something in. In this Reddit thread replying to my post about using borg in an automated backup system, user valkun expressed an interest in automating backup with USB hard drives. The optimal behavior would be to have backup run, periodically, after plugging in a USB-powered portable hard drive for backups, and then fall back to a monitoring ready-state when disconnected. This is something I'd really like to have for my laptop (which doesn't really have a backup solution at all, because it's not easy enough). But maybe what you really want is to automatically download your videos and pictures from your camera and feed them into a processing pipeline, or sync music to your phone in a fancy way that's not easy with other solutions. So, rather than post a lightly sanitized version of a setup I've worked out for myself, in this post we'll play with some systemd units and fairly simple bash, and build our way up to more generalizable ideas.
Googling around, the first systemd units I thought to try would be mount units, because duh we want to watch for a block device mounting and unmounting as the first trigger. Although possible, messing with two OS layers (both systemd units and udev rules) instead of one is an obvious increase in work. Also, I'm not so excited about the added not-quite-portable quality of writing udev rules, seems quite hand-made, meaning hand-installed and fussy. It might be that some distros don't have automatic device mounting built in, but since I run Ubuntu it's not something I worry about. Obviously they have their uses, but not needed when we already have automount and a powerful set of abstractions that are built into a familiar interface: the filesystem.
A little mise-en-place before we start:
We are going to need at least two terminals: one to do all our sudoing in (moving unit files, enabling and disabling them), and one to watch the .service units as they're working. I would also need one to use an editor with root privileges to modify our .path and .service units. I use nvim for basically everything (bit of a mental illness I suppose), so I'm not sure if something like atom or sublime or gedit or whatever might easily take the place of $ sudo nano to edit files in a root-owned directory. Additionally, if you have a favourite editor for modifying the bash files we'll be using, obviously you'll want to have that too. It doesn't really matter if you have lots of windows or just one
I suppose I'll be doing more of these semi-instructional posts in the future, so here's a repo I've made to stash related files to save you a little bit of copy & paste. If you don't have git or don't want to mess with it, the files are easily downloadable in many different ways. Or you can just Ctrl-C Ctrl-V. Finally, very important, you need to modify the files as they are to put your username (which you see at the terminal prompt in most default setups) in place of the $USER variable. systemd does not expand variables in the units like bash or your shell of choice does with .sh files. If the examples aren't working right away, check to make sure you don't have any $ENVVARs hanging around in your unit files.
For the last step, we'll want to have a device just for the fun of it, especially if you are going to use this for an automatic portable backup system.
Let's first look at the content of the units we're using here, and remember to edit the copies you've got for this demo with your username:
test-path.path[Unit] Description=PathTest [Path] PathModified=/home/$USER/Desktop [Install] WantedBy=multi-user.targettest-path.service[Unit] Description=PathTest [Service] ExecStart=/bin/echo "Something changed on your Desktop!"
Very basic unit files, and if you're unfamiliar with the [Install] attribute, it basically designates the portion of the service unit that you enable and start.
Copy test-path.path and test-path.service into place:
Terminal 1
$ sudo cp test-path* /etc/systemd/system/
Now in your terminal for watching things, start watching before you flick the switch:
Terminal 2$ journalctl -f -u test-pathTerminal 1$ sudo systemctl enable test-path.path $ sudo systemctl start test-path.path $ systemctl status test-path.path
So far, the service unit should be correctly installed, and you shouldn't have anything going on the journalctl -f as it's actually watching test-path.service for its events. Now put an empty file on the ~/Desktop and delete it, use touch, do whatever and see it in action on the monitoring terminal:
Terminal 1
$ echo "" > ~/Desktop/test.file
$ touch ~/Desktop/test.file
$ rm ~/Desktop/test.file
So by now you've hopefully seen that everything works as expected, and you can stop the journalctl -f with Ctrl-C in the terminal (it inputs a byte called SIGINT which signals the terminal to interrupt the process). So let's clean up and move to the next demo:
Terminal 1
$ sudo systemctl stop test-path.path
$ sudo systemctl disable test-path.path
$ sudo rm /etc/systemd/system/test-path*
For this second demo, we're going to have some more complex logic watching a path with error codes, and a RestartSec= attribute on the .service unit. As an aside, my expierments with .timer units was really frustrating, as they are designed for continuous execution, rather than be interrupted, meaning that once triggered they wouldn't stop running, which is not the behavior we're looking for. That led me to eventually finding the RestartSec= attribute option for .services, which is exactly what we want. There's also some logic in the .service file which can watch for the exit codes from the process it starts, restarting itself on a normal 0 exit (in this case, there are lots of options), or not when exit is 1. Let's have a look at the files:
filecheck.sh#!/usr/bin/env bash if [ -f $1 ]; then echo "File $1 exists!" exit 0 else echo "File $1 does not exist." exit 1 fitest-timer.path[Unit] Description=Testing Paths & Timer [Path] PathExists=/home/$USER/Desktop/test.file [Install] WantedBy=multi-user.targettest-timer.service[Unit] Description=Testing Paths & Timer [Service] # If your install location of filecheck.sh differs, change here ExecStart=/home/$USER/.local/bin/filecheck.sh /home/$USER/Desktop/test.file RestartSec=3s Restart=on-success
As you can see, filecheck.sh is passed a file, in this case ~/Desktop/test.file as defined in the .service file, checks to see if it exists, and prints a message and corresponding exit code. This service will execute every three seconds, unless test.file doesn't exist where test-timer.path is watching and sends the error exit code, stopping the service. Copy test-timer.path, test-timer.service and filecheck.sh into place, enable and start the watching .path unit, and monitor the service that unit fires with journalctl -f:
Terminal 1$ cp filecheck.sh ~/.local/bin $ chmod +x ~/.local/bin/filecheck.sh $ sudo cp test-timer* /etc/systemd/system/ $ sudo systemctl enable test-timer.path $ sudo systemctl start test-timer.pathTerminal 2$ journalctl -f -u test-timer
Now again make our test.file on the Desktop and watch what happens. Delete the file, change its name, move it somewhere else, and watch the service error. Restore the file to where it should be, watch the service resume. Amazing, right?! Stop, disable, and delete the timer-test unit files, and let's continue.
Okay, you made it this far, you probably already know what to do (if you're an accomplished Linux user and have rolled-your-own backup system). We're going to use sync.sh to conditionally rsync some file to some destination, if it exists. We're going to use a .path unit to watch for our USB destination, which I've assumed is labelled "Backup", but is easy enough for you to change to whatever your USB device is labelled (especially since you've got to replace all the $USERs). We're going to have a .service unit start the sync.sh process every five seconds, unless the destination /media/$USER/Backup doesn't exist, in which case sync.sh exits with an error and test-sync.service stops until test-sync.path starts it again. Let's look at the files:
sync.sh#!/usr/bin/env bash if [ -d $1 ]; then echo "Disk $1 is mounted" rsync $2 $1 exit 0 else echo "Disk $1 is missing" exit 1 fitest-sync.path[Unit] Description=SyncTest [Path] PathModified=/dev/disk/ PathExists=/media/$USER/Backuptest-sync.service[Unit] Description=SyncTest [Service] # If your install location of sync.sh differs, change here ExecStart=/home/$USER/.local/bin/sync.sh /media/$USER/Backup /home/$USER/Desktop/test.file RestartSec=5s Restart=on-success
So again, change the files so they'll work for you, and use the label of your USB device that you see under /dev/disk/by-label/ when you plug it in. If your device has a label with spaces in it, (which in bash you'd spell USB\ Device\ Name) getting the properly escaped version to use for a .path unit is easy using systemd-escape. Use it like:
$ systemd-escape /media/$USER/USB\ Device\ Name
and it returns "-media-andy-USB\x20Device\x20Name", properly substituting my user name and everything. Piece of cake!
Install, enable, start, monitor, and echo a test.file onto your Desktop. Use your terminal for watching things to observe success and failure messages from sync.sh by following the log for test-sync.service. For added fun, open yet another terminal and watch test.file on your backup:
Terminal 1$ sudo cp test-sync* /etc/systemd/system/ $ sudo systemctl enable sync-test.path $ sudo start sync-test.path $ echo ~/Desktop/test.fileTerminal 2$ journalctl -f -u test-syncTerminal 3$ watch cat /media/$USER/Backup/test.fileTerminal 1 (again)$ echo "here's some text to sync." > ~/Desktop/test.file
Echo some changes into the file, watch it change, everything is great, right? Now, without stopping anything, just unmount your USB drive. Watch the error thrown, and the service stop. Un-plug and re-plug your USB, and see it restart. Magic! Don't forget to tidy up all the junk I've encouraged you to put on your system, and get to work on some services for yourself.