Note: This post is part of a series. Each post builds on the previous ones. If you are just trying to add one thing to an existing system that was not built following this series, then I cannot promise that these instructions will work for you, although they probably will. If you’ve started from something other than a non-NOOBS Raspbian image, then you’ll probably need to adjust for that.

Please refer to the series Introduction for a list of all the different posts in the series.

Self-Promotion: I have recorded this series as a screencast for Pluralsight:
(http://www.pluralsight.com/courses/raspberry-pi-home-server)
If you have a Pluralsight subscription, please consider watching it. Reading the instructions is one thing, but watching it done demystifies the whole process.

Thank you!

---

Now that the Raspberry Pi Home Server is up and running off of the hard drive, we’re presented with a problem. We can’t just take a backup of the SD card anymore. In fact, what’s on the SD card probably shouldn’t change from this point on. All the important stuff is on the OS partition of the hard drive.

In this post, I’ll present a couple different methods for backing up the hard drive using a secondary computer, or just using the Pi itself and a second SD card.

Backing up from a Windows computer

I’ve been using and advocating HDD Raw Copy over Win32DiskImager in this second edition of the Raspberry Pi Home Server series because of its ability to compress the image files on the fly, and that ability can be put to good use here as well. All you need to do is hook the Pi’s hard drive up to your Windows computer, and use HDD Raw Copy to copy the drive to an image file just like you’ve been doing with the SD card. It’s exactly the same process. If you’re using Win32DiskImager, the process is the same, only the resulting image files will be the same size as the drive you’re backing up, which brings me to the downside of this method.

Both HDD Raw Copy and Win32DiskImager are going to want to back up the entire hard drive. Not just the OS partition, but the data partition as well. If you have the space to keep that kind of backup around, then good for you, you can restore the entire server, including the OS and Data partitions as well as all of their contents in one go. It’s going to be slow, and it’s going to take up space, but you can do it.

This is part of the reason I keep my actual shared stuff on a completely separate drive now. I like the separation between the devices that make up the Pi itself, and the stuff that the Pi is sharing. I only have to back up a 120GB drive, and it’s mostly empty space, so my compressed image files are not that large.

If you can build your setup like this, I highly recommend it. If not, then you’ll need to continue reading and decide which is right for you.

Backing up from the Pi itself

If, for some reason, you don’t have access to a full Windows, Mac, or Linux computer, you can back up your hard drive by using another Pi. If you’re stranded alone on a desert island and only have a single Pi with you, then you’ll need to do a little prep work, the first step of which will be establishing a power grid.

Since this is the worst-case scenario, we’ll get it out of the way now. We’re going to create another SD card specifically for making backups. The Raspbian Lite image will have everything we need on it, or you can use a full Raspbian installation too. Just prepare another SD card, exactly like we did in “Installing the OS“, and then proceed to the “Backing up from Linux” section.

Backing up from Mac OS

Mac OS is built on top of Unix, which Linux is a sort of clone of, so things work pretty much the same there. You’ll just need do the same thing as the Linux users in the next section.

Backing up from Linux

The process for backing up from Linux is exactly like what you did to create the SD card in the first place, except that you’ll be using dd to copy from the Pi’s OS partition to another spot in your filesystem. This could be somewhere on the hard drive of your main computer, but for this demo I’ll be creating an image of the OS partition on the Data partition of the same drive. Just adjust the file paths to match where you want the image and you’re set to go.

Shut down the Pi, and transfer the hard drive to your main computer, or swap SD cards to the new one you created above. If you’re doing this from a single Pi, then I would also take this opportunity to unplug any other drives you might be using, leaving nothing but the system drive attached, and then start up the Pi with the new SD card.

Use blkid to figure out where the Pi’s hard drive got attached (not mounted, mind you).

sudo blkid

In my case, the hard drive shows up at /dev/sda, and the partitions are /dev/sda1 and /dev/sda2, with sda1 being my OS partition, and sda2 being my data partition. If you’re using a separate Linux system, then sda is probably your own hard drive, and the Pi’s drive will be sdb, sdc, sdd, etc.

I’m writing my image to the second partition of the same hard drive, so I’ll need to mount it first so that I can write files to it. Since this is the first time I’m doing this, I’ll also need to create a placeholder directory for the drive to mount at.

sudo mkdir /mnt/data
sudo mount /dev/sda2 /mnt/data

Now, we’ll use the dd command to copy the sda1 partition to sda2 (or wherever you’ve decided to store the image). The dd command is part of “coreutils”, and newer versions have the ability to give you a progress report as they go. See which version you are running.

dd --version

If it says 8.24 or above, you’ll be able to get a status report with the “status=progress” switch. Back up the OS partition like this.

sudo dd bs=1M if=/dev/sd?? of=IMAGEPATH.img status=progress

Unfortunately, Raspbian is still on 8.23 at the time I’m writing this, so status=progress won’t work. If your OS partition is still relatively small, then doing without feedback is a perfectly viable option, so you can proceed like this.

sudo dd bs=1M if=/dev/sd?? of=IMAGEPATH.img

If your OS partition is very large, or you opted to have just one partition, then you’ll probably want some kind of feedback. Install the dcfldd package through apt-get.

sudo apt-get install dcfldd

The syntax for dcfldd is very similar to that of dd, but it will report its progress as it goes.

sudo dcfldd bs=1M if=/dev/sd?? of=IMAGEPATH.img

When the copy is complete, you can shut down the Pi, or safely remove the hard drive card from your main computer, plug it back into the Pi or swap your SD cards back to normal, make sure everything’s hooked back up like it was before, and restart the system.

Restoring from a backup image

If you need to restore your system to a previous state, the process is just the same, but with the “if” and “of” parameters flipped like they were when you created the original SD card. You’ll need to read from the image file and write it to the hard drive partition.

Shut down the system and move the hard drive to the computer that will be doing the restore, or swap to the “backup” SD card if you’re doing this directly from the Pi, and run the dd or dcfldd command as appropriate. Don’t forget to mount the data partition if you’re doing this from the same Pi. You might want to edit the /etc/fstab file so that this happens automatically whenever you’re using your “backup” SD card.

sudo dd bs=1M if=IMAGEPATH.img of=/dev/sd??

or, if you’re doing this from the same Pi

sudo mount /dev/sda2 /mnt/os
sudo dcfldd bs=1M if=/dev/sd?? of=IMAGEPATH.img

Easy, right?

What’s next?

Everything after this point is purely optional. Return to the Introduction, and follow whatever links you want, mixing and matching whatever additional components you want your server to have.

Note: This post is part of a series. Each post builds on the previous ones. If you are just trying to add one thing to an existing system that was not built following this series, then I cannot promise that these instructions will work for you, although they probably will. If you’ve started from something other than a non-NOOBS Raspbian image, then you’ll probably need to adjust for that.

Please refer to the series Introduction for a list of all the different posts in the series.

Self-Promotion: I have recorded this series as a screencast for Pluralsight:
(http://www.pluralsight.com/courses/raspberry-pi-home-server)
If you have a Pluralsight subscription, please consider watching it. Reading the instructions is one thing, but watching it done demystifies the whole process.

Thank you!

---

Running the Raspberry Pi off of an SD card is simple, affordable, and very convenient. SD cards are also very small compared to hard drives, though, and they can only be written to a finite number of times. That number is ridiculously large, but it can still be used up faster than you think by things like virtual memory swap files which are written and re-written constantly.

If you plan to run this server 24/7 and actually use it, you’re probably going to want something a little more robust. My current setup is using a 120GB SSD for the system drive. I’m using an SSD because it can be powered entirely from the Pi’s USB port. I could have gone with something smaller, both storage-wise and physically by buying a small MSata drive and an adapter, but in the end it was cheaper to buy a normal 2.5″ SSD and put it in a case I already had.

This SSD has two partitions on it. The first is the “os” partition, which I left free back in the post “Adding a Hard Drive“. The second is my “data” partition, and holds information about my collection such as MiniDLNA’s database. I don’t actually put my media files or public share on the data partition anymore. My public share now lives entirely on an external RAID enclosure. In the original version of this post, I used the RAID box as the system drive, but over time I have discovered that RAIDs don’t work very well in this capacity, so I just use the RAID for my shares now. Everything that is the Raspberry Pi Home Server itself lives on the SD card and on the SSD.

Your setup doesn’t need to have as many moving parts, though. A typical Raspberry Pi Home Server setup might consist of a Raspberry Pi attached to a single, very large, hard drive. You can get 5TB external drives for under $150 these days. You only need to partition off a small space for the Pi to boot from, and the rest is yours to store most, if not all, of your stuff. This is how my CrashPi works.

Booting from a hard drive

Okay, I’ll level with you. You can’t actually boot entirely from the hard drive, at least not yet. The Raspberry Pi 3 has a USB boot mode, but it’s still considered experimental at this time and you have to do some tweaks to even make it available. You can boot mostly from the hard drive today though, and it should be easy enough to adapt this post in the future when the Pi 3’s alternate boot modes are considered finished.

When the Raspberry Pi boots up, it looks to the first partition on the SD card for instructions on what to do next. That boot partition contains just a tiny piece of the boot process. You can easily move all the stuff that comes after that first step to a different device, such as the external hard drive. All that stays behind on the SD card is a very small boot partition, everything else moves to the hard drive.

The resulting system should run faster and smoother, and you won’t be using up your SD card’s limited number of write cycles on high-frequency stuff like a virtual memory swap file.

Acknowledgements

This article would not have been possible if not for others that went before me. In particular, Ted Hale’s article at http://raspberrypihobbyist.blogspot.com/2013/07/running-from-external-hard-drive.html is pretty much the blueprint I started from. In turn, that article was based on a forum post by “Rattus” at http://www.raspberrypi.org/phpBB3/viewtopic.php?f=26&t=10914&p=129474&hilit=resizefs#p122476.

I’ve added my own bits, such as consulting the boot/cmdline.txt file rather than simply assuming the root file system’s location, using UUIDs to identify the partitions, and expanding the existing swap file rather than creating a separate swap partition.

I’ve turned these resources into a walkthrough that matches the rest of this series, but I wanted to at least point out where my source material originated.

Shut down services

We’d like as little to be going on with this partition as possible while we’re copying it. If you have another Linux system, or another Raspberry Pi handy and a USB adapter for your SD card, then your best shot is to let another system do this copy, but for this tutorial, I’m assuming you only have the one Pi available. To make sure things are as “quiet” as possible, we’re going to want to exit the desktop, shut down any of the background services we’ve installed, and do the copy from the command line.

First, let’s boot to the command line. If you’ve configured the Pi to boot to the desktop, then unfortunately there’s no option to simply exit the desktop, so we’ll need to tweak the configuration first. Use the desktop configuration tool or raspi-config to set the pi to boot to the command line.

Reboot the Pi, and connect via SSH using a tool like PuTTY, or connect the Pi to a monitor and work directly from the Pi. I’ll be using PuTTY for this demo.

Once you’re logged in, shut down any of the services you’ve installed so far. For me, that’s Samba, MiniDLNA, OpenVPN, Resilio Sync, and NUT. If you have any other services like CrashPlan or Transmission running, you’ll want to stop those too.

sudo service samba stop
sudo service minidlna stop
sudo service openvpn stop
sudo service btsync stop
sudo service nut-monitor stop
sudo service nut-client stop
sudo service nut-server stop
sudo service nut-driver stop

We want as little activity going on as possible while we clone the OS partition.

Copy the old root partition to the new drive

The file /boot/cmdline.txt specifies the location of the root filesystem. You’ll need to edit this file later in order to boot from the hard drive, but for now, you just need to know for sure where the root filesystem currently resides. Show the cmdline file’s contents like this:

cat /boot/cmdline.txt

The part immediately to the right of “root=” specifies the device and partition that holds the root filesystem, and that’s the partition whose contents you need to move to the hard drive. In my case, it says “mmcblk0p2”. If you started from a raw Raspbian image, then yours probably says the same thing. I can’t guarantee that the default Raspbian image might not change in the future though, so it’s best to check.

Copy the existing root partition’s contents to the new 16GB partition (sda1) on the hard drive, changing the highlighted part to match the current root filesystem path from above.

Important: Be very careful here. Make sure that sda1 is the partition you set aside for the OS. If you configured your server differently, then double-check before continuing.

sudo dd if=/dev/mmcblk0p2 of=/dev/sda1 bs=32M conv=noerror,sync

This can run for quite a while, depending on how large your SD card is. Unfortunately, the dd command doesn’t give you any kind of feedback. In this case, no news is good news. As long as the drive appears to be busy, then keep waiting.

Here’s what that all means:

• “dd” copies things
• “if” specifies the input file, in this case an entire partitions
• “of” specifies the output file
• “bs” specifies how many bytes to copy in one chunk, here it’s 32 MB
• “conv” specifies how to convert things as they are copied
  • noerror says to continue if anything goes wrong
  • sync does some padding during the copy

When the copy process eventually finishes, check the target partition on the hard drive for errors.

sudo e2fsck -f /dev/sda1

Press enter at the prompts to fix any errors that it finds. You can probably expect one about the free block count, and one about the free inodes count.

Since the image that you just copied over from the SD card needs to be smaller than the partition you copied it to, you’ll want to expand it to fill up the available space.

sudo resize2fs /dev/sda1

Uniquely identifying a drive

This next section is a partial repeat from the “Adding a Hard Drive” post, but I’ll include it again for completeness, and so that you don’t have to bounce back and forth between the posts.

Warning: Because the next section involves copying around large strings of unmemorizable data, I recommend performing the steps through a remote SSH window, or from a terminal on the X desktop. The ids you’ll need to copy must be copied verbatim, letter for letter, or you’ll find yourself unable to boot, and you’ll have to start over again from your last backup.

Currently, the cmdline.txt file specifies that the root file system is on the second partition of the internal SD card (/dev/mmcblk0p2). We’d like that to say “/dev/sda1″ instead, but there’s a problem. As I mentioned earlier, we don’t have any guarantee that this particular drive will be called “sda” in the future. What we need is a way to uniquely refer to this drive no matter what letter it get assigned on any given day.

This was why we built the drive with a GUID Partition Table. Each partition on a GPT device is assigned a universally-unique identifier (UUID).

Note: The difference between GUID and UUID is not important here, they are different terms for the same thing; a very long, randomly-assigned number.

Get the partition UUID for /dev/sda1

sudo blkid /dev/sda1

The piece of information we’re interested is the “PARTUUID” value. Take a picture, write it down very carefully, or select and copy the text if you are doing this through a remote terminal connection like I am.

Boot Configuration

The new root filesystem’s partition is now uniquely identifiable, but the Raspberry Pi doesn’t know to use it. Edit the /boot/cmdline file to change where the bootloader will look for the root filesystem.

sudo nano /boot/cmdline.txt

Find the part that says “root=/dev/mmcblk0p…” and change it to “root=PARTUUID=” and whatever your Partition unique GUID was from above. Also add the string “rootdelay=5” at the end. This will give the Raspberry Pi time to discover the USB drive before it tries booting from it.

The result should look like this (sorry for the small picture):

Exit Nano, saving your changes (ctrl-x,y,enter)

Reboot (Important, do not skip this step)

Before the next set of changes will “stick”, you’ll need to reboot so that the Raspberry Pi uses the hard drive for the initial load. If you don’t reboot now, nothing you’re about to do will count, and you’ll just have to do it all again. You’ve been warned.

sudo reboot

If everything goes well, you should find yourself back at a login, and you can continue. If something went wrong, go back to your most recent backup and try again.

Mounting filesystems

Partitions and filesystems are not the same thing. Linux now knows what partition to load the OS from, and that’s great, but as things stand right now, it’s still going to mount the root filesystem from the SD card.

To complete the transition to the hard drive, you’ll need to edit the filesystem table. This file controls what gets mounted where, and in what order, and it needs to know where the root filesystem is. Take a look at the current contents.

cat /etc/fstab

Note: You’ll probably have an entry here for your data partition that we added earlier in the series.

That third line is the root filesystem, which you can tell by the “/” in the second column. Unfortunately, it’s still loading from the 2nd partition on the SD card (mmcblk0p2). You could change this to say “/dev/sda1”, but that would only work as long as the drive continues to get the name “sda”.

Fortunately, you can use a very similar UUID-based trick here to uniquely identify the filesystem no matter what letter the device gets. Filesystems have UUIDs too, and you can see them all with this command:

sudo blkid

If you look carefully, you may notice a problem. both /dev/mmblk0p2 and /dev/sda1 have the same UUID. So much for being unique, right? This is because of the way we cloned the old root filesystem into a new location. It brought the whole filesystem over, including its UUID. Before you can use a unique Id to identify the drive, you’ll need to make sure it’s actually unique.

You need to give /dev/sda1 a new UUID. You can hand-assign your favorite UUID, or just let the computer pick a random one. Use the following command to assign a new UUID to the first partition on the hard drive:

sudo tune2fs /dev/sda1 -U random

Note: At the time of this edit, there seems to be an issue with the newer “Jessie” release of Raspbian, and it may prevent you from changing the UUID on the partition. If you see an error that says “The UUID may only be changed when the filesystem is unmounted.”, even though it’s NOT mounted, then you’ll need to perform one more tweak. The magic words are

sudo tune2fs -O ^uninit_bg /dev/sda1

I can’t take credit for this one at all. I just found it here. After reading the man page on tune2fs, I still don’t honestly even understand what this has to do with anything, but it does seem to do the trick, so if you get the above error, give this command a shot, then try the “-U random” command again and it should succeed.

Either way, display the device Ids again, just to be sure

sudo blkid

Copy down the UUID value (not the PARTUUID value) for /dev/sda1. You’ll need it in a minute. Open the filesystem table in an editor.

sudo nano /etc/fstab

Change “/dev/mmcblk0p2” on the third line to “/dev/disk/by-uuid/” and the new UUID you just assigned to /dev/sda1.

The end result should look like this:

Your ids will obviously differ, but the important thing is that you have pointed the root filesystem to the hard drive. Any other mounts you’ve defined should appear here as well. Don’t worry about the columns lining up, it doesn’t matter, but I’m presenting mine to the public, so I’ve gone ahead and made it pretty

Exit Nano, saving your changes (ctrl-x,y,enter)

That’s it. you’re ready to reboot again, and this time, everything should be faster.

sudo reboot

You’ll notice that the activity light on the Raspberry Pi will not blink much anymore. That’s because the SD card is no longer being accessed for anything other than the boot partition. The hard drive’s activity light will now blink where the Raspberry Pi’s activity light used to.

When the system has rebooted, double-check your filesystem table to make sure your changes are still there.

cat /etc/fstab

If you see /dev/mmcblk0p2, and no line for your data partition, it’s because you skipped that “Reboot” step above. See? I told you it was important. Redo everything in the “Mounting filesystems” section, and reboot again.

Swap configuration

One thing you may have noticed if you’ve looked at other walkthroughs for booting from the hard drive is that they usually create a “swap” partition on the hard drive to be used as virtual memory.

If you look back at the filesystem table from the beginning of this post, though, you’ll notice that Raspbian never had a swap partition in the first place. That’s because Raspbian is set up to use swap files instead of swap partitions. Raspbian’s swap file lives at /var/swap, and since we just moved the whole root filesystem onto the hard drive, the swap file came along for the ride.

At this point, you’re already running your swap file from the hard drive, and you didn’t even have to do anything. Check it out with the “swapon” command:

sudo swapon –s

This shows the swap summary, which will tell you what swaps are in use. It should have one entry in it (/var/swap). It’s pretty small, though; only 100MB:

The partition we created for the root filesystem is 16GB (or more if you so chose). There’s plenty of space left to expand the swap file to something roomier.

Edit the swap file configuration:

sudo nano /etc/dphys-swapfile

Find the line that says “CONF_SWAPSIZE=100”. This is the size of the swap file in megabytes. Change the value to 2048, which will create a 2GB swap file, which is a more appropriate for a server running on a Raspberry Pi 2 or 3.

Close Nano, saving your work (ctrl-x,y,enter).

We’ll need to reboot one more time to get this to stick, but there’s another change to make first. If you’d like to return to booting to the desktop, now is the ideal time to set that back up using raspi-config.

sudo raspi-config

Go to “Boot Options”, then “Desktop/CLI”, and set it back to “Desktop”. When you exit raspi-config, it will offer to reboot. Go ahead and let it.

Once the system has rebooted, check the swap file by typing “swapon –s” again. You should see a table similar to the first time you ran this command, only now the swap file is much larger:

Wrapping up

You now have a Raspberry Pi that boots (mostly) from an external USB drive. It also uses this drive for its virtual memory swap file. The whole system should run more smoothly now, and you won’t have to worry about using up your SD card, if that’s the sort of thing you worry about.

Although most of the important stuff is on the hard drive, you should probably make at least one more backup of the SD card for safety.

What’s Next?

In the next post, I’ll show you how to make backups now that the SD card isn’t being used for the root filesystem anymore.

Note: This post is part of a series. Each post builds on the previous ones. If you are just trying to add one thing to an existing system that was not built following this series, then I cannot promise that these instructions will work for you, although they probably will. If you’ve started from something other than a non-NOOBS Raspbian image, then you’ll probably need to adjust for that.

Please refer to the series Introduction for a list of all the different posts in the series.

Self-Promotion: I have recorded this series as a screencast for Pluralsight:
(http://www.pluralsight.com/courses/raspberry-pi-home-server)
If you have a Pluralsight subscription, please consider watching it. Reading the instructions is one thing, but watching it done demystifies the whole process.

Thank you!

---

My previous home “server” before embarking on this series was just an old laptop of mine. It wasn’t particularly strong, but it could run CrashPlan and serve files. One advantage it had was that, being a laptop, it had a built-in battery and knew how to shut itself down if the power went out. Of course, I had to go downstairs and start it back up once the power was restored, but at least nothing got corrupted.

If you have a desktop computer at home or at work, you may also have an Uninterruptable Power Supply (UPS) under your desk. This is basically a battery big enough to power your computer and monitor long enough for you to save what you are doing and shut down gracefully. Most modern UPS units also have a USB port on them and come with software so that when the power goes out, the UPS can tell the computer to automatically suspend, hibernate, or shut itself down depending on how you’ve configured it.

It shouldn’t surprise you that there are UPS options available for the Raspberry Pi as well. Take the CW2 “Pi UPS” (http://www.piups.net), for example. This product will work fine for most simple Raspberry Pi projects running off of an SD card, but my particular installation has a RAID enclosure hooked up to it. I don’t think AA batteries are going to cut it. What I need is something that will keep the hard drives spinning long enough for the Pi to shut down safely.

I’ve had my Raspberry Pi Home Server plugged into a UPS for well over a year now, and it survives power outages just fine, shutting itself down safely, and restarting automatically. My Raspberry Pi Home Server is attached to a CyberPower SX550G. I chose this model because:

1. It has a USB connection to the computer.
2. It was on sale for $40.
3. I’m cheap.

It’s a 550VA battery backup, which probably wouldn’t get you very far with a full-sized desktop computer, but it should be more than adequate for a Router, a Raspberry Pi, and a couple of drives. It could probably run those devices for a good couple of hours if needed, actually. I haven’t done the math, but if not for the external drives, it could probably run the Pi for days.

So how can we get the Pi to use it intelligently? It certainly didn’t come with a ready-made software package for the Pi, although Linux software is available for it.

Network UPS Tools (NUT)

The Network UPS Tools package, aka “NUT” (http://www.networkupstools.org), is a collection of programs meant to make UPS hardware from different manufacturers work in roughly the same way. It’s available for a wide variety of platforms, one of which happens to be Debian Linux, of which Raspbian is a variant.

NUT consists of three major components

1. A driver to communicate with your particular UPS using whatever protocol it supports, and translate that into a common API.
2. A daemon (service) that connects via the driver and acts as a communication hub.
3. A client program that can perform various tasks such as shutting down the computer when the power state reported by the daemon own sitechanges.

There are a few other components, such as command-line utilities that can tell you about the current state of the UPS, or allow you to make changes to the UPS configuration. For the most part though, we’re only concerned with the three components I just mentioned.

Note: NUT works with a wide range of UPSes, but without owning one from each different manufacturer and/or vintage, I can’t create instructions specific to each model. Questions about how to get NUT working with your particular UPS are best addressed on the project’s own site, or its GitHub site. For the most part, though, they should all work the same way.

If your UPS is older, and doesn’t have a USB port, but has a 9-pin serial port, you are totally on your own. I tried to make an older serial-ported UPS work through an RS232/USB adapter, but as it turns out, the UPS didn’t actually communicate over the serial port. It just shorted two pins together to mean that everything was okay. Maybe you could hack something together using the GPIO pins. If so, you’re a better man than me.

Installing Network UPS Tools (NUT)

Simplicity itself. Like most things we’ve installed in this series, NUT is available through apt-get

sudo apt-get install nut

Select your driver

Find the driver for your particular UPS by looking at the compatibility list on the NUT site (http://www.networkupstools.org/stable-hcl.html). If you can purchase a UPS that’s on the list, then that’s great. If you don’t find your UPS on the list, that doesn’t mean it won’t work, though. For instance, my UPS isn’t on the list, but a ton of other devices from the same company are, and they all seem to use the same driver (usbhid-ups). I gave it a shot, and it works just fine. YMMV

Configure the UPS

Once you’ve identified the driver for your UPS, you’ll need to edit the ups configuration file.

sudo nano /etc/nut/ups.conf

This file, like all of the NUT configuration files, is very well documented inline, and explains all of its different options. At the bottom of the file, you’ll add an entry for your UPS. You’ll need to give it a name, specify a driver, and give it a description. There is also an option to specify a port. For units that connect via a USB cable, this is meaningless, but it’s still required, so use the value “auto”.

[RPHS]
    driver = usbhid-ups
    port = auto
    desc = "CyberPower SX550G"

The name in square brackets is so you can tell multiple UPS devices apart in case you have one NUT server monitoring multiple devices. Unless you’re building something really esoteric, you probably only have one UPS like me, so for lack of anything better to call it, I’ve named mine “RPHS” after the computer it serves. You can put anything you want in the description, so I just put the make and model of the device for reference. The end result should look something like this:

That’s it for configuring the UPS itself. Close and save the file (ctrl-x, y, enter).

Configure the daemon

A daemon is the Unix/Linux term for any invisible background process. Windows folks call these “services”. For NUT, there is a daemon that is in charge of listening to the UPS via the driver, and telling the client applications what to do. Edit its configuration file like this:

sudo nano /etc/nut/nut.conf

For this simple application, where both the client and the server programs will be on the same computer (the Pi), go to the bottom of the file and set the MODE to “standalone”.

Close and save the file.

Verify hardware configuration

To check whether the driver and daemon are configured correctly, you can simply start up the service.

sudo upsdrvctl start

The first time I tried to connect to the UPS I got the error “could not detach kernel driver from interface 0: Operation not permitted”. I rebooted and tried again, and everything was fine.

Notice that this time I got a message about “Duplicate driver instance detected”. This is because now that I have things configured correctly, the driver started up automatically when I rebooted. Not only that, but the daemon should be running now as well. Check it like this:

sudo service nut-server status

You should get a message that the NUT server is running. You can now ask the NUT server questions about the status of the UPS using “upsc”, one of several command line utilities that apt-get installed. Substitute the name you gave your UPS above as needed.

upsc rphs

You’ll get quite a long list of information about the configuration and status of your UPS.

Depending on the make and model, you’ll get more or less detailed information.

Configure the monitor

That’s two out of the three layers. Last but certainly not least is the “upsmon” client. This is the part that will actually shut down the computer when the NUT server says so.

Define credentials that the monitor client program will use to connect to the server.

sudo nano /etc/nut/upsd.users

Go to the bottom and define two users, one called “admin” and one called “upsmon”. You can actually name them anything you want, but naming the “monitor” user after the program that will use it (upsmon), seems to be the convention. You give the “admin” user rights to issue commands and change configurations with the “actions” and “instcmds” settings. The “upsmon” user has no such rights, it’s just there to listen and shut the computer down when the power goes out.

[admin]
    password = mypasswd
    actions = SET
    instcmds = ALL

[upsmon]
    password = mypasswd
    upsmon master

Since this is the computer that’s actually in charge of monitoring the UPS, set the “upsmon” setting to “master”.

Next, edit the configuration file for the upsmon client program.

sudo nano /etc/nut/upsmon.conf

This configuration file is pretty long, and has a lot of options. The one we’re interested in is about five pages down. Look for the example lines that start with “MONITOR”, and create a new entry on the blank line below that section. There are six parts to this setting.

1. The keyword “MONITOR”. It does have to be all uppercase, by the way.
2. The “system” name in the format UpsName@HostName. I called my UPS “RPHS”, and since we’re running in standalone mode, we can just use “localhost” for the host name, so the resulting “system name” is “RPHS@localhost”.
3. The “power value”. This only applies to big servers with multiple redundant power supplies. Just set it to “1”.
4. The user name that you established in the upsd.users file (upsmon) .
5. The password that you established in the upsd.users file (mypasswd).
6. A value indicating whether this computer is the master or slave. You can read about the distinction in the upsmon.conf file itself, but for a standalone system like this, use “master”.

When you’re done, it should look something like this.

This next part is optional, but handy for testing that your system is working. Scroll down a bit further, and look for the NOTIFYFLAG section. There are two entries there, “ONLINE” and “ONBATT”, but they seem to be commented out by default in the current installation. Remove the pound sign from the beginning of these two lines, so that NUT can tell us about the state of the UPS when it changes.

Close and save the file.

Next up, a little permissions wrangling. You need to set up the various configuration files to be readable by the NUT components that use them, but not by other users. This prevents anyone from reading the password, and sending unauthorized commands to the server to shut everything down. It may be overkill for a simple home network, but it’s also really simple to do.

sudo chown nut:nut /etc/nut/*
sudo chmod 640 /etc/nut/upsd.users /etc/nut/upsmon.conf

You should get no complaints

UPS Commands

Depending on the sophistication of your particular UPS, you may be able to send it commands to do things like initiate a self-test, or simulate a power failure. You can get a list of what your particular UPS supports with

sudo upscmd -l rphs

The number and type of commands will vary by manufacturer and model, so your output may not match mine. There are a few commands here worth mentioning, though. The “load.off” command will shut down the UPS immediately. Think of it as the “goodbye world” command. You generally don’t want to mess with that one. You could actually use this command to turn a second UPS on and off, allowing your Pi to control the power to something else. I’ll leave that one up to your imagination, but there are certainly cheaper ways to automate things.

The “beeper.mute” command will temporarily silence the warning beep that my UPS makes when the power goes out. This will make testing the system a bit less annoying here in the next section. The “beeper.disable” command would probably have the same effect, but on a more permanent basis.

Testing the system

Restart the NUT service and client daemons to ensure that they load all of the configuration changes we’ve made.

sudo service nut-server restart
sudo service nut-client restart

At this point, you can unplug your UPS from the wall and, after a short delay, you should get a message that the system is now running on battery power.

IMPORTANT: Wall messages like this don’t seem to show up in Terminal windows on the desktop. If you want to test this part for yourself, connect via SSH using a program like PuTTY on Windows.

If you plug it back in, you’ll get another message that power has been restored.

So far, so good. Now for the real test. Unplug the UPS from the wall and leave it unplugged. If your UPS supports it, now is a good time to issue that “beeper.mute” command. In my case, I issued the “upscmd rphs beeper.mute” command, and was prompted for a user name and password. We established the user name “admin” in the config file above.

You can sit and stare at the screen, wondering when it will shut down, or you can periodically check on the status to see how fast you’re using up the battery. I plugged my laptop charger and a television into the UPS to speed things along. A good, old-fashioned 100-Watt bulb would help, too.

Eventually, when the battery gets low enough, the system should shut itself down.

Plug the UPS back in, and if you’re lucky, everything will start back up again. For some models, even after the power is restored, you may have to physically press a button on the UPS to start everything back up. Mine starts up on its own, so a few minutes later I was back up and running.

Email notifications (optional)

Wall messages are just fine if you’re connected via SSH or looking at a monitor connected directly to the Pi, but if your server is headless or you’re running the desktop, you won’t see them. We need a way to be notified when something goes wrong. NUT can be configured to call any arbitrary command when it has something to say, so we’re going to set it up to send out email messages. This can inform you of power issues even when you’re not connected to the Pi at all.

We’re going to need to install a couple utilities to make this happen. I’ll be using ssmtp because it’s the simplest thing that works. Because it only knows how to send emails, it doesn’t leave any services running in the background using up CPU time. If you’ve already configured some other kind of mail agent on your system, then you’ll need to adjust accordingly.

sudo apt-get install ssmtp mailutils

Next, we’ll edit ssmtp’s configuration:

sudo nano /etc/ssmtp/ssmtp.conf

For this demonstration, I’ll be configuring ssmtp to work with gmail. If you want to use your ISP’s email services, then check out the ssmtp documentation, but most of the concepts should be the same.

Change the “mailhub” setting to point to your email server. For gmail, that will be “smtp.gmail.com:587”. For gmail, you’ll also need to add the “UseSTARTTLS=YES” setting. Scroll to the bottom of the file and add settings for “AuthUser”, “AuthPass”, and uncomment the “FromLineOverride”. The result should look something like this:

Note: If you have two-factor authentication set up with gmail, then the AuthPass setting will need to be an application-specific password. You can read more about that here.

Test your email setup by sending yourself an email from the command line.

echo "Test body" | mail -s "Test subject" [YOUREMAILADDRESS]

Check your inbox for the email. If you get it, then you’re ready to configure NUT to use ssmtp. Some wireless providers have special email addresses you can use to send texts to your phone. I have configured my NUT installation to send me texts rather than plain old emails, so I’ll notice them immediately.

Next, edit the NUT client configuration one more time to set up the email notifications.

sudo nano /etc/nut/upsmon.conf

Scroll down to find the NOTIFYCMD section, and add a line that says

NOTIFYCMD /etc/nut/upssched-cmd.sh

You presumably don’t want to be notified about every single power event that happens, so scroll down to the NOTIFYFLAGS section again, and add “+EXEC” to the end of the messages you want to receive emails for. I’m going to enable emails for the ONLINE and ONBATT events. The end result looks like this:

All that’s left is to create the script that we named in the NOTIFYCMD setting, “upssched-cmd.sh”.

sudo nano /etc/nut/upssched-cmd.sh

Paste in the following, substituting the email address where you want the notifications to go.

#!/bin/sh
echo "$*" | mail -s "Message from RPHS" [YOUREMAILADDRESS]

Exit Nano, saving the file (Ctrl-X,Y), and make it executable.

sudo chown nut:nut /etc/nut/upssched-cmd.sh
sudo chmod 700 /etc/nut/upssched-cmd.sh

Test the new script by calling it in the same way that NUT will in the event of a power event.

sudo /etc/nut/upssched-cmd.sh "This is a test message"

If you got that message, then everything is configured. Restart the NUT client so that it will pick up on the configuration changes.

sudo service nut-client restart

And now you can test the system again by unplugging the UPS from the wall. If everything is configured correctly, you should get an email (or text message) telling you that the power has gone out. Plug the UPS back in, and you should get another message telling you about it. Now you’ll know about power problems even when you’re not at home.

Adding more Pis to the same UPS (optional)

With Raspberry Pis being so cheap, you may very well have more than one. Perhaps you’ve split CrashPlan or MiniDLNA off onto a Pi all by itself. Maybe you went a little crazy and built a Pi-powered super-computer cluster. If, for any of these reasons, you have multiple Pis, and you’d like them to share the same UPS, how will they all know when the power’s gone out? After all, you can only plug the USB cable into one of them, and buying a separate UPS for each Pi would be overkill, right?

The great thing about NUT being split up into layers is that you can run multiple client instances against one server. Normally, this is the sort of thing you’d only see in a large server rack, where you’ve assigned one computer to watch the battery and tell all the others when the power goes out. It’s not the sort of thing regular home users usually care about, but Raspberry Pi owners are not regular home users.

If you do happen to have multiple Pis, getting them all to share a single UPS and shut down gracefully is actually really simple.

Reconfigure NUT on the server

Earlier, we configured NUT in “standalone” mode, which means that the server and client were on the same machine. It also means that the server isn’t expecting any other clients to be talking to it, so it simply isn’t listening for them. With a little bit of reconfiguration, we can tell the NUT server to listen for clients other than itself.

Edit the NUT configuration file.

sudo nano /etc/nut/nut.conf

Scroll down to the bottom, and change the mode from “standalone” to “netserver” (not “nutsever”, mind you).

Close and save the file (ctrl-x, y, enter)

Next, you’ll need to configure credentials for the remote machines to use when connecting to the server.

sudo nano /etc/nut/upsd.users

Add a new user at the bottom. You can call it anything you like, but I’ve chosen “upsmon-remote”. Give it a password, and specify that it is a upsmon slave.

Next, you need to tell the NUT server to listen for client connections. Edit the UPS daemon configuration file.

sudo nano /etc/nut/upsd.conf

Find the “listen” section, and add a listen directive using the IP address of your server. If you don’t configure anything, the NUT server will listen only to the computer that it’s on, which is localhost, or IP address 127.0.0.1. Since you now want NUT to listen for clients on other systems as well, you’ll need to specify a real IP address.

Adding the server’s own IP address might seem a little redundant, but remember that NUT is made to work on all kinds of equipment, and it’s very common for larger-scale servers to have multiple network interface cards, and you’d only want it to listen for NUT traffic on the “inward” facing one.

Because you will still want to run a ups monitoring client locally on the server itself, and you’ve now overridden the defaults by specifying one IP address, you’ll need to explicitly add an entry for localhost (127.0.0.1) back in as well, otherwise the local ups monitor will stop working. The result should look something like this:

Note: You’ll need to substitute your own Pi’s address in place of 192.168.1.11. That’s just the address where my own Pi server lives. The default port that NUT listens to is 3493. If you wanted to change that, for some reason, you would do that here. I’m going to leave it at the default port, though.

Reboot the server to get things reconfigured and restarted.

sudo reboot

Double check that you can still communicate with the NUT locally:

sudo upsc rphs

You should get the same kind of listing of UPS settings and status as you did in standalone mode:

If you get some kind of error instead, make sure that you added an entry for localhost (127.0.0.1) in addition to the Pi’s regular IP address.

You can actually run NUT configured this way, but without any clients connected, and it will behave in pretty much the same way as it did in standalone mode, but it will still be listening for additional clients over the network.

Configure the client machines

Install NUT on each additional Pi that you intend to run from the UPS.

sudo apt-get install nut

For these additional client-only installations, you can skip configuring a UPS driver and proceed directly to configuring the client.

sudo nano /etc/nut/nut.conf

Go to the bottom of the file, and set MODE to “netclient”.

Next, edit the upsmon configuration file.

sudo nano /etc/nut/upsmon.conf

This time, configure the client to talk to the remote server for information, and configure it as a slave. This time, the “system name” will not be “RPHS@localhost”, but “RPHS@192.168.1.11” (again, substitute your server’s IP address).

If you remember from earlier, the first part of this “system name” is the name you gave your UPS, and the second part is the address of the server it’s attached to. The name and password should match the “upsmon-remote” name and password from above, and the final parameter will be “slave” instead of “master”. The result should look like this:

Note: My other Pis are all busy right now, so I’m doing the edits (but not saving them) on my original server for these screenshots. Please ignore the name in the title bar.

The difference between “master” and “slave” has to do with when each one will shut down. When the UPS says the power has dipped below the safe threshold (20% for my UPS), the server (the master), will tell all of the clients (the slaves) to shut down first before shutting down itself.

Like on the server, you’ll need to set up the permissions so that the NUT configuration files are readable by the nut user. Unlike the server, however, you don’t need to touch the upsd.users file because users are defined on the server, not the client.

sudo chown nut:nut /etc/nut/*
sudo chmod 640 /etc/nut/upsmon.conf

Check connectivity

From the slave system, you can issue the same kinds of commands you could run from the master system. Use this to verify that everything is communicating properly.

sudo upsc RPHS@192.168.1.11

I’ve used the IP address because name resolution didn’t work for me. It’s one of those things I’ll probably learn to set up someday, just not today.

Test the system

Once again, unplug the UPS, and you should see warning messages appear on both the server and client(s) right away. Again, you may want to mute the beeper if your hardware supports it, and plug something in to help the battery drain faster.

Apart from the name of the server sending the message, and the fact that the client shut down first, you wouldn’t be able to tell the difference between the client and server installations, even though only the server is actually attached to the UPS.

Lather, Rinse, Repeat

Repeat as necessary for each additional Raspberry Pi you are plugging in to the same UPS. That’s all there is to it. When the power goes out, all your Pis can now shut themselves down safely.

Wrapping up

You know the drill by now. Shut down the server, and back up the SD card.

What’s Next?

In the next post, we’ll reconfigure the Pi to boot from the hard drive instead of the SD card.

Note: This post is part of a series. Each post builds on the previous ones. If you are just trying to add one thing to an existing system that was not built following this series, then I cannot promise that these instructions will work for you, although they probably will. If you’ve started from something other than a non-NOOBS Raspbian image, then you’ll probably need to adjust for that.

Please refer to the series Introduction for a list of all the different posts in the series.

Self-Promotion: I have recorded this series as a screencast for Pluralsight:
(http://www.pluralsight.com/courses/raspberry-pi-home-server)
If you have a Pluralsight subscription, please consider watching it. Reading the instructions is one thing, but watching it done demystifies the whole process.

Thank you!

---

Update – Replacing Expired Signing Keys

Apparently, the Resilio signing keys have a habit of expiring every December. You’ll first notice it when you go to do an apt-get update, and receive an error that says something like “EXPKEYSIG 05CD43032484414B Resilio, Inc.”.

Take note of the signature. That’s the seemingly random string of letters and numbers in the error message. You’ll want to remove the old signing key because apt-get is going to go on complaining about it forever if you don’t. You’ll still be able to update Resilio Sync with a new key, which we’ll install in a moment, but unless you clear out the old one, you can expect to keep seeing the error.

Fortunately this is easy to do. Copy down the key signature from the error message, and substitute it in the following comment. In my case, it was “05CD43032484414B”, so the command to clear it out is:

sudo apt-key del 05CD43032484414B

Next, you’ll need to install an up-to-date signing key before apt will be happy again. You can install the new key in exactly the same way you installed it originally.

wget -qO - https://linux-packages.resilio.com/resilio-sync/key.asc | sudo apt-key add -

That’s it. You can keep Resilio Sync up to date again… until the next time their key expires… sometime next December.

---

Introduction

There are plenty of cloud sync solutions out there. You have Microsoft’s OneDrive, Google Drive, DropBox, and SpiderOak just to name a few. One thing they all have in common is that a copy of your stuff is on someone else’s system. Perhaps you’re uncomfortable with that, even if they’ve promised that they won’t peek at it. Or maybe you’re just looking for yet another thing for your Raspberry Pi to do.

The same people that brought you BitTorrent, the peer-to-peer file sharing application, have created Resilio Sync (formerly BitTorrent Sync). It’s a file synchronization application based on the BitTorrent protocol. Think of it as a peer-to-peer network where all of the peers belong to you (Your desktop, laptop, tablet, phone, etc). If you put something in a sync folder on one computer, it shows up in the sync folder on all of the other computers.

The trouble is that in order for a file to travel from one computer to another, they both have to be up and running at the same time so that they can talk to each other. You may want to share files between work and home, but by the time you get to work, your home computer will have fallen asleep and vice versa. What you need is a node in your network that’s always on. The Raspberry Pi can be that node.

This post is going to be pretty short because, to be honest, the installation is dead simple.

Note: You may want to check out the Syncthing post in this series as well. Look at both options and decide which you like better. It seems that running both Resilio Sync and Syncthing at the same time works just fine, but you probably won’t need both.

Install Resilio Sync

There is an apt-get package available for Resilio sync, but it’s not listed on the well-known public repositories yet, so we’ll need to edit apt-get’s list of sources to add the new repository.

As with some of the other posts, we’re going to create a separate file to contain the sources needed for Resilio Sync. This is the preferred way to keep everything organized so you don’t lose track of which source was added to support which package. This new file must be in the /etc/apt/sources.list.d folder, and must have the extension .list. Name the file something that lets others know what package it was created to support.

Although the product’s official name has changed from “BitTorrent Sync” to “Resilio Sync”, the package name is still “btsync”. Since the previous version of this post was published, an official package has been released. Create a new APT source file using nano.

sudo nano /etc/apt/sources.list.d/btsync.list

Add the following line to the file. This will let APT know where to get the files it needs to install. If you followed the previous version of this article, and already have a btsync.list file, make sure you clear out any existing entries so that this is the only line in the file.

deb http://linux-packages.getsync.com/btsync/deb btsync non-free

Close and save the file (ctrl-x, y, enter).

You’ll need to import the signing key used by the new repository before you can use apt-get to install things from it. This time, the key is available from a public key server, so the commands are going to be quite different.

Import the repository’s signing key with the following commands:

sudo apt-get install apt-transport-https
wget -qO - https://linux-packages.resilio.com/resilio-sync/key.asc | sudo apt-key add -

Note: You may already have https support for apt-get installed as part of installing other components, in which case the first command won’t find anything to do, but I’m including it here just to be sure.

Now you can update apt-get’s list of available packages to take the new repository into account and install BitTorrent Sync the same as any other software package in this series.

sudo apt-get update
sudo apt-get install btsync

That’s it, BitTorrent Sync is installed, and should already be started automatically. You can check on the status of the service like any other daemon.

sudo service btsync status

All configuration is done using the web interface, but you can’t see it yet. At the time I’m writing this, btsync will only show its web interface to the local computer, meaning we’ll need to either hook the Pi up to a monitor, or connect via the remote desktop using VNC.

This is good for security, and you might consider leaving it this way if you don’t mind opening up a VNC window whenever you want to make changes. If you’d prefer the convenience of connecting from any computer in the house, then you can make a small configuration change to allow for that. Open btsync’s configuration file using nano.

sudo nano /etc/btsync/config.json

In the “webui” section, change the “listen” value from “127.0.0.1:8888” to “0.0.0.0:8888”, and then restart the btsync service.

sudo service btsync restart

Configuring BitTorrent Sync

The rest of the configuration will be done from the browser. If you left the web interface locked down, then you’ll need to open the browser on the Pi’s desktop, and go to “localhost:8888”. If you opened it up, then you can use a browser on your regular computer and go to “http://rphs:8888”, substituting the name of your server.

You’ll get a prompt to name this instance.

Once again, I’ll call mine RPHS. Dismiss the dialog, and you’ll be taken to the main Sync interface. You can start adding folders to sync, but first we should address some security concerns. If you left btsync configured to only work directly from the Pi, then this isn’t as much of an issue, but if you set it to be accessible from any computer on the network, then you’ll want to give the admin user a password.

Click on the gear icon in the upper-right, and then “Preferences”

Next, go to the Login tab, and fill your new password in twice.

Since you just changed the admin password, your existing browser session is no longer trusted, so you should be prompted immediately to re-authenticate.

Type the user name “admin” and your new password to continue”. Next, go back to the General tab, and lets make sure that btsync is storing our stuff on the hard drive, not on the tiny SD card. Click on the Change button next to the “Default folder location” button, and choose a new location for your stuff. This could be inside the public share if you want. I’m going to put mine in a new top-level folder outside the public share. You can create this folder yourself, or let btsync do it for you from the dialog. Close the configuration dialog when you’re done.

 

 

 

At the “Link Devices” page, you have a decision to make. If you’re already using Sync 2.0 on other computers, then click “Link a device already running Sync 2.0”, and follow the prompts to link up with your existing devices. I’ll be setting this device up as my first, so I’ll click “This is my first Sync 2.0 device”.

Choose a name for this btsync instance. I usually name mine after the computer I’m installing it on, so I’ll call mine “RPHS”.

Creating a Sync folder

Click on the “Add Folder” link in the upper-left to create your first Sync folder. The browser window will open. Navigate your way to the folder you wish to synchronize, or click the “New Folder” link in the lower left of the dialog to create a new folder. I’ve created a “public” folder inside the btsync folder.

In the next dialog, you get to set permissions for this share. Since this is my public share (at least to those I share the key with), I will choose “Read & Write”, and leave the other options alone.

Now, click on the “Copy” button in the lower-right, and copy out the text from the dialog. You’ll need this to set up the other computers that will sync to this folder. Close the copy dialog, and then the share dialog to return to the main btsync interface.

Set up other computers

Now that we’ve created a sync folder on the Pi, we just need to install BitTorrent Sync on another computer, and start syncing. BitTorrent Sync clients exist for Windows, Mac, Linux (obviously), and all major mobile platforms. Instructions for installing and configuring the clients will vary by platform, so I won’t cover them all here, but they should all be straightforward.

For this demo, I’ve installed BitTorrent Sync on my main Windows 10 computer. From the main menu, click “Enter a key or link”.

You’ll be prompted for the key that identifies the folder on another computer (the Pi) that you want to synchronize with.

Paste in the text you copied from the dialog on the btsync admin interface, and click “Next”. You’ll get a summary of the folder you are about to connect to, what your rights will be, and you’ll get to choose where those files should be stored on this computer. You’ll also get a reminder that some features, like “Selective Sync” are only available if you upgrade to the Pro version, which is actually pretty cheap.

Click the “Connect” button, and you’re almost there. Back on the browser, you should now see a notice that there is an approval pending.

Click the check mark to approve the connect request, and you’re all set. Files dropped into the folder on either computer will now synchronize to the other.

Wrapping up

As usual, shut down and take a backup of the SD card once you’re happy with the way things are working.

Unlike other “cloud sync” technologies, you own the cloud in this case, and you can do pretty much whatever you want with it. There are no size limits other than your available storage, and you can create as many individual shares as you want. Because of the granular way in which BitTorrent Sync shares your stuff between computers, you can choose to share individual folders with friends and family, and control whether they get full control, or read-only access.

BitTorrent Sync gives you total control of your own private cloud.

Troubleshooting

Files aren’t syncing

If files from one system aren’t showing up on the other, or files only seem to be syncing in one direction, make sure that you’ve shared the right kind of link. Sending a read-only link to your other computer will result in a one-way sync. You may add or edit files on the other computer, but their changes will never make it back to the Pi. When you copy the link, make sure you selected “Read/Write”.

What’s next?

The core functionality of the server is just about done now. In the next post, I’ll show you how to set up an uninterruptible power supply (UPS) using “Network UPS Tools”, or NUT.

Note: This post is part of a series. Each post builds on the previous ones. If you are just trying to add one thing to an existing system that was not built following this series, then I cannot promise that these instructions will work for you, although they probably will. If you’ve started from something other than a non-NOOBS Raspbian image, then you’ll probably need to adjust for that.

Please refer to the series Introduction for a list of all the different posts in the series.

Self-Promotion: I have recorded this series as a screencast for Pluralsight:
(http://www.pluralsight.com/courses/raspberry-pi-home-server)
If you have a Pluralsight subscription, please consider watching it. Reading the instructions is one thing, but watching it done demystifies the whole process.

Thank you!

---

Update: If you want to know what’s going on under the covers, or want tighter control over your exact configuration, then by all means, read this post in its entirety. I encourage you to do so just to gain an understanding of what’s going on. If, however, you’re just looking for the fastest way to set up a home VPN on a Raspberry Pi, then you should check out PiVPN. It’s basically everything in this article, completely automated. That means that everything you’re about to read can be condensed down to a single command.

curl -L https://install.pivpn.io | bash

After recently having my cable modem upgraded (and finding that it out-performed my own wireless router), I was having some trouble getting my VPN working again. Rather than go through all the steps of my own post, I thought I’d give PiVPN a shot at it. Ten minutes later, most of which was eaten up by generating Diffie-Hellman keys, I had new ovpn files for my various devices, and everything just worked. PiVPN is awesome. It makes me wish there were similar scripts for the other aspects of running a Raspberry Pi Home Server.

---

Introduction

Now that the whole house is humming along, sharing files, downloading things, and backing everyone up, you might be wondering if there’s anything left that the Raspberry can do for you. The answer is yes. In this article, we’ll set up the Raspberry Pi to act as an OpenVPN server, allowing you to securely access your home network from anywhere. OpenVPN is an open-source, cross-platform, virtual private networking (VPN) application. VPNs let you route internet traffic through a secure, encrypted channel, back to a network that you trust and/or control. You may have used one in order to securely access resources on the network at your office when you’re on the road. Developers sometimes use them to simulate traffic coming into their network from outside for testing. You can add these same abilities to your home network so that you can get to your stuff from work, or a hotel, or anywhere else with internet access. Running your own VPN means that no matter where you get an internet connection from, you are effectively “at home”. You don’t need to worry about fellow patrons at the coffee shop listening in on your network traffic because the traffic between you and your VPN is highly encrypted.

Acknowledgements

Once again, I didn’t invent this stuff. Most of the information about how to set up OpenVPN comes from a whitepaper by Eric Jodoin of the SANS institute. That whitepaper was later paraphrased and simplified in a pair of posts by Lauren Orsini. Both are excellent reading, and go into far more depth about how all this stuff works than I plan to. I’m just putting it into the same format as the other posts in the series, and organizing them in a logical progression, building on top of the previous posts in this series.

Prerequisites

In order to connect to your home network’s VPN when you are away from home, you are going to need either a static IP address, or a dynamic IP resolution service like www.no-ip.org. My home router updates no-ip automatically, so I have not set up a program on the Raspberry Pi to do this. Other tutorials exist out there to handle this part.

Internet security

Warning: The explanation that follows is super-non-technical™, and probably wildly inaccurate in many important ways. I am not a security or cryptography expert, but this is, in layman’s terms, how internet security works. When you visit your bank’s website, and something in your address bar turns green, or grows a little lock, it means that someone at the bank went to some authority that we’ve all agreed to trust, and got a certificate that says “Yup these guys are the bank alright”, and installed it on the web server you’re talking to. As long as you trust the people that made the certificate to only give it to the company that paid for it, and as long as you trust the bank to only install the certificate on their own servers, then you have a way to prove that the server you’re talking to belongs to the company you think it does, or at least a company that the authority vouched for.

Although I’m sure you trust your own word that the Raspberry Pi Home Server that you’ve been building is your own, your other computers are still going to want proof that the thing on the other side of the internet is, in fact, your server, and not someone else pretending to be your server. That’s kind of the whole point of this exercise, after all. Since you trust yourself, you can act as your own “certificate authority” and make your own certificates. You then install your homemade certificates on both the server and the client machines, and they can use that certificate to encrypt traffic back and forth between them.

Install OpenVPN

First things first, you’ll need to install the OpenVPN software onto the Raspberry Pi. You’ll also need the OpenSSL and EasyRSA packages in order to secure your connection later on. Installing them is as simple as…

sudo apt-get install openvpn openssl easy-rsa

That’s the easy part. Now comes the configuration. The EasyRSA installer has created some sample configuration files for us, and they’ll form the skeleton of the real configuration. Copy the entire directory of sample configuration files like this:

sudo cp -r /usr/share/easy-rsa /etc/openvpn/easy-rsa

Open the “vars” file for editing.

sudo nano /etc/openvpn/easy-rsa/vars

Find the “export EASY_RSA” entry, set it as follows:

export EASY_RSA="/etc/openvpn/easy-rsa"

A little further down, you’ll see a line that begins with “export KEY_SIZE”. In current installations, this should already be set to 2048, so you shouldn’t need to change it. Previous installations used 1024 bit keys, but the current recommendataions are to use 2048 for increased security.

Move to the bottom of the file, and change the defaults that are defined there to match your location and network. This will save you some time later on when you are asked to provide this information again for each user you set up. Mine looks like this:

export KEY_COUNTRY="US"
export KEY_PROVINCE="OH"
export KEY_CITY="Columbus"
export KEY_ORG="Home"
export KEY_EMAIL="melgrubb@…"
export KEY_OU="RPHS"

KEY_NAME affects the name of the resulting key file, but is otherwise arbitrary. The organization unit (OU) setting is not important for a small home network, so I’ve just gone ahead and used the server name.

export KEY_NAME="RPHS"

You can leave the other settings at their defaults. Close Nano, saving the file (ctrl-x, y, enter)

Become a certificate authority

In order to create certificates, you’ll need… wait for it… a certificate. In this case, it’s a “root certificate”. This is the kind of thing that one of the trusted authorities out on the web would have. The “easy-rsa” package you installed earlier can generate such a certificate for you. Run the following commands to set up a key server. Notice that the “sudo su” command is being used here. You’re going to stay in “god mode” for pretty much the remainder of this post.

sudo su
cd /etc/openvpn/easy-rsa
source ./vars
./clean-all
./build-ca

This last command will prompt you for a lot of values, fortunately, you set up reasonable default values above, so you can just hit enter to accept them. When prompted for “Common Name”, I’ve used the server’s name again (RPHS). When you create certificates for other computers or users, you’ll want to use their names here.

When that finished, enter the following command, substituting the name of your server, and accepting the defaults again. You’ll get a couple extra questions this time. Make sure the challenge password and company name are left blank, and accept any other defaults.

./build-key-server RPHS

Answer yes to the “Sign the certificate?” and “commit?” prompts.

Next, we’ll build the Diffie-Hellman parameters file.

./build-dh

Go get some coffee or something, this step is likely to take a very long time. Sometimes it does, sometimes it doesn’t. It’s pretty random what kind of wait you’ll have, but since the change from 1024 bit keys to 2048 bit keys, the likelihood of this taking a long time has definitely increased.

The screen will fill up with a lot of dots and plus signs while it works to let you know that it hasn’t gone to sleep or locked up. As long as the symbols keep coming, let it keep doing its thing.

When it’s done, generate a hash-based message authentication code (HMAC). This is yet another layer of protection, and helps to prevent denial of service (DOS) attacks without taxing the Pi’s processor too much with decryption of bogus data.

openvpn --genkey --secret keys/ta.key

Configure the OpenVPN Server

Now it’s finally time to edit the OpenVPN configuration and tie up the loose ends.

nano /etc/openvpn/server.conf

You’ll notice that the editor is totally blank. That’s because this file doesn’t exist yet. Paste in the following text, substituting your own values for the highlighted values. You’ll need your Raspberry Pi’s IP address, the IP address of your router, and the name you used above when calling build-key-server.

Note: I’ve also highlighted the protocol (udp) and port (1194). You can change these to something else if needed. I’ll talk about this more in the troubleshooting section at the end.

local 192.168.1.XXX # YOUR PI'S IP ADDRESS
dev tun
proto udp
port 1194
ca /etc/openvpn/easy-rsa/keys/ca.crt
cert /etc/openvpn/easy-rsa/keys/RPHS.crt
key /etc/openvpn/easy-rsa/keys/RPHS.key
dh /etc/openvpn/easy-rsa/keys/dh2048.pem
server 10.8.0.0 255.255.255.0
# server and remote endpoints 
ifconfig 10.8.0.1 10.8.0.2 
# Add route to Client routing table for the OpenVPN Server
push "route 10.8.0.1 255.255.255.255" 
# Add route to Client routing table for the OpenVPN Subnet
push "route 10.8.0.0 255.255.255.0" 
# your local subnet 
push "route 192.168.1.0 255.255.255.0" # YOUR PI'S IP SUBNET
# Set primary domain name server address to the SOHO Router
# If your router does not do DNS, you can use Google DNS 8.8.8.8
push "dhcp-option DNS 192.168.1.1" # YOUR ROUTER'S IP ADDRESS
# Override the Client default gateway by using 0.0.0.0/1 and
# 128.0.0.0/1 rather than 0.0.0.0/0. This has the benefit of 
# overriding but not wiping out the original default gateway.
push "redirect-gateway def1" 
client-to-client
duplicate-cn
keepalive 10 120
tls-auth /etc/openvpn/easy-rsa/keys/ta.key 0
cipher AES-128-CBC
comp-lzo
user nobody
group nogroup
persist-key
persist-tun
status /var/log/openvpn-status.log 20
log /var/log/openvpn.log
verb 1

Exit nano, saving your changes (ctrl-x, y, enter) Next, you need to allow the Raspberry Pi to forward IP traffic, which it does not do by default.

nano /etc/sysctl.conf

Find the line that says “Uncomment the next line to enable packet forwarding for IPv4”, and uncomment the line immediately after it.

# Uncomment the next line to enable packet forwarding for IPv4
net.ipv4.ip_forward=1

Exit Nano, saving your changes (ctrl-x,y,enter), and force a reload of the settings.

sysctl -p

Configure Firewall

The Raspberry Pi has its own firewall, which must be configured to allow the VPN traffic through. Create a script file to automate the opening of the appropriate ports.

nano /etc/firewall-openvpn-rules.sh

Copy in the following text, substituting your own Raspberry PI’s IP address where highlighted.

#!/bin/sh

iptables -t nat -A POSTROUTING -s 10.8.0.0/24 -o eth0 -j SNAT --to-source 192.168.1.XXX

Change the permissions on the file you just created so that it can be executed, and assign ownership to the root user.

chmod 700 /etc/firewall-openvpn-rules.sh
chown root /etc/firewall-openvpn-rules.sh

This script file needs to run every time the Raspberry Pi boots up in order to do us any good. Edit the /etc/network/interfaces file.

nano /etc/network/interfaces

Find the line that configures the wired ethernet port. If you are running your server wirelessly, then you’ll need to adjust accordingly. Insert a new line, indented underneath so that the result looks like this:

...
iface eth0 inet dhcp
    pre-up /etc/firewall-openvpn-rules.sh
...

This will ensure that the firewall rules are applied to that network interface even before it has started up.

One last thing, and this is new with the Jessie release. As commenter Craig pointed out in the previous version of this post, we need to stop OpenVPN from starting TOO soon. Edit the OpenVPN service file.

sudo nano /lib/systemd/system/openvpn@.service

Add the followling line at the end of the first section (Unit).

After=multi-user.target

Finally, reboot in order to apply the changes, and everything should be up and running.

sudo reboot

Generate keys

You may be able to just take your bank’s word for it that they are who they say they are, but VPN servers like the one we’re building want proof of the client’s identity as well. They won’t let just anyone in. You need to give a key to each device or user you want to allow to connect to the VPN server. You have a decision to make at this point. You could generate a unique key for each individual device that you want to connect via VPN, or you could take a shortcut and generate a key for each user. The difference is whether you expect to need to connect more than one device at the same time. If you don’t need to connect more than one device per user at the same time, generate a key named for the user. If you think users will need more than one device connected at the same time, I’d suggest naming the key after the device.  Whichever you decide, generate a key like this, substituting the name of the user or computer that will use this key to connect:

sudo su
cd /etc/openvpn/easy-rsa
source ./vars
./build-key-pass NAME

The PEM password is a password you’ll need in order to use the resulting key file. Pick something nice and strong, but also something you won’t forget. If you want to be really paranoid, you could randomly generate one and keep it in a password safe. The choice is yours.

Accept the remaining defaults again, again leaving the challenge password and company name blank. Sign and commit the certificate when prompted.

Almost done. Next we need to convert the user/computer’s key into a format usable by OpenVPN.

cd keys
openssl rsa -in NAME.key -des3 -out NAME.3des.key

Use the same password as you did before. You’ll have to enter it three times. Technically, the first time is a different password, but how are you supposed to keep them straight?

Generating client keys

Connecting a VPN client to a remote server takes a bit of configuration, too. The OpenVPN client has to know where the server is, and it has to have a copy of the keys we generated earlier. All of this configuration gets wrapped up into a file with a .ovpn extension. You can create these by hand if you like, but Eric Jodoin, the author of the original SANS.org article was kind enough to write a script to do it for us. Create the script file.

nano /etc/openvpn/easy-rsa/keys/MakeOVPN.sh

This is a new file, so it will be totally blank. Paste in the following:

#!/bin/bash

# Default Variable Declarations 
DEFAULT="Default.txt" 
FILEEXT=".ovpn" 
CRT=".crt" 
KEY=".3des.key" 
CA="ca.crt" 
TA="ta.key" 
 
#Ask for a Client name 
echo "Please enter an existing Client Name:"
read NAME 
 
#1st Verify that client's Public Key Exists 
if [ ! -f $NAME$CRT ]; then 
echo "[ERROR]: Client Public Key Certificate not found: $NAME$CRT" 
exit 
fi 
echo "Client's cert found: $NAME$CR" 
 
#Then, verify that there is a private key for that client 
if [ ! -f $NAME$KEY ]; then 
echo "[ERROR]: Client 3des Private Key not found: $NAME$KEY" 
exit 
fi 
echo "Client's Private Key found: $NAME$KEY"

#Confirm the CA public key exists 
if [ ! -f $CA ]; then 
echo "[ERROR]: CA Public Key not found: $CA" 
exit 
fi 
echo "CA public Key found: $CA" 

#Confirm the tls-auth ta key file exists 
if [ ! -f $TA ]; then 
echo "[ERROR]: tls-auth Key not found: $TA" 
exit 
fi 
echo "tls-auth Private Key found: $TA" 
 
#Ready to make a new .opvn file - Start by populating with the default file 
cat $DEFAULT > $NAME$FILEEXT 
 
#Now, append the CA Public Cert 
echo "<ca>" >> $NAME$FILEEXT 
cat $CA >> $NAME$FILEEXT 
echo "</ca>" >> $NAME$FILEEXT

#Next append the client Public Cert 
echo "<cert>" >> $NAME$FILEEXT 
cat $NAME$CRT | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' >> $NAME$FILEEXT 
echo "</cert>" >> $NAME$FILEEXT 
 
#Then, append the client Private Key 
echo "<key>" >> $NAME$FILEEXT 
cat $NAME$KEY >> $NAME$FILEEXT 
echo "</key>" >> $NAME$FILEEXT 
 
#Finally, append the TA Private Key 
echo "<tls-auth>" >> $NAME$FILEEXT 
cat $TA >> $NAME$FILEEXT 
echo "</tls-auth>" >> $NAME$FILEEXT 
 
echo "Done! $NAME$FILEEXT Successfully Created."

Exit Nano, saving your changes (ctrl-x,y,enter) Once again, because this is a script, permissions will have to be altered to allow it to run. Make sure you match the casing of the script name here.

chmod 700 /etc/openvpn/easy-rsa/keys/MakeOVPN.sh

Create the Default.txt file to hold the default values the script will use. The casing isn’t important, but it must match what was specified at the top of the script file. I’m keeping the capitalized “D” just to keep it the same as anyone else who followed Eric’s instructions.

nano /etc/openvpn/easy-rsa/keys/Default.txt

Paste in the following, substituting your public IP address for the highlighted text. If you don’t have a static public IP address, you can use a dynamic name from a service like DynDNS or no-ip here as well. The “1194” is the standard port number OpenVPN uses, adjust as needed to match your network configuration.

Note: Once again, I’ve highlighted the protocol (udp) and port (1194), which you can change if needed. See the troubleshooting section at the end for more information.

client
dev tun
proto udp
remote YOUR_PUBLIC_IP_ADDRESS 1194
resolv-retry infinite
nobind
persist-key
persist-tun
mute-replay-warnings
ns-cert-type server
key-direction 1 
cipher AES-128-CBC
comp-lzo
verb 1
mute 20

Exit Nano, saving your changes (ctrl-x,y,enter) Execute the script to create a .ovpn file. Remember to use the user or device name you chose earlier when creating the client key.

cd /etc/openvpn/easy-rsa/keys
./MakeOVPN.sh

The result is a NAME.ovpn file in the /etc/openvpn/easy-rsa/keys folder on the Raspberry Pi. That’s great, but we need the key on the client machine. You can copy the file using a secure copy program like WinSCP, copy it to a flash drive and move it by hand, or any other number of ways to move a file around. Since this is my own private home server, I’m going to put the file on the data share, at least temporarily. Once the key is installed and working on the client, I’ll delete it from the server.

cp /etc/openvpn/easy-rsa/keys/NAME.ovpn /mnt/data/

Keys like this aren’t something you should leave lying around. On the other hand, you should probably have a backup of them somewhere. If you put them on a flash drive, go put it in a safe or something. Don’t let anyone get a hold of your keys, or they have a free pass into your home network, and you may not even notice it. You can always go back and generate new keys, delete the compromised ones, and continue on, of course.

Port forwarding

Before you’ll be able to connect to your home network from outside, you’ll need to set up your router to forward all traffic on port 1194 to the Raspberry Pi. I can’t tell you how to configure the firewall on your router at home because I don’t know what kind of router you have. An excellent resource that may have information specifically for your router is http://portforward.com/.

Client configuration

I’m using the OpenVPN client for Windows, but the instructions should be similar for other platforms. You can download open-source clients for Windows, and source tarballs for other systems from here.

Note: Don’t try to download client software from the links on the front page of the OpenVPN site or you’ll just end up with “SecureTunnel”, a paid-subscription-based system that lets you do exactly what you’re already set up to do on your own. Get the .ovpn file that you generated on the Raspberry Pi over to the computer you’re going to connect from, and put it in the OpenVPN config folder. For Windows users, this should be C:\Program Files\OpenVPN\config.

Connecting the client

You’ll need to be somewhere other than on your own network for this next part. Otherwise you’re seriously crossing the streams, shutting down the containment grid, etc. Disconnect from your home network and tether yourself to a phone or something before continuing. Run the OpenVPN GUI application. It should have created a shortcut in your start menu for Windows 7 users, or on your app list for Windows 8 users. Run it, and it should pick up on the .ovpn file and open a connection. Right-click on the notification area icon (for Windows users), and select “Connect”.

You’ll be prompted for the password you created earlier, and if everything is configured correctly, the OpenVPN icon should turn green, and you’ll be effectively connecting to the outside world as part of your home network. There are, of course, many issues you could run into when using a VPN. Most of them are explained pretty well on the HowTo page of the OpenVPN site. One of the more vexing problems is that of disambiguating IP addresses between your home network, and the network you are connected to. See “Numbering Private Subnets” for more information.

Troubleshooting

“I can’t connect from work”
This one got me, too. If you’ve tested your VPN, and you know it works, but it doesn’t work from a specific network, then port 1194 might be blocked on that network. I couldn’t connect from work when I was using udp port 1194, although it would work find when I tethered my computer to my phone. If that’s the case, then you can change the server.conf and .ovpn files to use a different protocol and port. Changing “udp” to “tcp” and “1194” to “443” will usually work from anywhere, as long as you’re not using that port for something else already. TCP port 443 is what https traffic uses, so the odds of it being blocked are pretty slim. Remember to update your router’s port-forwarding rules, update BOTH files, and restart the OpenVPN server (sudo service openvpn restart) as well as your OpenVPN client, and it should connect just fine.

What’s next

In the next post, we’ll add Resilio Sync, allowing you to build your own private synchronization “cloud”.

Important: This post is part of a series. Each post builds on the previous ones. If you are trying to add just one thing to an existing system that was not built by following this series in order then I cannot promise that these instructions will work for you. In most cases they will, though. Please refer to the Introduction for a list of all the different posts in the series.

Self-Promotion: I have recorded this series as a screencast for Pluralsight:
(http://www.pluralsight.com/courses/raspberry-pi-home-server)
If you have a Pluralsight subscription, please consider watching it. Reading the instructions is one thing, but actually watching it being done demystifies the whole process.

Thank you!

---

Now that the Raspberry Pi is on the network, and sitting on top of a (hopefully) massive pile of your media on its external drive, it would be nice to have it share that media in a form other than simple file shares. File shares are great for a lot of things, but Media can be done a little better.

DLNA (Digital Living Network Alliance… what a horrible acronym) is the technology standard that lets you play songs from your phone and have them come out of the stereo in your living room. It’s not ubiquitous yet, but it’s catching on. It also lets programs like Windows Media Player play music and video “libraries” that are being shared by other computers on the network.

The Raspberry Pi is about to become one of those libraries.

Note: It is also possible for a Raspberry Pi 2 or above to run Plex Media Server, which is a nicer experience overall, but as of the time of this writing, still doesn’t get along with Windows Media Player. WMP will see a Plex server, but can’t seem to play any media off of it. A walkthrough of installing and configuring Plex Media Server is included in my Pluralsight course, but I’m not including it in the blog just yet because of the compatibility issues with WMP, and because MiniDLNA is a smaller, leaner package which I think works a bit better on the Raspberry Pi. I’ll be adding a separate post about Plex Media Server later on, at which time I’ll revise this note.

Installing MiniDLNA

This installation will be done from the command line, and involves a bit more configuration than some of the other features in this series. Get yourself logged in to a command line as “pi”, and start by installing MiniDLNA. The version in the repositories isnt always totally up to date, but it has worked well for me so far.

sudo apt-get install minidlna

As installs go, this one’s pretty quick, but there is a bit of configuration to be done before it’s ready to go. Start by editing the MiniDLNA defaults:

sudo nano /etc/default/minidlna

Find the line that says “#USER=”minidlna””, and remove the pound sign / hash mark from the beginning, so that if just says “USER=”minidlna””. Do the same for the “GROUP=” line right below it. Close and save the file (ctrl-x,y,enter).

Next, edit the main MiniDLNA configuration file.

sudo nano /etc/minidlna.conf

There are several things to be done here. The first is to set up the folders that will hold the various kinds of media you want shared. A little way down the file, find the first line that isn’t a comment. It should say something like “media_dir=/var/lib/minidlna”. On my drive, there are separate folders for Music, Pictures, and Videos, so I’ll add multiple media_dir lines, one for each kind of media. The comments in the file do a pretty good job explaining the syntax, but for reference, here’s what mine says:

media_dir=A,/mnt/data/public/Music
media_dir=P,/mnt/data/public/Pictures
media_dir=V,/mnt/data/public/Videos

Note that the paths are case sensitive, so while my “Public” share is capitalized, the actual folder is not. The folders within my public folder are, however. This is more for demonstration purposes than anything. Feel free to capitalize your folders however you like.

The letters A, P, and V indicate what kind of content each folder contains (audio, pictures, or video). Next, change the database and log folders to go to the data partition as well. This will conserve space on the relatively small OS partition. The lines to configure this are just below the media_dir lines modified above. Remove the pound signs, and change the folder paths:

db_dir=/mnt/data/MiniDLNA
log_dir=/mnt/data/MiniDLNA/log

The result should look something like this:

Further down the file, uncomment the two settings “inotify” and “notify_interval”. This will allow MiniDLNA to update its database approximately every fifteen minutes with new or updated files.

Close and save the file (ctrl-x,y,enter).

If you were to start MiniDLNA and check its logs right now, you would find that it complains that the number of “watches” is too low. This is how many folders and files it can be watching for changes at a time, and currently its default value is too low in its opinion. It will probably work just fine as is, but we can shut the error up easily enough before continuing. Edit the sysctl.conf file.

sudo nano /etc/sysctl.conf

At the bottom of the file, add a line that says

fs.inotify.max_user_watches = 65536

This is the smallest number that MiniDLNA won’t complain about. Close and save the file.

Next, you need to set up MiniDLNA to run automatically at boot time.

sudo update-rc.d minidlna defaults

And finally, reboot the Pi to take both of these changes into account.

sudo reboot

When everything has restarted, you should have a working MiniDLNA installation, with no complaints in the log file.

Warning: Depending on the size of your media collection, it may take the poor little Raspberry Pi quite a while to index everything (mine took almost two days). If you look at your CPU usage, you’ll see that it stays pretty much pegged for as long as it takes. Don’t worry, you haven’t reached the Pi’s limit, at least not permanently. When it has finished indexing, you’ll see its usage drop back to almost nothing.

You should now be able to open something like Windows Media Player, and see an entry for “RPHS: minidlna” in the list on the left. Drilling down into this entry will show you the media that has been indexed so far.

I am not a MiniDLNA expert, by any means, so you might want to look elsewhere for more configuration information. Also, MiniDLNA has recently changed names to become “ReadyMedia”, but the Raspberry Pi version still calls itself “MiniDLNA”, at least for now.

Forcing a re-scan

As commenter Ivelin Tenev points out, MiniDLNA may not notice files that you add when it’s not running. If you have disconnected your data drive from the Pi, loaded files on it, and then re-connected it, MiniDLNA may not notice the new files because it wasn’t “watching” at the time the files were added. You can force a re-scan by typing:

sudo minidlna -R

and then restart the minidlna service like this

sudo service minidlna restart

What’s next?

In the next post, we’ll add an OpenVPN server, allowing you to securely connect to your home network when you’re away.

Note: This post is part of a series. Each post builds on the previous ones. If you are just trying to add one thing to an existing system that was not built following this series, then I cannot promise that these instructions will work for you, although they probably will. If you’ve started from something other than a non-NOOBS Raspbian image, then you’ll probably need to adjust for that.

Please refer to the series Introduction for a list of all the different posts in the series.

Self-Promotion: I have recorded this series as a screencast for Pluralsight:
(http://www.pluralsight.com/courses/raspberry-pi-home-server)
If you have a Pluralsight subscription, please consider watching it. Reading the instructions is one thing, but watching it done demystifies the whole process.

Thank you!

---

Now that the Raspberry Pi has the space to hold your stuff, it’s about time it started doing something useful around the house. Its first task will be to share files from the hard drive. In the previous post we added a hard drive with two partitions, one ext4 partition for the operating system, and one NTFS partition to hold data. If you decided to format your data partition using the ext4 filesystem, then you’ll need to keep that in mind as we proceed.

Prerequisites

• A Raspberry Pi
• A mounted, USB hard drive with space for the stuff you want to share
  Mine is the second hard drive partition, mounted at /mnt/data

Install Samba

The “Samba” package creates network file shares in a way that Mac, Windows, and other Linux computers on the network will understand. Install Samba as follows:

sudo apt-get install samba samba-common-bin

That’s all there is to the installation, but there’s a bit of configuration before there will be anything to see over the network.

Set up shares

In the previous version of this article, I used Webmin to do the heavy lifting of setting up the shares. In all the time I’ve been running my Raspberry Pi Home Server, I’ve found that this is the only thing I ever really used Webmin for, though. As a result, this version of the series will set up the shares manually, and the Webmin article has been moved to the end as an optional step.

To create a share, you first create a folder that you want to share, tell Samba to share it, and set up the permissions correctly so that users connecting to the share through Samba will be able to write to it.

You can get into all kinds of detail about who can and can’t write what and where, but I’ll leave that up to you to explore on your own. What I’ll be setting up is a simple public share to be used by the family members in my house. The permissions will be wide open on this share.

Create the folder to be shared

First, create a folder to be shared. I’ll call mine “public”, and put it in the root of the hard drive I mounted in a previous post.

mkdir /mnt/data/public

Take a look at the permissions for the newly-created folder

ls -l /mnt/data

You’ll see that the new folder belongs to the “root” user and group even though we created it without using “sudo”. This is because my filesystem is NTFS, and it doesn’t support Linux’s file system permission system. If you made your data partition ext4, then this is going to cause problems for the Samba user. Assign the folder’s ownership to the “nobody” user and the “nogroup” group.

sudo chown nobody:nogroup /mnt/data/public

Check out the result

ls -l /mnt/data

If your filesystem is NTFS, this will have had no effect. If your filesystem is ext4 though, you should see that the public folder is now owned by “nobody” and “nogroup”. Now let’s share this folder with the network.

Open the Samba configuration file using nano

sudo nano /etc/samba/smb.conf

This file is pretty well documented, and will explain most things that you need inline. We’ll be making a number of changes.

Enable WINS support

Windows Internet Name Service (WINS) is kind of like the Domain Name System (DNS) that turns web addresses into IP addresses when you use a browser. WINS does this inside of your own network so that instead of having to remember your server’s IP address, you can refer to it by name.

Scroll down in the file a little until you see the “Global Settings” section. If your home network is using a different workgroup name than “WORKGROUP”, you’ll want to change that here. If you don’t know what that means, then you probably haven’t changed your workgroup name, so you can just leave it alone.

Just below that is a commented-out line that says “wins support = no”. Uncomment this by removing the pound sign from the beginning of the line, and change “no” to “yes”.

Scroll down a little, and just before the Networking section, add a line that says

guest account = nobody

Scroll down to the bottom of the file, and add a new section that defines the new public share.

[public]
   path = /mnt/data/public
   public = yes
   writeable = yes

Close and save the file (Ctrl-X,Y), and then restart the Samba service

sudo service smbd restart

That should do it. You should now be able to see the public share from the other computers on your network. You should also be able to refer to your server by name now instead of by address, and that includes when connecting via RealVNC.

Give it a try now. Open a file manager, and navigate to “\\rphs\public”. You should be able to create and edit files in this share without any restrictions.

Disclaimer

This is a wide-open file share. Anyone on the local network can add, edit, or delete the files it contains. I’m more interested in having a place to put stuff than in controlling access to it. Controlling access to the drive contents is beyond the scope of this post, but may be a subject for a future post.

Make some folders

Create some file folders in the Public share to hold your stuff, and copy it in. Music, Pictures, Videos, anything you want. For this server, I’ve created the following folder structure.

• Documents
• Downloads
• Music
• Pictures
• Videos

You don’t have to use the same structure as me, of course, but this is the one I’ll be using in future posts, so if you structure your differently, just remember to adjust any paths in future posts accordingly.

Load it up

When it comes to copying the files in, you can save some time, and skip the network bottleneck by moving the external hard drive to the computer that currently has the data. Because the data partition was formatted as NTFS in the previous post, when you plug it in to your Windows or Mac computer, you’ll just see a drive called “Data”, or whatever you named it.

Remember to shut down the Raspberry Pi before removing the hard drive, of course. Just put it back when you’re done copying files, and start up the Raspberry Pi as usual.

Wrapping up

You now have a public share that exposes the data partition on the hard drive attached to the Raspberry Pi Home Server. You can use this to store your media and documents on the network so that they are accessible to your other computers and devices.

You’ve finished another article, so it’s time to shut down the Pi, and take a backup of the SD card.

What’s next?

In the next post, we’ll get a media server up and running so that you can stream music and movies all over your house from the hard drive share.