Note: This post is part of a series. Each post builds on the previous ones. If you are just trying to add one thing to an existing system that was not built following this series, then I cannot promise that these instructions will work for you, although they probably will. If you’ve started from something other than a non-NOOBS Raspbian image, then you’ll probably need to adjust for that.

Please refer to the series Introduction for a list of all the different posts in the series.

Self-Promotion: I have recorded this series as a screencast for Pluralsight:
(http://www.pluralsight.com/courses/raspberry-pi-home-server)
If you have a Pluralsight subscription, please consider watching it. Reading the instructions is one thing, but watching it done demystifies the whole process.

Thank you!

---

At this point in the previous version of this series, I demonstrated both mounting a hard drive and booting from it. After building and rebuilding this server many times as part of maintaining this series, I found that booting from the hard drive this early in the process makes the rest of the series inconvenient because you can no longer take backups of the whole system by simply taking an image of the SD card.

As a result, booting from the hard drive is now a separate post, and one of the last in the series. I found this to be much more user friendly, and simplifies things if you mess something up and want to rewind to your last “save point”. We’ll get everything set up and working first, and then transfer the OS to the hard drive in this version of the series.

Gather your tools

This post uses some software packages that are not included by default in the Raspbian distribution. You’re going to need NTFS filesystem support, and the gdisk tool. Install them before continuing. One or more of these tools may already be installed on your system, but telling apt-get to install something that’s already there won’t hurt anything.

sudo apt-get install ntfs-3g ntfs-config gdisk

Pick a hard drive

Pretty much any external USB hard drive will do. For my own server at home these days, I’m using a 120GB SSD for my OS drive, and a dual-drive 2TB RAID enclosure (Cavalry CADA-SA2) for my data. This might seem like overkill for a Raspberry Pi, but one of my home server’s jobs is to centralize and share my music collection. It took me four solid weeks to rip every CD I own, and I do not plan to repeat that process. My family photos and videos are on the RAID drive as well. Basically, this is all stuff I don’t want to lose due to a hard drive failure.

Note: Yes, I have a separate offline backup too, but swapping out a dead drive in a RAID array is easier (and faster) than rebuilding the whole thing by hand from a backup.

Linux device names

Before we start, lets take a moment to discuss the way in which Linux refers to devices in general, and more specifically the way it refers to hard drives. Windows users are used to seeing multiple drives show up with different letters. You have your “C” drive, where Windows and all of your programs are installed. Maybe you also have a second hard drive called “D”, or maybe “D” is CD or DVD drive on your computer. “A” and “B” used to refer to floppy drives, and are reserved for historical purposes. Each of these devices is completely separate island of stuff.

Linux has no drive letters. Linux uses what’s called a “unified file system”, meaning that files, folders, and even entire drives appear together under one root. You can see all of these different devices in the “/dev” folder. Take a look:

ls /dev

I won’t go into what all of these entries mean, but you can see that there are a lot of them. If I were to plug in an external hard drive, or perhaps a flash drive, I would see new entries pop up in the list. Here’s another screen shot after I’ve plugged in a flash drive.

Look about halfway down the third column, and you’ll see new entries called “sda”, “sda1”, and “sda2”. These represent the flash drive, and the two partitions it contains, respectively. You’ll most likely only see one partition on a typical flash drive, but this one has two partitions for demonstration purposes.

Storage devices like a flash drive or a hard drive can be divided up into multiple sections called partitions that can make it appear to be multiple physical drives. This list of where each partition begins and ends is called the “partition table”, and every drive has one.

That explains the “1” and the “2”, but where did that “a” come from? On a computer where drives are connected via SCSI (Small Computer System Interface), “a” would be the first device, “b” would be the second, etc. Drives connected via IDE (Integrated Drive Electronics), or SATA (Serial AT Attachment) have similar concepts of “first”, “second”, etc.

This is not the case with USB. USB devices have no concept of “first” or “second”. The USB ports on your computer aren’t numbered, at least not externally. You could plug the same device into any USB port and it should work just the same. USB device names are assigned on a first-come, first-served basis. The first drive to be ready gets to be “sda”, the next one “sdb” and so on. If you plug in a third device, you could expect it to be called “sdc”.

If you only have one drive, then it would obviously always be called “sda”, and there’s no problem, but since you’re building a server, it’s conceivable, and even likely that you will eventually end up with more than one hard drive attached to it. Perhaps one drive will house videos, and another one will have music on it.

Even if you don’t see a need for that yet, you’ll end up filling up a drive eventually, and you’ll want to add another one. As soon as you add more USB drives, however, you can’t count on their names being the same every time the machine starts up. As a result, drives may end up mounted in the wrong places, and programs won’t find their files where they expect them to be because the drive that used to be “sda” was a little slower off the line one morning and got stuck with the name “sdb” instead.

Let’s take care of that particular problem before going any further.

Format the hard drive

If the drive you’re planning to use already has stuff on it, then back its contents up somewhere. and restore them when you’re done. This approach calls for nuking and repaving the whole hard drive.

Caution: Even if you wanted to live dangerously and manipulate the partitions in place, I wouldn’t do it without a backup copy first. There are just too many things that could go wrong. What if the power went out partway through the process? Don’t risk it. Just copy your stuff to another drive, and copy it back when you’re done.

The key to fixing the device naming problem requires us to have a specific kind of partition table called a GUID Partition Table. This is different than the older Master Boot Record which is used by most smaller drives or older Windows machines.

In order to create a GPT, you’ll need to use Linux’s Partition Editor (parted) application. Raspbian already includes parted, so there’s nothing to install. Without hooking up the hard drive just yet, run it with the following command:

sudo parted

You should see something almost, but not quite entirely unlike a command line. To see a list of all the known devices and partitions, type “print all”. Parted should print out a table with two rows similar to this:

The rows in the table represent the partitions. The columns will tell you where each partition begins and ends, how large it is, and what kind of filesystem it contains.

If you see more than one table at this point, then you probably have another drive plugged in somewhere. I strongly advise shutting down and removing all drives other than the SD card before continuing.

The SD card in the image above was created using the Raspbian image file, and has two partitions on it, a small “boot” partition, and the root filesystem partition. If you started by using NOOBS, which you really shouldn’t, then you probably have five partitions. Future releases could divide things up differently, so don’t worry if your output doesn’t look like the example.

I’ll say it again here, but I really don’t recommend building a server from a NOOBS image. NOOBS is convenient for experimentation and classroom use, but it complicates a server build without adding anything of value.

Attach the hard drive

Shut down, plug your external drive into a free USB port on the Raspberry Pi, and restart.

Note: You could try plugging in the drive with the Pi running, but unless the drive has its own power supply, this might draw too much power and cause the Pi to restart involuntarily.

If you’re booting to the desktop, then there’s one more thing you need to do. When running the desktop, the Pi will auto-mount devices in a folder called “media” under the root. Take a look at what’s there now.

ls /media/pi

You’ll see different stuff depending on what the drive used to be used for. The point is that the drive is “mounted”, and that’s going to prevent us from formatting it easily. If your drive has more than one partition on it, you can expect to see more than one thing in the /media/pi folder.

Un-mount the devices one at a time like this:

sudo umount /media/pi/NAME

Where “NAME” is the name you saw listed above.

You can also do this from the desktop using the “File Manager” application. Open it by clicking the Menu icon, then “Accessories”, and then “File Manager”. At the top of the left pane should be a drop-down labelled “Directory Tree”. Switch that to “Places”, and any mounted devices should show up at the bottom. Right-click on your devices and then “Unmount”.

Run parted again

sudo parted

List the known partitions again.

print all

This time, you should see two tables like this:

In my example, the second one is the external drive, although the order is not guaranteed here.

Notice the headers above the tables. The header tells you information about the drive in general. In this case, the header tells us that the hard drive I’m using for this demo was assigned the name “/dev/sda”, that it’s 120GB in size, and that its partition table type is “msdos”. That means it’s using the old MBR-style partition table, and that’s not going to work for this server.

Select the external drive by typing “select /dev/sda”. Type “print” all by itself to make sure. You should see the second table from above again. Read the header carefully, and make sure it’s referring to the external drive, because you’re about to blow away the partition table.

Create a new partition table with the command “mklabel gpt”. Read the warning carefully, and make sure it refers to “/dev/sda”.

Answer “y” to the prompt, and in a few seconds, you will have a nice blank slate to work with. Check the results with another “print”. The header should now say that the partition table type is “gpt”, and there should be no partitions on it.

Create new partitions

Next, you’ll need to create two new partitions; one to hold the root filesystem, which I will cover later on in the series, and one to hold your data. You could make one big partition, combining the root filesystem and data storage in one, but there are some benefits to keeping the data partition separate. The root filesystem must be formatted using Linux’s native filesystem, ext4. The data partition, however, can be any type you want, which means you can pick something that other, non-Linux computers will understand.

Root Partition

This is where Linux and your programs will go later on in the series. You can think of it as Windows’ C: drive. The size is up to you, but 16GB is probably fine for the simple server described in this series. If your existing SD card is larger, or you are planning to install a lot of programs, go bigger. Just make sure the partition is at least as large as the SD card you are currently using.

Create a new, partition starting at the beginning of the drive:

mkpart os 0gb 16gb

Substitute your own value for 16gb so that it matches or exceeds the size of your SD card. I recommend going slightly larger than your SD card, myself, just to be sure everything fits. I typically build on an 8GB card, but create a 16GB partition on the drive.

Data Partition

This is where you’ll put your “stuff”, and it should take up the rest of the available drive space. The mkpart command can interpret the beginning and ending parameters in a variety of different formats. Use this ability to create a second partition that begins where the first partition ends, and takes up all the rest of the space on the drive:

mkpart data 16GB 100%

Finally, use the “print” command again to see the results.

That’s it for the partitions, so type “q” and press enter to exit parted. You can ignore the warning about updating /etc/fstab for now. We’ll get to that soon enough. You now have two partitions, but they’re completely blank at this point.

Format the data partition

Before you can store anything on a partition, you need to format it. Choosing NTFS as the filesystem will allow you to simply plug the drive into any Windows, MAC, or Linux desktop computer and manipulate the drive contents. This can be especially helpful for loading up large amounts of data, such as the family’s music and photo collection, without having to squeeze it all through the network connection. If your main computer is a Mac or another Linux machine, then this may not matter as much to you since those systems understand ext4.

Format the second partition as follows, filling in whatever name you want the partition to have. I’ve called mine “Data”.

sudo mkfs.ntfs -Q -L Data /dev/sda2

Note: Don’t forget the –Q (Quick) or you’ll be waiting around a while.

At this point, I would shut down the Pi, attach the hard drive to your regular computer, and make sure you can read the partition. In some recent versions of Windows, partitions formatted as NTFS on a Linux system may fail to mount. This has happened to me on some drives and not others, so make sure you check yours before moving forward. If the drive doesn’t show up in Windows, start up Disk Management (Windows key, type “disk”, and click on “Create and format hard disk partitions”). Find the data partition you created above, right-click on it, and let Windows format it itself. This will ensure you have the best compatibility moving forward.

Uniquely identifying a filesystem

Warning: Because the next section involves copying around large strings of unmemorizable data, I recommend performing the steps through a remote SSH window, or from a terminal window on the desktop either directly or via VNC. The Ids you’ll need to copy need to be copied verbatim, letter for letter, or you’ll find yourself unable to boot, and you’ll have to start over again from your last backup.

As I mentioned earlier, just because the data partition is called /dev/sda2 right now doesn’t mean that it will have that name in the future if you add more drives to the system. What we need is a way to uniquely refer to this drive no matter what letter it get assigned on any given day.

This was why we built the drive with a GUID Partition Table. Each partition on a GPT device is assigned a universally-unique identifier (UUID).

Note: The difference between GUID and UUID is not important here, they are, for all intents and purposes, the same thing; a very long, randomly-assigned number.

Instead of identifying the filesystem’s partition by location, we want to refer to it by Id so that it can be found no matter what order the drives were discovered in. Use the “blkid” comman to find out more about /dev/sda2 (the data partition):

sudo blkid

You should see a listing of information about all of your drives and their partitions, like this:

The piece of information we’re interested is the “UUID” of sda2. In my case that’s on the next to last line of the above screenshot, not the “PARTUUID” on the last line. If you partitioned your drive differently, for instance putting the big partition first, then you’ll need to adjust accordingly. You can double check this using parted and the “print all” command again to be sure.

Note: PARTUUID uniquely identifies the partition, and will come into play later on when we boot from the hard drive. “UUID” identifies the filesystem on that partition, and it’s what we need for mounting.

Take a picture, write it down very carefully, or select and copy the text if you are doing this through a remote terminal connection like I am. If you’re on the desktop directly or through VNC, you could also open up the built-in desktop text editor, and paste the UUID there. You can find it at Menu -> Accessories -> Text Editor.

Mounting filesystems

Linux has assigned the name /dev/sda2 to our data partition, but you can’t just use that name to refer to it. In the Linux world, you first need to tell the OS where that filesystem belongs in the grand scheme of things. To do this, you’ll need to edit the filesystem table. This file controls what gets mounted where, and in what order. Take a look at the current contents.

cat /etc/fstab

The second line is the boot partition, where things like config.txt and cmdline.txt live. The third line is the root filesystem, which you can tell by the “/” in the second column. At this point, it’s loading from the 2nd partition on the SD card (mmcblk0p2). We’ll come back to this in a later post and change it to the OS partition on the hard drive.

Edit the fstab file:

sudo nano /etc/fstab

Add a new line after /dev/mmcblk0p2, but before the comments. The first column identifies the partition. You’ll need to use the format “/dev/disk/by-uuid/” and then the UUID you copied down above. Press TAB, and the path to the location in the filesystem where you would like the partition to mount. I recommend /mnt/data. Tab again, and specify the filesystem type. If you’re using NTFS, just put the letters “ntfs”, if you’re using ext4, type that. Tab again and type the word “defaults”. Tab again and type a zero, then another tab and another zero.

The end result should look like this:

NOTE: I’ve replaced the tabs with spaces to fit it on one screen, either one works just as well. You can pretty up the columns so they line up if you like.

Exit Nano, saving your changes (ctrl-x,y,enter)

The next time the system reboots, it will try to mount the hard drive at /mnt/data. Before that can succeed, you need to create a folder in that location. You can think of it as a placeholder where the actual drive will appear later on.

sudo mkdir /mnt/data

To prove that the drive mounts successfully, create a file in the new folder using the “touch” command.

sudo touch /mnt/data/placeholder

Look at the folder’s contents

ls /mnt/data

You should see a file called “placeholder”. If you ever look in this folder and see that file, it should act as a warning that the drive has not mounted properly.

That’s it. you’re ready to reboot the Pi, and start using your new data drive.

sudo reboot

When the system has rebooted, look at the contents of /mnt/data again.

ls /mnt/data

Instead of the placeholder file, you should see an empty drive. If you formatted the drive as ext4, you’ll see a lost+found folder, but nothing else. If you see the placeholder file, then something has gone wrong. Verify your changes to fstab, and see if you skipped a step, or used the wrong UUID above.

Wrapping up

You now have an external drive to store your stuff in, and it should mount in a predictable place every time the server reboots. The next few posts will start taking advantage of this new storage space.

Shut down the Pi, and take a backup of the SD card.

What’s next?

In the next post, we’ll put the new drive to work sharing files over the local network.