Note: This post is part of a series. Each post builds on the previous ones. If you are just trying to add one thing to an existing system that was not built following this series, then I cannot promise that these instructions will work for you, although they probably will. If you’ve started from something other than a non-NOOBS Raspbian image, then you’ll probably need to adjust for that.

Please refer to the series Introduction for a list of all the different posts in the series.

Self-Promotion: I have recorded this series as a screencast for Pluralsight:
(http://www.pluralsight.com/courses/raspberry-pi-home-server)
If you have a Pluralsight subscription, please consider watching it. Reading the instructions is one thing, but watching it done demystifies the whole process.

Thank you!

Update: If you want to know what’s going on under the covers, or want tighter control over your exact configuration, then by all means, read this post in its entirety. I encourage you to do so just to gain an understanding of what’s going on. If, however, you’re just looking for the fastest way to set up a home VPN on a Raspberry Pi, then you should check out PiVPN. It’s basically everything in this article, completely automated. That means that everything you’re about to read can be condensed down to a single command.

curl -L https://install.pivpn.io | bash

After recently having my cable modem upgraded (and finding that it out-performed my own wireless router), I was having some trouble getting my VPN working again. Rather than go through all the steps of my own post, I thought I’d give PiVPN a shot at it. Ten minutes later, most of which was eaten up by generating Diffie-Hellman keys, I had new ovpn files for my various devices, and everything just worked. PiVPN is awesome. It makes me wish there were similar scripts for the other aspects of running a Raspberry Pi Home Server.

Introduction

Now that the whole house is humming along, sharing files, downloading things, and backing everyone up, you might be wondering if there’s anything left that the Raspberry can do for you. The answer is yes. In this article, we’ll set up the Raspberry Pi to act as an OpenVPN server, allowing you to securely access your home network from anywhere. OpenVPN is an open-source, cross-platform, virtual private networking (VPN) application. VPNs let you route internet traffic through a secure, encrypted channel, back to a network that you trust and/or control. You may have used one in order to securely access resources on the network at your office when you’re on the road. Developers sometimes use them to simulate traffic coming into their network from outside for testing. You can add these same abilities to your home network so that you can get to your stuff from work, or a hotel, or anywhere else with internet access. Running your own VPN means that no matter where you get an internet connection from, you are effectively “at home”. You don’t need to worry about fellow patrons at the coffee shop listening in on your network traffic because the traffic between you and your VPN is highly encrypted.

Acknowledgements

Once again, I didn’t invent this stuff. Most of the information about how to set up OpenVPN comes from a whitepaper by Eric Jodoin of the SANS institute. That whitepaper was later paraphrased and simplified in a pair of posts by Lauren Orsini. Both are excellent reading, and go into far more depth about how all this stuff works than I plan to. I’m just putting it into the same format as the other posts in the series, and organizing them in a logical progression, building on top of the previous posts in this series.

Prerequisites

In order to connect to your home network’s VPN when you are away from home, you are going to need either a static IP address, or a dynamic IP resolution service like www.no-ip.org. My home router updates no-ip automatically, so I have not set up a program on the Raspberry Pi to do this. Other tutorials exist out there to handle this part.

Internet security

Warning: The explanation that follows is super-non-technical™, and probably wildly inaccurate in many important ways. I am not a security or cryptography expert, but this is, in layman’s terms, how internet security works. When you visit your bank’s website, and something in your address bar turns green, or grows a little lock, it means that someone at the bank went to some authority that we’ve all agreed to trust, and got a certificate that says “Yup these guys are the bank alright”, and installed it on the web server you’re talking to. As long as you trust the people that made the certificate to only give it to the company that paid for it, and as long as you trust the bank to only install the certificate on their own servers, then you have a way to prove that the server you’re talking to belongs to the company you think it does, or at least a company that the authority vouched for.

Although I’m sure you trust your own word that the Raspberry Pi Home Server that you’ve been building is your own, your other computers are still going to want proof that the thing on the other side of the internet is, in fact, your server, and not someone else pretending to be your server. That’s kind of the whole point of this exercise, after all. Since you trust yourself, you can act as your own “certificate authority” and make your own certificates. You then install your homemade certificates on both the server and the client machines, and they can use that certificate to encrypt traffic back and forth between them.

Install OpenVPN

First things first, you’ll need to install the OpenVPN software onto the Raspberry Pi. You’ll also need the OpenSSL and EasyRSA packages in order to secure your connection later on. Installing them is as simple as…

sudo apt-get install openvpn openssl easy-rsa

That’s the easy part. Now comes the configuration. The EasyRSA installer has created some sample configuration files for us, and they’ll form the skeleton of the real configuration. Copy the entire directory of sample configuration files like this:

sudo cp -r /usr/share/easy-rsa /etc/openvpn/easy-rsa

Open the “vars” file for editing.

sudo nano /etc/openvpn/easy-rsa/vars

Find the “export EASY_RSA” entry, set it as follows:

export EASY_RSA="/etc/openvpn/easy-rsa"

A little further down, you’ll see a line that begins with “export KEY_SIZE”. In current installations, this should already be set to 2048, so you shouldn’t need to change it. Previous installations used 1024 bit keys, but the current recommendataions are to use 2048 for increased security.

Move to the bottom of the file, and change the defaults that are defined there to match your location and network. This will save you some time later on when you are asked to provide this information again for each user you set up. Mine looks like this:

export KEY_COUNTRY="US"
export KEY_PROVINCE="OH"
export KEY_CITY="Columbus"
export KEY_ORG="Home"
export KEY_EMAIL="melgrubb@…"
export KEY_OU="RPHS"

KEY_NAME affects the name of the resulting key file, but is otherwise arbitrary. The organization unit (OU) setting is not important for a small home network, so I’ve just gone ahead and used the server name.

export KEY_NAME="RPHS"

You can leave the other settings at their defaults. Close Nano, saving the file (ctrl-x, y, enter)

Become a certificate authority

In order to create certificates, you’ll need… wait for it… a certificate. In this case, it’s a “root certificate”. This is the kind of thing that one of the trusted authorities out on the web would have. The “easy-rsa” package you installed earlier can generate such a certificate for you. Run the following commands to set up a key server. Notice that the “sudo su” command is being used here. You’re going to stay in “god mode” for pretty much the remainder of this post.

sudo su
cd /etc/openvpn/easy-rsa
source ./vars
./clean-all
./build-ca

This last command will prompt you for a lot of values, fortunately, you set up reasonable default values above, so you can just hit enter to accept them. When prompted for “Common Name”, I’ve used the server’s name again (RPHS). When you create certificates for other computers or users, you’ll want to use their names here.

When that finished, enter the following command, substituting the name of your server, and accepting the defaults again. You’ll get a couple extra questions this time. Make sure the challenge password and company name are left blank, and accept any other defaults.

./build-key-server RPHS

Answer yes to the “Sign the certificate?” and “commit?” prompts.

Next, we’ll build the Diffie-Hellman parameters file.

./build-dh

Go get some coffee or something, this step is likely to take a very long time. Sometimes it does, sometimes it doesn’t. It’s pretty random what kind of wait you’ll have, but since the change from 1024 bit keys to 2048 bit keys, the likelihood of this taking a long time has definitely increased.

The screen will fill up with a lot of dots and plus signs while it works to let you know that it hasn’t gone to sleep or locked up. As long as the symbols keep coming, let it keep doing its thing.

When it’s done, generate a hash-based message authentication code (HMAC). This is yet another layer of protection, and helps to prevent denial of service (DOS) attacks without taxing the Pi’s processor too much with decryption of bogus data.

openvpn --genkey --secret keys/ta.key

Configure the OpenVPN Server

Now it’s finally time to edit the OpenVPN configuration and tie up the loose ends.

nano /etc/openvpn/server.conf

You’ll notice that the editor is totally blank. That’s because this file doesn’t exist yet. Paste in the following text, substituting your own values for the highlighted values. You’ll need your Raspberry Pi’s IP address, the IP address of your router, and the name you used above when calling build-key-server.

Note: I’ve also highlighted the protocol (udp) and port (1194). You can change these to something else if needed. I’ll talk about this more in the troubleshooting section at the end.

local 192.168.1.XXX # YOUR PI'S IP ADDRESS
dev tun
proto udp
port 1194
ca /etc/openvpn/easy-rsa/keys/ca.crt
cert /etc/openvpn/easy-rsa/keys/RPHS.crt
key /etc/openvpn/easy-rsa/keys/RPHS.key
dh /etc/openvpn/easy-rsa/keys/dh2048.pem
server 10.8.0.0 255.255.255.0
# server and remote endpoints 
ifconfig 10.8.0.1 10.8.0.2 
# Add route to Client routing table for the OpenVPN Server
push "route 10.8.0.1 255.255.255.255" 
# Add route to Client routing table for the OpenVPN Subnet
push "route 10.8.0.0 255.255.255.0" 
# your local subnet 
push "route 192.168.1.0 255.255.255.0" # YOUR PI'S IP SUBNET
# Set primary domain name server address to the SOHO Router
# If your router does not do DNS, you can use Google DNS 8.8.8.8
push "dhcp-option DNS 192.168.1.1" # YOUR ROUTER'S IP ADDRESS
# Override the Client default gateway by using 0.0.0.0/1 and
# 128.0.0.0/1 rather than 0.0.0.0/0. This has the benefit of 
# overriding but not wiping out the original default gateway.
push "redirect-gateway def1" 
client-to-client
duplicate-cn
keepalive 10 120
tls-auth /etc/openvpn/easy-rsa/keys/ta.key 0
cipher AES-128-CBC
comp-lzo
user nobody
group nogroup
persist-key
persist-tun
status /var/log/openvpn-status.log 20
log /var/log/openvpn.log
verb 1

Exit nano, saving your changes (ctrl-x, y, enter) Next, you need to allow the Raspberry Pi to forward IP traffic, which it does not do by default.

nano /etc/sysctl.conf

Find the line that says “Uncomment the next line to enable packet forwarding for IPv4”, and uncomment the line immediately after it.

# Uncomment the next line to enable packet forwarding for IPv4
net.ipv4.ip_forward=1

Exit Nano, saving your changes (ctrl-x,y,enter), and force a reload of the settings.

sysctl -p

Configure Firewall

The Raspberry Pi has its own firewall, which must be configured to allow the VPN traffic through. Create a script file to automate the opening of the appropriate ports.

nano /etc/firewall-openvpn-rules.sh

Copy in the following text, substituting your own Raspberry PI’s IP address where highlighted.

#!/bin/sh

iptables -t nat -A POSTROUTING -s 10.8.0.0/24 -o eth0 -j SNAT --to-source 192.168.1.XXX

Change the permissions on the file you just created so that it can be executed, and assign ownership to the root user.

chmod 700 /etc/firewall-openvpn-rules.sh
chown root /etc/firewall-openvpn-rules.sh

This script file needs to run every time the Raspberry Pi boots up in order to do us any good. Edit the /etc/network/interfaces file.

nano /etc/network/interfaces

Find the line that configures the wired ethernet port. If you are running your server wirelessly, then you’ll need to adjust accordingly. Insert a new line, indented underneath so that the result looks like this:

...
iface eth0 inet dhcp
    pre-up /etc/firewall-openvpn-rules.sh
...

This will ensure that the firewall rules are applied to that network interface even before it has started up.

One last thing, and this is new with the Jessie release. As commenter Craig pointed out in the previous version of this post, we need to stop OpenVPN from starting TOO soon. Edit the OpenVPN service file.

sudo nano /lib/systemd/system/openvpn@.service

Add the followling line at the end of the first section (Unit).

After=multi-user.target

Finally, reboot in order to apply the changes, and everything should be up and running.

sudo reboot

Generate keys

You may be able to just take your bank’s word for it that they are who they say they are, but VPN servers like the one we’re building want proof of the client’s identity as well. They won’t let just anyone in. You need to give a key to each device or user you want to allow to connect to the VPN server. You have a decision to make at this point. You could generate a unique key for each individual device that you want to connect via VPN, or you could take a shortcut and generate a key for each user. The difference is whether you expect to need to connect more than one device at the same time. If you don’t need to connect more than one device per user at the same time, generate a key named for the user. If you think users will need more than one device connected at the same time, I’d suggest naming the key after the device.  Whichever you decide, generate a key like this, substituting the name of the user or computer that will use this key to connect:

sudo su
cd /etc/openvpn/easy-rsa
source ./vars
./build-key-pass NAME

The PEM password is a password you’ll need in order to use the resulting key file. Pick something nice and strong, but also something you won’t forget. If you want to be really paranoid, you could randomly generate one and keep it in a password safe. The choice is yours.

Accept the remaining defaults again, again leaving the challenge password and company name blank. Sign and commit the certificate when prompted.

Almost done. Next we need to convert the user/computer’s key into a format usable by OpenVPN.

cd keys
openssl rsa -in NAME.key -des3 -out NAME.3des.key

Use the same password as you did before. You’ll have to enter it three times. Technically, the first time is a different password, but how are you supposed to keep them straight?

Generating client keys

Connecting a VPN client to a remote server takes a bit of configuration, too. The OpenVPN client has to know where the server is, and it has to have a copy of the keys we generated earlier. All of this configuration gets wrapped up into a file with a .ovpn extension. You can create these by hand if you like, but Eric Jodoin, the author of the original SANS.org article was kind enough to write a script to do it for us. Create the script file.

nano /etc/openvpn/easy-rsa/keys/MakeOVPN.sh

This is a new file, so it will be totally blank. Paste in the following:

#!/bin/bash

# Default Variable Declarations 
DEFAULT="Default.txt" 
FILEEXT=".ovpn" 
CRT=".crt" 
KEY=".3des.key" 
CA="ca.crt" 
TA="ta.key" 
 
#Ask for a Client name 
echo "Please enter an existing Client Name:"
read NAME 
 
#1st Verify that client's Public Key Exists 
if [ ! -f $NAME$CRT ]; then 
echo "[ERROR]: Client Public Key Certificate not found: $NAME$CRT" 
exit 
fi 
echo "Client's cert found: $NAME$CR" 
 
#Then, verify that there is a private key for that client 
if [ ! -f $NAME$KEY ]; then 
echo "[ERROR]: Client 3des Private Key not found: $NAME$KEY" 
exit 
fi 
echo "Client's Private Key found: $NAME$KEY"

#Confirm the CA public key exists 
if [ ! -f $CA ]; then 
echo "[ERROR]: CA Public Key not found: $CA" 
exit 
fi 
echo "CA public Key found: $CA" 

#Confirm the tls-auth ta key file exists 
if [ ! -f $TA ]; then 
echo "[ERROR]: tls-auth Key not found: $TA" 
exit 
fi 
echo "tls-auth Private Key found: $TA" 
 
#Ready to make a new .opvn file - Start by populating with the default file 
cat $DEFAULT > $NAME$FILEEXT 
 
#Now, append the CA Public Cert 
echo "<ca>" >> $NAME$FILEEXT 
cat $CA >> $NAME$FILEEXT 
echo "</ca>" >> $NAME$FILEEXT

#Next append the client Public Cert 
echo "<cert>" >> $NAME$FILEEXT 
cat $NAME$CRT | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' >> $NAME$FILEEXT 
echo "</cert>" >> $NAME$FILEEXT 
 
#Then, append the client Private Key 
echo "<key>" >> $NAME$FILEEXT 
cat $NAME$KEY >> $NAME$FILEEXT 
echo "</key>" >> $NAME$FILEEXT 
 
#Finally, append the TA Private Key 
echo "<tls-auth>" >> $NAME$FILEEXT 
cat $TA >> $NAME$FILEEXT 
echo "</tls-auth>" >> $NAME$FILEEXT 
 
echo "Done! $NAME$FILEEXT Successfully Created."

Exit Nano, saving your changes (ctrl-x,y,enter) Once again, because this is a script, permissions will have to be altered to allow it to run. Make sure you match the casing of the script name here.

chmod 700 /etc/openvpn/easy-rsa/keys/MakeOVPN.sh

Create the Default.txt file to hold the default values the script will use. The casing isn’t important, but it must match what was specified at the top of the script file. I’m keeping the capitalized “D” just to keep it the same as anyone else who followed Eric’s instructions.

nano /etc/openvpn/easy-rsa/keys/Default.txt

Paste in the following, substituting your public IP address for the highlighted text. If you don’t have a static public IP address, you can use a dynamic name from a service like DynDNS or no-ip here as well. The “1194” is the standard port number OpenVPN uses, adjust as needed to match your network configuration.

Note: Once again, I’ve highlighted the protocol (udp) and port (1194), which you can change if needed. See the troubleshooting section at the end for more information.

client
dev tun
proto udp
remote YOUR_PUBLIC_IP_ADDRESS 1194
resolv-retry infinite
nobind
persist-key
persist-tun
mute-replay-warnings
ns-cert-type server
key-direction 1 
cipher AES-128-CBC
comp-lzo
verb 1
mute 20

Exit Nano, saving your changes (ctrl-x,y,enter) Execute the script to create a .ovpn file. Remember to use the user or device name you chose earlier when creating the client key.

cd /etc/openvpn/easy-rsa/keys
./MakeOVPN.sh

The result is a NAME.ovpn file in the /etc/openvpn/easy-rsa/keys folder on the Raspberry Pi. That’s great, but we need the key on the client machine. You can copy the file using a secure copy program like WinSCP, copy it to a flash drive and move it by hand, or any other number of ways to move a file around. Since this is my own private home server, I’m going to put the file on the data share, at least temporarily. Once the key is installed and working on the client, I’ll delete it from the server.

cp /etc/openvpn/easy-rsa/keys/NAME.ovpn /mnt/data/

Keys like this aren’t something you should leave lying around. On the other hand, you should probably have a backup of them somewhere. If you put them on a flash drive, go put it in a safe or something. Don’t let anyone get a hold of your keys, or they have a free pass into your home network, and you may not even notice it. You can always go back and generate new keys, delete the compromised ones, and continue on, of course.

Port forwarding

Before you’ll be able to connect to your home network from outside, you’ll need to set up your router to forward all traffic on port 1194 to the Raspberry Pi. I can’t tell you how to configure the firewall on your router at home because I don’t know what kind of router you have. An excellent resource that may have information specifically for your router is http://portforward.com/.

Client configuration

I’m using the OpenVPN client for Windows, but the instructions should be similar for other platforms. You can download open-source clients for Windows, and source tarballs for other systems from here.

Note: Don’t try to download client software from the links on the front page of the OpenVPN site or you’ll just end up with “SecureTunnel”, a paid-subscription-based system that lets you do exactly what you’re already set up to do on your own. Get the .ovpn file that you generated on the Raspberry Pi over to the computer you’re going to connect from, and put it in the OpenVPN config folder. For Windows users, this should be C:\Program Files\OpenVPN\config.

Connecting the client

You’ll need to be somewhere other than on your own network for this next part. Otherwise you’re seriously crossing the streams, shutting down the containment grid, etc. Disconnect from your home network and tether yourself to a phone or something before continuing. Run the OpenVPN GUI application. It should have created a shortcut in your start menu for Windows 7 users, or on your app list for Windows 8 users. Run it, and it should pick up on the .ovpn file and open a connection. Right-click on the notification area icon (for Windows users), and select “Connect”.

You’ll be prompted for the password you created earlier, and if everything is configured correctly, the OpenVPN icon should turn green, and you’ll be effectively connecting to the outside world as part of your home network. There are, of course, many issues you could run into when using a VPN. Most of them are explained pretty well on the HowTo page of the OpenVPN site. One of the more vexing problems is that of disambiguating IP addresses between your home network, and the network you are connected to. See “Numbering Private Subnets” for more information.

Troubleshooting

“I can’t connect from work”
This one got me, too. If you’ve tested your VPN, and you know it works, but it doesn’t work from a specific network, then port 1194 might be blocked on that network. I couldn’t connect from work when I was using udp port 1194, although it would work find when I tethered my computer to my phone. If that’s the case, then you can change the server.conf and .ovpn files to use a different protocol and port. Changing “udp” to “tcp” and “1194” to “443” will usually work from anywhere, as long as you’re not using that port for something else already. TCP port 443 is what https traffic uses, so the odds of it being blocked are pretty slim. Remember to update your router’s port-forwarding rules, update BOTH files, and restart the OpenVPN server (sudo service openvpn restart) as well as your OpenVPN client, and it should connect just fine.

What’s next

In the next post, we’ll add Resilio Sync, allowing you to build your own private synchronization “cloud”.

Important: This post is part of a series. Each post builds on the previous ones. If you are trying to add just one thing to an existing system that was not built by following this series in order then I cannot promise that these instructions will work for you. In most cases they will, though. Please refer to the Introduction for a list of all the different posts in the series.

Self-Promotion: I have recorded this series as a screencast for Pluralsight:
(http://www.pluralsight.com/courses/raspberry-pi-home-server)
If you have a Pluralsight subscription, please consider watching it. Reading the instructions is one thing, but actually watching it being done demystifies the whole process.

Thank you!

Now that the Raspberry Pi is on the network, and sitting on top of a (hopefully) massive pile of your media on its external drive, it would be nice to have it share that media in a form other than simple file shares. File shares are great for a lot of things, but Media can be done a little better.

DLNA (Digital Living Network Alliance… what a horrible acronym) is the technology standard that lets you play songs from your phone and have them come out of the stereo in your living room. It’s not ubiquitous yet, but it’s catching on. It also lets programs like Windows Media Player play music and video “libraries” that are being shared by other computers on the network.

The Raspberry Pi is about to become one of those libraries.

Note: It is also possible for a Raspberry Pi 2 or above to run Plex Media Server, which is a nicer experience overall, but as of the time of this writing, still doesn’t get along with Windows Media Player. WMP will see a Plex server, but can’t seem to play any media off of it. A walkthrough of installing and configuring Plex Media Server is included in my Pluralsight course, but I’m not including it in the blog just yet because of the compatibility issues with WMP, and because MiniDLNA is a smaller, leaner package which I think works a bit better on the Raspberry Pi. I’ll be adding a separate post about Plex Media Server later on, at which time I’ll revise this note.

Installing MiniDLNA

This installation will be done from the command line, and involves a bit more configuration than some of the other features in this series. Get yourself logged in to a command line as “pi”, and start by installing MiniDLNA. The version in the repositories isnt always totally up to date, but it has worked well for me so far.

sudo apt-get install minidlna

As installs go, this one’s pretty quick, but there is a bit of configuration to be done before it’s ready to go. Start by editing the MiniDLNA defaults:

sudo nano /etc/default/minidlna

Find the line that says “#USER=”minidlna””, and remove the pound sign / hash mark from the beginning, so that if just says “USER=”minidlna””. Do the same for the “GROUP=” line right below it. Close and save the file (ctrl-x,y,enter).

Next, edit the main MiniDLNA configuration file.

sudo nano /etc/minidlna.conf

There are several things to be done here. The first is to set up the folders that will hold the various kinds of media you want shared. A little way down the file, find the first line that isn’t a comment. It should say something like “media_dir=/var/lib/minidlna”. On my drive, there are separate folders for Music, Pictures, and Videos, so I’ll add multiple media_dir lines, one for each kind of media. The comments in the file do a pretty good job explaining the syntax, but for reference, here’s what mine says:

media_dir=A,/mnt/data/public/Music
media_dir=P,/mnt/data/public/Pictures
media_dir=V,/mnt/data/public/Videos

Note that the paths are case sensitive, so while my “Public” share is capitalized, the actual folder is not. The folders within my public folder are, however. This is more for demonstration purposes than anything. Feel free to capitalize your folders however you like.

The letters A, P, and V indicate what kind of content each folder contains (audio, pictures, or video). Next, change the database and log folders to go to the data partition as well. This will conserve space on the relatively small OS partition. The lines to configure this are just below the media_dir lines modified above. Remove the pound signs, and change the folder paths:

db_dir=/mnt/data/MiniDLNA
log_dir=/mnt/data/MiniDLNA/log

The result should look something like this:

Further down the file, uncomment the two settings “inotify” and “notify_interval”. This will allow MiniDLNA to update its database approximately every fifteen minutes with new or updated files.

Close and save the file (ctrl-x,y,enter).

If you were to start MiniDLNA and check its logs right now, you would find that it complains that the number of “watches” is too low. This is how many folders and files it can be watching for changes at a time, and currently its default value is too low in its opinion. It will probably work just fine as is, but we can shut the error up easily enough before continuing. Edit the sysctl.conf file.

sudo nano /etc/sysctl.conf

At the bottom of the file, add a line that says

fs.inotify.max_user_watches = 65536

This is the smallest number that MiniDLNA won’t complain about. Close and save the file.

Next, you need to set up MiniDLNA to run automatically at boot time.

sudo update-rc.d minidlna defaults

And finally, reboot the Pi to take both of these changes into account.

sudo reboot

When everything has restarted, you should have a working MiniDLNA installation, with no complaints in the log file.

Warning: Depending on the size of your media collection, it may take the poor little Raspberry Pi quite a while to index everything (mine took almost two days). If you look at your CPU usage, you’ll see that it stays pretty much pegged for as long as it takes. Don’t worry, you haven’t reached the Pi’s limit, at least not permanently. When it has finished indexing, you’ll see its usage drop back to almost nothing.

You should now be able to open something like Windows Media Player, and see an entry for “RPHS: minidlna” in the list on the left. Drilling down into this entry will show you the media that has been indexed so far.

I am not a MiniDLNA expert, by any means, so you might want to look elsewhere for more configuration information. Also, MiniDLNA has recently changed names to become “ReadyMedia”, but the Raspberry Pi version still calls itself “MiniDLNA”, at least for now.

Forcing a re-scan

As commenter Ivelin Tenev points out, MiniDLNA may not notice files that you add when it’s not running. If you have disconnected your data drive from the Pi, loaded files on it, and then re-connected it, MiniDLNA may not notice the new files because it wasn’t “watching” at the time the files were added. You can force a re-scan by typing:

sudo minidlna -R

and then restart the minidlna service like this

sudo service minidlna restart

What’s next?

In the next post, we’ll add an OpenVPN server, allowing you to securely connect to your home network when you’re away.

Note: This post is part of a series. Each post builds on the previous ones. If you are just trying to add one thing to an existing system that was not built following this series, then I cannot promise that these instructions will work for you, although they probably will. If you’ve started from something other than a non-NOOBS Raspbian image, then you’ll probably need to adjust for that.

Please refer to the series Introduction for a list of all the different posts in the series.

Self-Promotion: I have recorded this series as a screencast for Pluralsight:
(http://www.pluralsight.com/courses/raspberry-pi-home-server)
If you have a Pluralsight subscription, please consider watching it. Reading the instructions is one thing, but watching it done demystifies the whole process.

Thank you!

Now that the Raspberry Pi has the space to hold your stuff, it’s about time it started doing something useful around the house. Its first task will be to share files from the hard drive. In the previous post we added a hard drive with two partitions, one ext4 partition for the operating system, and one NTFS partition to hold data. If you decided to format your data partition using the ext4 filesystem, then you’ll need to keep that in mind as we proceed.

Prerequisites

• A Raspberry Pi
• A mounted, USB hard drive with space for the stuff you want to share
  Mine is the second hard drive partition, mounted at /mnt/data

Install Samba

The “Samba” package creates network file shares in a way that Mac, Windows, and other Linux computers on the network will understand. Install Samba as follows:

sudo apt-get install samba samba-common-bin

That’s all there is to the installation, but there’s a bit of configuration before there will be anything to see over the network.

Set up shares

In the previous version of this article, I used Webmin to do the heavy lifting of setting up the shares. In all the time I’ve been running my Raspberry Pi Home Server, I’ve found that this is the only thing I ever really used Webmin for, though. As a result, this version of the series will set up the shares manually, and the Webmin article has been moved to the end as an optional step.

To create a share, you first create a folder that you want to share, tell Samba to share it, and set up the permissions correctly so that users connecting to the share through Samba will be able to write to it.

You can get into all kinds of detail about who can and can’t write what and where, but I’ll leave that up to you to explore on your own. What I’ll be setting up is a simple public share to be used by the family members in my house. The permissions will be wide open on this share.

Create the folder to be shared

First, create a folder to be shared. I’ll call mine “public”, and put it in the root of the hard drive I mounted in a previous post.

mkdir /mnt/data/public

Take a look at the permissions for the newly-created folder

ls -l /mnt/data

You’ll see that the new folder belongs to the “root” user and group even though we created it without using “sudo”. This is because my filesystem is NTFS, and it doesn’t support Linux’s file system permission system. If you made your data partition ext4, then this is going to cause problems for the Samba user. Assign the folder’s ownership to the “nobody” user and the “nogroup” group.

sudo chown nobody:nogroup /mnt/data/public

Check out the result

ls -l /mnt/data

If your filesystem is NTFS, this will have had no effect. If your filesystem is ext4 though, you should see that the public folder is now owned by “nobody” and “nogroup”. Now let’s share this folder with the network.

Open the Samba configuration file using nano

sudo nano /etc/samba/smb.conf

This file is pretty well documented, and will explain most things that you need inline. We’ll be making a number of changes.

Enable WINS support

Windows Internet Name Service (WINS) is kind of like the Domain Name System (DNS) that turns web addresses into IP addresses when you use a browser. WINS does this inside of your own network so that instead of having to remember your server’s IP address, you can refer to it by name.

Scroll down in the file a little until you see the “Global Settings” section. If your home network is using a different workgroup name than “WORKGROUP”, you’ll want to change that here. If you don’t know what that means, then you probably haven’t changed your workgroup name, so you can just leave it alone.

Just below that is a commented-out line that says “wins support = no”. Uncomment this by removing the pound sign from the beginning of the line, and change “no” to “yes”.

Scroll down a little, and just before the Networking section, add a line that says

guest account = nobody

Scroll down to the bottom of the file, and add a new section that defines the new public share.

[public]
   path = /mnt/data/public
   public = yes
   writeable = yes

Close and save the file (Ctrl-X,Y), and then restart the Samba service

sudo service smbd restart

That should do it. You should now be able to see the public share from the other computers on your network. You should also be able to refer to your server by name now instead of by address, and that includes when connecting via RealVNC.

Give it a try now. Open a file manager, and navigate to “\\rphs\public”. You should be able to create and edit files in this share without any restrictions.

Disclaimer

This is a wide-open file share. Anyone on the local network can add, edit, or delete the files it contains. I’m more interested in having a place to put stuff than in controlling access to it. Controlling access to the drive contents is beyond the scope of this post, but may be a subject for a future post.

Make some folders

Create some file folders in the Public share to hold your stuff, and copy it in. Music, Pictures, Videos, anything you want. For this server, I’ve created the following folder structure.

• Documents
• Downloads
• Music
• Pictures
• Videos

You don’t have to use the same structure as me, of course, but this is the one I’ll be using in future posts, so if you structure your differently, just remember to adjust any paths in future posts accordingly.

Load it up

When it comes to copying the files in, you can save some time, and skip the network bottleneck by moving the external hard drive to the computer that currently has the data. Because the data partition was formatted as NTFS in the previous post, when you plug it in to your Windows or Mac computer, you’ll just see a drive called “Data”, or whatever you named it.

Remember to shut down the Raspberry Pi before removing the hard drive, of course. Just put it back when you’re done copying files, and start up the Raspberry Pi as usual.

Wrapping up

You now have a public share that exposes the data partition on the hard drive attached to the Raspberry Pi Home Server. You can use this to store your media and documents on the network so that they are accessible to your other computers and devices.

You’ve finished another article, so it’s time to shut down the Pi, and take a backup of the SD card.

What’s next?

In the next post, we’ll get a media server up and running so that you can stream music and movies all over your house from the hard drive share.

Note: This post is part of a series. Each post builds on the previous ones. If you are just trying to add one thing to an existing system that was not built following this series, then I cannot promise that these instructions will work for you, although they probably will. If you’ve started from something other than a non-NOOBS Raspbian image, then you’ll probably need to adjust for that.

Please refer to the series Introduction for a list of all the different posts in the series.

Self-Promotion: I have recorded this series as a screencast for Pluralsight:
(http://www.pluralsight.com/courses/raspberry-pi-home-server)
If you have a Pluralsight subscription, please consider watching it. Reading the instructions is one thing, but watching it done demystifies the whole process.

Thank you!

At this point in the previous version of this series, I demonstrated both mounting a hard drive and booting from it. After building and rebuilding this server many times as part of maintaining this series, I found that booting from the hard drive this early in the process makes the rest of the series inconvenient because you can no longer take backups of the whole system by simply taking an image of the SD card.

As a result, booting from the hard drive is now a separate post, and one of the last in the series. I found this to be much more user friendly, and simplifies things if you mess something up and want to rewind to your last “save point”. We’ll get everything set up and working first, and then transfer the OS to the hard drive in this version of the series.

Gather your tools

This post uses some software packages that are not included by default in the Raspbian distribution. You’re going to need NTFS filesystem support, and the gdisk tool. Install them before continuing. One or more of these tools may already be installed on your system, but telling apt-get to install something that’s already there won’t hurt anything.

sudo apt-get install ntfs-3g ntfs-config gdisk

Pick a hard drive

Pretty much any external USB hard drive will do. For my own server at home these days, I’m using a 120GB SSD for my OS drive, and a dual-drive 2TB RAID enclosure (Cavalry CADA-SA2) for my data. This might seem like overkill for a Raspberry Pi, but one of my home server’s jobs is to centralize and share my music collection. It took me four solid weeks to rip every CD I own, and I do not plan to repeat that process. My family photos and videos are on the RAID drive as well. Basically, this is all stuff I don’t want to lose due to a hard drive failure.

Note: Yes, I have a separate offline backup too, but swapping out a dead drive in a RAID array is easier (and faster) than rebuilding the whole thing by hand from a backup.

Linux device names

Before we start, lets take a moment to discuss the way in which Linux refers to devices in general, and more specifically the way it refers to hard drives. Windows users are used to seeing multiple drives show up with different letters. You have your “C” drive, where Windows and all of your programs are installed. Maybe you also have a second hard drive called “D”, or maybe “D” is CD or DVD drive on your computer. “A” and “B” used to refer to floppy drives, and are reserved for historical purposes. Each of these devices is completely separate island of stuff.

Linux has no drive letters. Linux uses what’s called a “unified file system”, meaning that files, folders, and even entire drives appear together under one root. You can see all of these different devices in the “/dev” folder. Take a look:

ls /dev

I won’t go into what all of these entries mean, but you can see that there are a lot of them. If I were to plug in an external hard drive, or perhaps a flash drive, I would see new entries pop up in the list. Here’s another screen shot after I’ve plugged in a flash drive.

Look about halfway down the third column, and you’ll see new entries called “sda”, “sda1”, and “sda2”. These represent the flash drive, and the two partitions it contains, respectively. You’ll most likely only see one partition on a typical flash drive, but this one has two partitions for demonstration purposes.

Storage devices like a flash drive or a hard drive can be divided up into multiple sections called partitions that can make it appear to be multiple physical drives. This list of where each partition begins and ends is called the “partition table”, and every drive has one.

That explains the “1” and the “2”, but where did that “a” come from? On a computer where drives are connected via SCSI (Small Computer System Interface), “a” would be the first device, “b” would be the second, etc. Drives connected via IDE (Integrated Drive Electronics), or SATA (Serial AT Attachment) have similar concepts of “first”, “second”, etc.

This is not the case with USB. USB devices have no concept of “first” or “second”. The USB ports on your computer aren’t numbered, at least not externally. You could plug the same device into any USB port and it should work just the same. USB device names are assigned on a first-come, first-served basis. The first drive to be ready gets to be “sda”, the next one “sdb” and so on. If you plug in a third device, you could expect it to be called “sdc”.

If you only have one drive, then it would obviously always be called “sda”, and there’s no problem, but since you’re building a server, it’s conceivable, and even likely that you will eventually end up with more than one hard drive attached to it. Perhaps one drive will house videos, and another one will have music on it.

Even if you don’t see a need for that yet, you’ll end up filling up a drive eventually, and you’ll want to add another one. As soon as you add more USB drives, however, you can’t count on their names being the same every time the machine starts up. As a result, drives may end up mounted in the wrong places, and programs won’t find their files where they expect them to be because the drive that used to be “sda” was a little slower off the line one morning and got stuck with the name “sdb” instead.

Let’s take care of that particular problem before going any further.

Format the hard drive

If the drive you’re planning to use already has stuff on it, then back its contents up somewhere. and restore them when you’re done. This approach calls for nuking and repaving the whole hard drive.

Caution: Even if you wanted to live dangerously and manipulate the partitions in place, I wouldn’t do it without a backup copy first. There are just too many things that could go wrong. What if the power went out partway through the process? Don’t risk it. Just copy your stuff to another drive, and copy it back when you’re done.

The key to fixing the device naming problem requires us to have a specific kind of partition table called a GUID Partition Table. This is different than the older Master Boot Record which is used by most smaller drives or older Windows machines.

In order to create a GPT, you’ll need to use Linux’s Partition Editor (parted) application. Raspbian already includes parted, so there’s nothing to install. Without hooking up the hard drive just yet, run it with the following command:

sudo parted

You should see something almost, but not quite entirely unlike a command line. To see a list of all the known devices and partitions, type “print all”. Parted should print out a table with two rows similar to this:

The rows in the table represent the partitions. The columns will tell you where each partition begins and ends, how large it is, and what kind of filesystem it contains.

If you see more than one table at this point, then you probably have another drive plugged in somewhere. I strongly advise shutting down and removing all drives other than the SD card before continuing.

The SD card in the image above was created using the Raspbian image file, and has two partitions on it, a small “boot” partition, and the root filesystem partition. If you started by using NOOBS, which you really shouldn’t, then you probably have five partitions. Future releases could divide things up differently, so don’t worry if your output doesn’t look like the example.

I’ll say it again here, but I really don’t recommend building a server from a NOOBS image. NOOBS is convenient for experimentation and classroom use, but it complicates a server build without adding anything of value.

Attach the hard drive

Shut down, plug your external drive into a free USB port on the Raspberry Pi, and restart.

Note: You could try plugging in the drive with the Pi running, but unless the drive has its own power supply, this might draw too much power and cause the Pi to restart involuntarily.

If you’re booting to the desktop, then there’s one more thing you need to do. When running the desktop, the Pi will auto-mount devices in a folder called “media” under the root. Take a look at what’s there now.

ls /media/pi

You’ll see different stuff depending on what the drive used to be used for. The point is that the drive is “mounted”, and that’s going to prevent us from formatting it easily. If your drive has more than one partition on it, you can expect to see more than one thing in the /media/pi folder.

Un-mount the devices one at a time like this:

sudo umount /media/pi/NAME

Where “NAME” is the name you saw listed above.

You can also do this from the desktop using the “File Manager” application. Open it by clicking the Menu icon, then “Accessories”, and then “File Manager”. At the top of the left pane should be a drop-down labelled “Directory Tree”. Switch that to “Places”, and any mounted devices should show up at the bottom. Right-click on your devices and then “Unmount”.

Run parted again

sudo parted

List the known partitions again.

print all

This time, you should see two tables like this:

In my example, the second one is the external drive, although the order is not guaranteed here.

Notice the headers above the tables. The header tells you information about the drive in general. In this case, the header tells us that the hard drive I’m using for this demo was assigned the name “/dev/sda”, that it’s 120GB in size, and that its partition table type is “msdos”. That means it’s using the old MBR-style partition table, and that’s not going to work for this server.

Select the external drive by typing “select /dev/sda”. Type “print” all by itself to make sure. You should see the second table from above again. Read the header carefully, and make sure it’s referring to the external drive, because you’re about to blow away the partition table.

Create a new partition table with the command “mklabel gpt”. Read the warning carefully, and make sure it refers to “/dev/sda”.

Answer “y” to the prompt, and in a few seconds, you will have a nice blank slate to work with. Check the results with another “print”. The header should now say that the partition table type is “gpt”, and there should be no partitions on it.

Create new partitions

Next, you’ll need to create two new partitions; one to hold the root filesystem, which I will cover later on in the series, and one to hold your data. You could make one big partition, combining the root filesystem and data storage in one, but there are some benefits to keeping the data partition separate. The root filesystem must be formatted using Linux’s native filesystem, ext4. The data partition, however, can be any type you want, which means you can pick something that other, non-Linux computers will understand.

Root Partition

This is where Linux and your programs will go later on in the series. You can think of it as Windows’ C: drive. The size is up to you, but 16GB is probably fine for the simple server described in this series. If your existing SD card is larger, or you are planning to install a lot of programs, go bigger. Just make sure the partition is at least as large as the SD card you are currently using.

Create a new, partition starting at the beginning of the drive:

mkpart os 0gb 16gb

Substitute your own value for 16gb so that it matches or exceeds the size of your SD card. I recommend going slightly larger than your SD card, myself, just to be sure everything fits. I typically build on an 8GB card, but create a 16GB partition on the drive.

Data Partition

This is where you’ll put your “stuff”, and it should take up the rest of the available drive space. The mkpart command can interpret the beginning and ending parameters in a variety of different formats. Use this ability to create a second partition that begins where the first partition ends, and takes up all the rest of the space on the drive:

mkpart data 16GB 100%

Finally, use the “print” command again to see the results.

That’s it for the partitions, so type “q” and press enter to exit parted. You can ignore the warning about updating /etc/fstab for now. We’ll get to that soon enough. You now have two partitions, but they’re completely blank at this point.

Format the data partition

Before you can store anything on a partition, you need to format it. Choosing NTFS as the filesystem will allow you to simply plug the drive into any Windows, MAC, or Linux desktop computer and manipulate the drive contents. This can be especially helpful for loading up large amounts of data, such as the family’s music and photo collection, without having to squeeze it all through the network connection. If your main computer is a Mac or another Linux machine, then this may not matter as much to you since those systems understand ext4.

Format the second partition as follows, filling in whatever name you want the partition to have. I’ve called mine “Data”.

sudo mkfs.ntfs -Q -L Data /dev/sda2

Note: Don’t forget the –Q (Quick) or you’ll be waiting around a while.

At this point, I would shut down the Pi, attach the hard drive to your regular computer, and make sure you can read the partition. In some recent versions of Windows, partitions formatted as NTFS on a Linux system may fail to mount. This has happened to me on some drives and not others, so make sure you check yours before moving forward. If the drive doesn’t show up in Windows, start up Disk Management (Windows key, type “disk”, and click on “Create and format hard disk partitions”). Find the data partition you created above, right-click on it, and let Windows format it itself. This will ensure you have the best compatibility moving forward.

Uniquely identifying a filesystem

Warning: Because the next section involves copying around large strings of unmemorizable data, I recommend performing the steps through a remote SSH window, or from a terminal window on the desktop either directly or via VNC. The Ids you’ll need to copy need to be copied verbatim, letter for letter, or you’ll find yourself unable to boot, and you’ll have to start over again from your last backup.

As I mentioned earlier, just because the data partition is called /dev/sda2 right now doesn’t mean that it will have that name in the future if you add more drives to the system. What we need is a way to uniquely refer to this drive no matter what letter it get assigned on any given day.

This was why we built the drive with a GUID Partition Table. Each partition on a GPT device is assigned a universally-unique identifier (UUID).

Note: The difference between GUID and UUID is not important here, they are, for all intents and purposes, the same thing; a very long, randomly-assigned number.

Instead of identifying the filesystem’s partition by location, we want to refer to it by Id so that it can be found no matter what order the drives were discovered in. Use the “blkid” comman to find out more about /dev/sda2 (the data partition):

sudo blkid

You should see a listing of information about all of your drives and their partitions, like this:

The piece of information we’re interested is the “UUID” of sda2. In my case that’s on the next to last line of the above screenshot, not the “PARTUUID” on the last line. If you partitioned your drive differently, for instance putting the big partition first, then you’ll need to adjust accordingly. You can double check this using parted and the “print all” command again to be sure.

Note: PARTUUID uniquely identifies the partition, and will come into play later on when we boot from the hard drive. “UUID” identifies the filesystem on that partition, and it’s what we need for mounting.

Take a picture, write it down very carefully, or select and copy the text if you are doing this through a remote terminal connection like I am. If you’re on the desktop directly or through VNC, you could also open up the built-in desktop text editor, and paste the UUID there. You can find it at Menu -> Accessories -> Text Editor.

Mounting filesystems

Linux has assigned the name /dev/sda2 to our data partition, but you can’t just use that name to refer to it. In the Linux world, you first need to tell the OS where that filesystem belongs in the grand scheme of things. To do this, you’ll need to edit the filesystem table. This file controls what gets mounted where, and in what order. Take a look at the current contents.

cat /etc/fstab

The second line is the boot partition, where things like config.txt and cmdline.txt live. The third line is the root filesystem, which you can tell by the “/” in the second column. At this point, it’s loading from the 2nd partition on the SD card (mmcblk0p2). We’ll come back to this in a later post and change it to the OS partition on the hard drive.

Edit the fstab file:

sudo nano /etc/fstab

Add a new line after /dev/mmcblk0p2, but before the comments. The first column identifies the partition. You’ll need to use the format “/dev/disk/by-uuid/” and then the UUID you copied down above. Press TAB, and the path to the location in the filesystem where you would like the partition to mount. I recommend /mnt/data. Tab again, and specify the filesystem type. If you’re using NTFS, just put the letters “ntfs”, if you’re using ext4, type that. Tab again and type the word “defaults”. Tab again and type a zero, then another tab and another zero.

The end result should look like this:

NOTE: I’ve replaced the tabs with spaces to fit it on one screen, either one works just as well. You can pretty up the columns so they line up if you like.

Exit Nano, saving your changes (ctrl-x,y,enter)

The next time the system reboots, it will try to mount the hard drive at /mnt/data. Before that can succeed, you need to create a folder in that location. You can think of it as a placeholder where the actual drive will appear later on.

sudo mkdir /mnt/data

To prove that the drive mounts successfully, create a file in the new folder using the “touch” command.

sudo touch /mnt/data/placeholder

Look at the folder’s contents

ls /mnt/data

You should see a file called “placeholder”. If you ever look in this folder and see that file, it should act as a warning that the drive has not mounted properly.

That’s it. you’re ready to reboot the Pi, and start using your new data drive.

sudo reboot

When the system has rebooted, look at the contents of /mnt/data again.

ls /mnt/data

Instead of the placeholder file, you should see an empty drive. If you formatted the drive as ext4, you’ll see a lost+found folder, but nothing else. If you see the placeholder file, then something has gone wrong. Verify your changes to fstab, and see if you skipped a step, or used the wrong UUID above.

Wrapping up

You now have an external drive to store your stuff in, and it should mount in a predictable place every time the server reboots. The next few posts will start taking advantage of this new storage space.

Shut down the Pi, and take a backup of the SD card.

What’s next?

In the next post, we’ll put the new drive to work sharing files over the local network.

Note: This post is part of a series. Each post builds on the previous ones. If you are just trying to add one thing to an existing system that was not built following this series, then I cannot promise that these instructions will work for you, although they probably will. If you’ve started from something other than a non-NOOBS Raspbian image, then you’ll probably need to adjust for that.

Please refer to the series Introduction for a list of all the different posts in the series.

Self-Promotion: I have recorded this series as a screencast for Pluralsight:
(http://www.pluralsight.com/courses/raspberry-pi-home-server)
If you have a Pluralsight subscription, please consider watching it. Reading the instructions is one thing, but watching it done demystifies the whole process.

Thank you!

So far in this series, we’ve set up the Raspberry Pi for remote login via SSH. Sometimes, though, you just want to see a desktop, and that’s not easy when the computer you’re trying to see has no monitor, keyboard, or mouse.

Adding remote desktop abilities to the Raspberry Pi is now trivially easy. Since the introduction of the Pixel desktop, Raspbian images have had a RealVNC server built right in. All you have to do is enable it.

If, for some reason, you don’t like RealVNC, or don’t want to use the latest version of Raspbian, you can still go back and read my instructions for installing X11VNC in the previous version of this post.

Enabling RealVNC

First things first. You can’t remote in to the desktop if it’s not running. For servers, I usually like to leave the desktop disabled, and set the computer to boot to the command line (CLI). This frees up resources for the other programs that I’m running. With the Pi 2 and 3, this becomes less of an issue, and the desktop environment really isn’t that much of a drag. The choice is yours, but I’m going to leave my Pi booting to the desktop for the moment, but I’ll walk you through two different ways to enable the RealVNC server.

Method 1 (if you’re booting to the command-line)

If your Pi is set to boot straight to the desktop, then you can skip this section and move on to Method 2.

If the Pi is still hooked up to a monitor and keyboard, you can do this directly from the Pi itself. Mine is not, so I’ll be connecting via SSH using PuTTY. Either way, log in, get to a command line, and run raspi-config.

Note: With the November 2016 release of Raspbian, SSH is disabled by default. If you want to log in remotely via SSH, make sure you run raspi-config and enable it before removing the monitor and keyboard,

sudo raspi-config

Go to “Advanced Options”, and then select the “VNC” option.

At the prompt, select “Yes” to enable the VNC server. You’ll get an acknowledgement that the server was enabled, which you can dismiss, and then select “Finish” to exit raspi-config.

If you’re reading this section, then I assume your pi boots to the command line, so there’s no desktop to see right now. You need to get the desktop running before there will be anything to look at. Interestingly, the RealVNC server’s background process (or daemon in *nix terms) is running even though the desktop isn’t. If I try to connect the VNC viewer now, it’ll connect just fine, but I’ll see nothing but black.

To start up the desktop, I’ll need to run it manually from the command line like this:

sudo startx

This will start up the desktop, and suspend the command-line window until I shut down the desktop. It should look something like this:

Don’t worry about that “FATAL” message at the bottom. It says that to everyone.

At this point, the desktop should be up and running, and you can skip ahead to the “Connecting a Viewer” section.

Method 2 (if you’re booting to the desktop)

If you’ve left the Pi booting to the desktop, the process is pretty much the same except that we’ll be using the desktop version of raspi-config.

Note: You could still open a terminal window and run the command-line raspi-config as well, if you like.

The RealVNC server can be turned on from the desktop by clicking on the Menu button, then “Preferences”, and then “Raspberry Pi Configuration”.

Go to the “Interfaces” tab, and select “Enable” on the row labelled “VNC”.

Click OK to exit the configuration tool, and that’s it. Your VNC server is ready to go. Notice that my screenshots above have a “VNC” logo in the upper-right. I’m using VNC to get my screenshots at this point because it’s just easier that way.

Connecting a Viewer

Now that the Pi is set up to share its screen, and the desktop is running, you’ll need a “viewer” in order to see it. Since the Pi is running a RealVNC viewer, the most painless way to connect is with a RealVNC viewer application, which you can download here. Download the version appropriate for your regular computer.

I can’t make observations about all of the platforms the viewer is available for, but I’m a Windows user, and the Windows version of the RealVNC viewer is a portable application that requires no installation. You just download it, run it, and tell it what to connect to.

The first time you run the VNC viewer, you’ll need to give it the address of the Pi. You’ll then be prompted to provide a name and password.

Use the same values you would to log in directly on the Pi, and a new viewer window should open up showing yet another login prompt. This time, it’s the same one you’d see if you were looking at a monitor connected directly to the Pi. It’s a little weird logging in twice, but if you check off “Remember password” in the “Authentication” dialog above, you can skip over the first of these two prompts in the future.

You can resize this window, or tell it to go full-screen. If you move it to a second monitor and go full-screen, you can hardly tell the difference between the VNC window and the real thing.

There’s a slight hiccup, though. If the Raspberry Pi wasn’t plugged into a monitor when it booted up, it will assume that since there’s nothing on the other end of the HDMI connection, you must be using the composite video output, and it will go into a low resolution mode that looks like this:

Cute, but not very useful. Fortunately, we can simply tell the Raspberry Pi what resolution to use, and it won’t try to negotiate the best resolution with the monitor anymore. Edit the main configuration file like this:

sudo nano /boot/config.txt

Look for the line that says “hdmi_force_hotplug=1”, and uncomment it by removing the pound sign from the beginning of the line. Do the same for the “hdmi_group” and “hdmi_mode” lines. You’ll need to provide values for them, though. You can find a pretty exhaustive list on the eLinux site. Pick something that fits comfortable on your primary computer’s monitor. I’m using group 1, mode 4, which is 720p. It gives enough space to read, but fits on my monitor without taking over the whole screen.

Just to prove the point, I made these edits while connected to the too-small virtual desktop, and you can see from this screenshot that the default resolution is waaaay too small to be very useful.

Save your changes and exit nano by pressing Ctrl-X-Y.

Shut down the server, and if you haven’t done so already, unplug it from the monitor. Restart the server by reconnecting the power. Now, when you connect via VNC, the virtual screen will be a much more reasonable size.

When you’re done, you can log out of the Pi by clicking on the Menu button, then “Shutdown”. Depending on whether you boot to the desktop or the command line, the dialog you get will be slightly different. For those that boot to the desktop, the last option will say “Logout” like this:

Clicking on “Logout” will log you out of the Pi, but won’t close your connection to it. You should see the login prompt again. At this point, it’s safe to just close the VNC viewer window.

If you have your Pi configured to boot to the command line, and had to manually start up the desktop, then the last option on the Shutdown dialog will say “Exit to Command Line” instead.

Clicking on “Exit to command line” may not look like anything has happened because the VNC viewer window will just keep showing that last image of the desktop it ever got, but if you try to interact with anything on the desktop now, you’ll find that it’s no longer listening. You’re can just go ahead and close the VNC viewer window. You’ll notice that your command prompt in your SSH windows has come back now, though.

You can continue using the command line via SSH, or disconnect from there as usual.

Wrapping up

You’ve reached another milestone, and I strongly recommend politely shutting down the Raspberry Pi (sudo shutdown –h now), and taking another backup of the SD card.

What’s next?

In the next post, we’ll add a hard drive, so we have a place to keep stuff.

Note: This post is part of a series. Each post builds on the previous ones. If you are just trying to add one thing to an existing system that was not built following this series, then I cannot promise that these instructions will work for you, although they probably will. If you’ve started from something other than a non-NOOBS Raspbian image, then you’ll probably need to adjust for that.

Please refer to the series Introduction for a list of all the different posts in the series.

Self-Promotion: I have recorded this series as a screencast for Pluralsight:
(http://www.pluralsight.com/courses/raspberry-pi-home-server)
If you have a Pluralsight subscription, please consider watching it. Reading the instructions is one thing, but watching it done demystifies the whole process.

Thank you!

Now that you’ve turned your Raspberry Pi into a working computer, it’s time to get connected.

Prerequisites

• A Raspberry Pi with the Raspbian Linux distribution installed as described in the “Installing the OS” post earlier in this series.
• You should be logged in as the “pi” user account.
• You need to be at a command prompt. If you set your Pi up to boot to the command line, then you’re already there. If you set it up to boot to the desktop, then you’ll need to open the Terminal program.

Assigning a static IP address

Since this series is about turning the Raspberry Pi into a server, it’s important that it have a consistent IP address so that you can connect to it, and so that you can tell your router where to redirect traffic for things like web traffic or other services. There are two ways to accomplish this.

Method 1: Let the Router Do it

You can use your router’s configuration to permanently “lease” an IP address to a specific device. In other words, you tell your router to always hand out the same IP address to the same device when asked. There are definite advantages to letting your router handle the IP assignments. It centralizes the management of network resources in one place, rather than distributing that configuration across the various devices on the network. Also, there are certain Universal Plug and Play (UPnP) functions that work better if the router is in charge of handling out the IP addresses.

This is the method I use, personally. Instructions for how to configure an individual router vary by manufacturer and model, so I won’t cover them in detail here, but the gist of it is that you associate an IP address with the hardware MAC address of the network card built into the Raspberry Pi.

To get the MAC address, type the following at the command prompt:

ifconfig

The result should look something like this:

The MAC address appears on the first line, next to the label “HWaddr”. Follow the directions for your specific router, and assign a permanent IP lease to this MAC address. Make sure it’s not in the range that the DHCP server uses as it’s “pool”. For my router, everything from 100-255 is the DHCP pool, so anything below 100 is fair game for leasing or static assignment.

Method 2: Let the Pi Do it

The second method is to configure the IP address directly on the Raspberry Pi itself. Here is where you will first start to see the difference in attitude from the Windows world to the Linux world. Windows is about convenience and ease of use for the average non-technical user. Linux is all about highly-knowledgeable users tweaking things exactly the way they want them. Linux settings are stored in plain text files in ordinary folders. The trick is mostly in knowing which settings go in which file, and where it lives.

The Jessie release of Raspbian changed how static IP addresses are configured, so this part is a bit different than in the previous version of this series.

Just like with the router’s DHCP lease-based example above, you’ll need some information first. Type the following at the command prompt.

route -n

You’ll see something like this:

Write down the Gateway address. It’s on the line with the “Flags” value of “UG”. This is usually the address of your home network router. In my case it’s 192.168.1.1

Next, edit the “dhcpcd” file. If you have been building Raspberry Pi servers for a while, you’ll notice that this process is different than it was under the older “Wheezy” release of Raspbian.

sudo nano /etc/dhcpcd.conf

“nano” is a text editor that is included in Raspbian, and is convenient for making configuration changes. You need “sudo” to run it with elevated privileges because regular users aren’t allowed to edit system configuration files, and while it’s certainly above average, the “pi” user is just a regular user at the end of the day.

The dhcp configuration file will open in the editor, and should look like this.

Scroll all the way to the bottom of the file and insert the following lines, substituting the values from above, and the IP address you want the Pi to have where appropriate. For the last value (domain_name_servers), you have a few different choices.

For Google, use the value “8.8.8.8 8.8.4.4”
(Note the space between the first and second addresses)

For OpenDNS, use “208.67.222.222 208.67.220.220”
(again, with a space between the addresses)

If you don’t want to use either of those, you can use whatever your ISP provides you. You can find these addresses in the admin pages of your router, or you can just use the command “ipconfig /all” from a windows machine on your network to retrieve it.

interface eth0
static ip_address={DESIRED IP ADDRESS}/24
static routers={GATEWAY ADDRESS}
static domain_name_servers={DNS ADDRESSES}

Hold down Ctrl, and press “x” to exit nano. It will prompt you to save your changes, press “y” to save them, and accept the default filename, which is the same as the file you loaded to begin with.

Before these changes will take effect, you’ll need to reboot the Raspberry Pi. Type the following and press enter to restart the server.

sudo reboot

Note: If you had just typed “reboot” without the “sudo” you would have been denied. Regular users are not allowed to reboot a Linux machine. Remember that, like Unix, Linux can have multiple users logged in at the same time, and letting one of them reboot the machine when the others might be in the middle of something would be… rude.

Once the machine has rebooted, type “ifconfig” again to see the new configuration. You should see the IP address you assigned listed as the first item on the second row. In this example, the Pi has the IP address of 192.168.1.9.

Going headless

Now that the Raspberry Pi Home Server is located at a known address, you could disconnect the Raspberry Pi from its monitor and keyboard and do everything remotely. Actually, you could have done this a few steps ago, but it’s easier to connect when you know the IP address will be the same each time.

Find an SSH client program that you like, and use it to connect to the raspberry Pi. The particulars of configuring a connection will vary by program, but in Putty, all I need to do is provide the IP address of the computer I want to connect to and click “Open”.

At this point, you could disconnect the Raspberry Pi from your monitor and keyboard, hide it in a closet somewhere, and do the rest of your configuration remotely. I’d recommend leaving it hooked up, though, at least for now. You’ll be setting up a remote desktop a little later on, and it’s nice to have the visual feedback on the monitor to let you know things are working.

I’ll be performing the remaining configuration steps in this series remotely. It makes for better screenshots and is how I intend to interact with the server from now on anyway.

Wrapping up

You’ve reached another milestone, and I strongly recommend politely shutting down the Raspberry Pi (sudo shutdown –h now), and taking another backup image of the SD card.

What’s next?

In the next post, we’ll add remote desktop capabilities.

Note: This post is part of a series. Each post builds on the previous ones. If you are just trying to add one thing to an existing system that was not built following this series, then I cannot promise that these instructions will work for you, although they probably will. If you’ve started from something other than a non-NOOBS Raspbian image, then you’ll probably need to adjust for that.

Please refer to the series Introduction for a list of all the different posts in the series.

Self-Promotion: I have recorded this series as a screencast for Pluralsight:
(http://www.pluralsight.com/courses/raspberry-pi-home-server)
If you have a Pluralsight subscription, please consider watching it. Reading the instructions is one thing, but watching it done demystifies the whole process.

Thank you!

Raspbian comes with a lot of applications pre-installed that make it ideal for its intended educational purpose, but are not needed for a server. This isn’t so much about space. All told, these packages don’t really take up that much room, but they do clog up the update/upgrade pipeline. You can just leave this stuff there, but eventually a large package like Wolfram or LibreOffice will have an update, and you’ll get stuck waiting longer than you’d like for an “apt-get upgrade” to complete. We can remove some of the items which will never be used on the server by using apt-get to remove the unneeded packages.

Note: You can pick and choose from the following items to remove one at a time, or you can cram them all together in one line. Skip ahead to the “everything” section near the end for an example.

Wolfram / Mathematica

sudo apt-get purge wolfram-engine

This one might take a little while, so be patient.

LibreOffice

LibreOffice actually consists of several different packages; Base, Calc, Draw, Impress, Math, and Writer. Fortunately, apt-get supports wildcards. Remove all of LibreOffice like this.

sudo apt-get purge libreoffice*

Minecraft

sudo apt-get purge minecraft-pi

Programming tools

If you’re not planning to use them, sonic-pi and scratch can go as well.

sudo apt-get purge sonic-pi scratch

Everything

If you want to get rid of all of these packages at once, all you need to do is add them one after another onto the end of the remove command. The following example is all one line, although it’s probably going to wrap on-screen.

sudo apt-get purge wolfram-engine libreoffice* minecraft-pi sonic-pi scratch

Cleaning up

Now that we’ve uninstalled unneeded packages, we can tell apt to remove the downloaded packages. I know it seems like the same thing, but it’s not. The downloaded packages are basically the installers. Not only don’t we want these packages, but we don’t want to keep their installers around, taking up space either.

sudo apt-get clean
sudo apt-get autoremove

Clean gets rid of the extra top-level packages, the ones you actually ask for when you say “apt get install foo”, and autoremove gets rid of their dependencies which are the supporting packages that you didn’t explicitly ask for, but were needed to support something that you did install.

Fixing the Menu

Now that some of these programs are gone, we’re left with “dead” icons in the menu bar, and within the menu itself. We’ll need to clean them up. Fortunately this is a lot easier than it was in the past. The Pixel desktop release of Raspbian seems to have learned to self-prune dead icons, so all you have to do is

sudo reboot

That’s certainly a lot better than the last time I tried this.

Note: This post is part of a series. Each post builds on the previous ones. If you are just trying to add one thing to an existing system that was not built following this series, then I cannot promise that these instructions will work for you, although they probably will. If you’ve started from something other than a non-NOOBS Raspbian image, then you’ll probably need to adjust for that.

Please refer to the series Introduction for a list of all the different posts in the series.

Self-Promotion: I have recorded this series as a screencast for Pluralsight:
(http://www.pluralsight.com/courses/raspberry-pi-home-server)
If you have a Pluralsight subscription, please consider watching it. Reading the instructions is one thing, but watching it done demystifies the whole process.

Thank you!

Getting Started

The first thing any computer needs is an operating system. There are several Linux distributions (distros) to choose from. You can think of a distribution as an “Edition” of Linux. Just like Windows comes in Home, Business, and Enterprise editions (or whatever they’re called this week), each Linux distribution consists of the same core OS “kernel”, but with different combinations of additional features added. In the Linux world, these “features” could include things like the entire windowing system, so the differences between distros are a little more drastic than Windows editions, but you get the point.

In this series we’ll be building on top of Raspbian, which is a Raspberry Pi variation of the Debian Linux distribution. It is available from the Raspberry Pi Foundation’s download page as an image file.

Note: At this point, you can follow the directions at the Raspberry Pi Foundation’s own help pages. The tutorial there is very complete, and has videos to explain the process in greater detail than I’ll be going into here. In the interest of being complete, I’m going to summarize the instructions here as part of the series.

Prerequisites

• A Raspberry Pi, power supply, etc.
• An HDMI or composite monitor, keyboard, and a mouse (optional)
• A wired network connection
• An SD card of at least 8GB. If you plan to boot from a hard drive, as covered later in the series, then you don’t need anything bigger than this.

Create the SD card

Go to the Raspberry Pi Foundation’s download page, and get the current Raspbian image. You can download this directly, or you can save the foundation some bandwidth costs and download via BitTorrent.

Get a copy of Win32DiskImager or HDD Raw Copy, and use that to write the image file to your SD card. These days I prefer HDD Raw Copy because it can create compressed backups of an SD card in one step.

More detailed instructions for multiple operating systems are available from the Raspberry Pi Foundation’s site. When you’re done, eject or “safely remove” the SD card from your computer.

First Boot

Hook the Raspberry Pi up to your monitor, network, keyboard, and mouse. Insert the SD card you prepared above. Finally, plug the power supply into the Raspberry Pi. You’ll see a screen with some Raspberries in the top-left corner, and some text beneath them.

Pretty quickly, the whole screen will be replaced by a splash screen like this.

The splash screen is new as of the 2016/09/23 release, and gives us some useful information about the version of the OS in the lower-right corner, as well as a kind of one-at-a-time series of status messages in the lower left. After a while, this screen will be replaced with the Desktop.

Your wallpaper may be different than mine depending on the release you’ve installed. Also, if you don’t like the image, you can always change it. Use the “Appearance” application under Start -> Preferences -> Appearance Settings. You could right-click the desktop and choose preferences from there, but that’s actually a separate, older program, and your wallpaper won’t carry over to the login screen. Using the Appearance application will keep everything in sync. If you want your old Raspberry logo wallpaper back, you can find it at /usr/share/raspberrypi-artwork/raspberry-pi-logo-small.png

Update All the Things

Before you start getting moved in, you should update the software that’s on the Pi so far. Depending on how long it’s been since your image file was created, many of the pre-installed programs may have already released updates and/or bug fixes. Lets get them all up to date. This is easily done from the command line.

We’re going to use APT to update the installed software. APT stands for “Advanced Packaging Tool”. APT is actually a suite of different tools, but the one we’ll be concerned with the most is apt-get, and its job is to install and maintain software packages along with any other packages that they depend on. Throughout this series, we’ll use apt-get to install nearly everything we need.

Open a command prompt by clicking on the Menu button in the top-left corner of the screen, then hover over “Accessories”, and then click on “Terminal”. You should see a black window with a prompt in it like this:

Note: The Terminal program is also available from the toolbar at the top of the screen. In my screenshot above, it’s the fourth icon from the left, including the menu button itself.

The first apt-get command we’ll use tells apt-get to update its list of available software packages by downloading fresh lists of packages from a variety of sources over the internet. It’s important to do this first so that apt-get has an accurate picture of what the most recent version of each package is. Even if your OS image was made fresh yesterday, the software packages included in it are being constantly and independently updated on their own schedules.

At the prompt, type the following:

sudo apt-get update

Wait for this command to complete. Depending on the speed of your Pi, and the speed of your internet connection, this might take a few minutes. When you’re done, it should look something like this:

The next command, “upgrade” tells apt-get to take the version information it just retrieved using the update command, and figure out what changes to make in order to bring everything up to date. Even if the image you just burned was released this morning, many of the packages it contains may have been updated since the image was locked down for release.

Note: This is actually an ideal time to take a side-trip to the supplemental post “Trimming the Fat”. You may want to go read and complete that task before continuing in order to prevent upgrading a lot of packages you won’t even end up using.

sudo apt-get upgrade -y

The “-y” tells apt-get not to stop and ask before continuing. Without the “-y”, apt-get would list out the proposed changes, and ask you to confirm them before actually making any changes. Again, depending on how long it has been since your OS image was created, this step might take a while. My upgrade took quite a long time. Be patient, it’s worth it.

Some of these changes will require a reboot before they become effective, so we’ll do that now. You can do this from the desktop menu, but since we’re already at a command prompt, let’s just do it from there.

sudo reboot

Once the system has rebooted, it’s time to start configuring and customizing the Pi. Click the Menu button in the top left, and select Preferences -> Raspberry Pi Configuration.

This will open a modern, graphical, and more convenient version of the command-line “raspi-config” program that we used in the past. This program allows you to make changes to the configuration of the OS itself, and certain hardware settings. We’ll use it to start “moving in” to the fresh OS.

Expand Filesystem

OS Image files are typically shrunken down to their bare minimum in order to take up less room, so if you put a 4GB image on a 16GB SD card, you’ll need to expand the file system to get the other 12GB of space on the card back. As of the 2016-09-23 release, this resizing should be done automatically the first time you boot the system, but we’ll do it just to be sure, and also because this behavior could possibly change in later releases.

Click on the “Expand Filesystem” button, and you should see a confirmation dialog like this.

Click OK to dismiss the dialog. We won’t be rebooting the server just yet, though. We’ll finish the initial configuration first, and then reboot when we’re done.

Before we set the rest of the system options, and especially before setting a password, let’s make sure the localization settings are right, otherwise you may have trouble with the password, especially if your password contains special characters. Certain key combinations on the default British English keyboard may not match your preferred settings, and you might not be able to get in if you don’t do this first. For instance, if I hit the backslash key on my keyboard right now, it will type the # character instead. You can see how this would be a problem if I wanted a backslash in my password.

Localization Options

The last tab is for setting various language and keyboard options. Click the “Locale” button, and change the drop-downs in the resulting dialog to match where you live. I live in the United States, so my settings look like this:

Next, set your time zone. The Pi doesn’t have a battery-backed clock onboard, so it needs to sync up with an internet time server every time it boots up. It needs to know its location in order to get the offset correct.

Next, you’ll need to tell the Pi what kind of keyboard layout you’re using. If you don’t set this up correctly, you’ll find that some of the keys on your keyboard won’t work the way you expect. I’m using a US layout keyboard, so my settings look like this:

Type your intended password in the box, just to make sure it comes out the way you intended, and then click OK to dismiss the Keyboard Layout dialog.

There’s one last option on the Localization tab, and that’s “Wi-Fi Country”. This was added after the release of the Raspberry Pi 3, which has on-board Wi-Fi. If you plan to use the on-board Wi-Fi, then you’ll need to set up a country for that as well since the standards vary slightly from country to country. I don’t recommend running a server over Wi-Fi if you can help it, but the option is there if you need it.

Now head back to the “System” tab, and we’ll pick up where we left off.

Change password

By default, the Raspbian OS defines a user named “pi”, which we’ll be using to do all of the configuration and setup in this series. The pi user’s default password is “raspberry”, but since “pi” is an all-powerful administrative user, you’ll want it to have a different password than everyone else’s, right? Click the “Change Password” button, and enter the default password once and your new password twice in the dialog.

Important: If you’re planning to use any “special characters” in your password, make sure you’re gotten the keyboard localization squared away before changing your password. In fact, open up the text editor (Menu -> Accessories -> Text Editor) and try typing your password there first, just to make sure it comes out the way you intended. Not all the keys map to the same symbols on the different keyboard mappings.

Note: You may or may not be prompted for the current password, depending on the specific image you burned. In this screenshot, I was not prompted for the existing password.

Other “System” options.

There are several other options that can be set up on the “System” tab of the configuration dialog. Let’s walk through them one at a time.

By default, every Raspberry Pi is named “raspberrypi”. Since you may end up with multiple Pis on your home network, you will need a way to tell them apart. Fill in a new name for your server in the “Hostname” box. I call my own server “RPHS”. You are, of course, free to call yours whatever you want, but you’ll need to adjust configuration settings accordingly later on in this series.

The Raspberry Pi Home Server will spend most of its life disconnected from any kind of monitor, so there’s no need to boot to the desktop every time the machine starts up. A Raspberry Pi 2 or 3 has plenty of horsepower though, and booting to the desktop makes it much easier to “remote into” the server later on for maintenance, so I’m going to leave my Pi booting to the desktop for now. You can always change this setting later on if you like. Just remember that when future articles tell you to type a command, you’ll need to do that from the Terminal.

You can continue to let the Pi automatically log in as the “pi” user if you want, but since this is a server you should probably disable this option. There usually won’t be anyone using the Pi directly anyway. Uncheck the “Login as user ‘pi'” option to tell the Pi to prompt for a name and password rather than just logging in automatically.

Out of the box, the Pi will try to boot as fast as possible, without waiting for the network connection to be complete. A server isn’t much use without a network, so it’s probably better to tell the Pi to wait for the network connection to come up before continuing. Check the “Wait for network” checkbox to tell the Pi to wait until it has established a network connection before performing most of its boot-up routine.

The Underscan setting won’t affect anything once the Pi is no longer attached to a monitor, but I have disabled it in order to get better looking screenshots. If the image on your monitor has black bars all the way around it, then you might want to try toggling this setting and see if you like the resulting image better. If not, you can always set it back again.

That’s everything you need on the “System” tab. The end result should look something like this:

Performance Options

There are only two settings on the Performance tab. The “Overclock” setting lets you squeeze a little more speed out of your system.

CAUTION: The ability to overclock varies from board to board, and is affected by your power supply as well. If you are using some old leftover cell-phone charger you happen to have lying around in a drawer, this may not work for you. If you’ve invested extra money in a high-quality power supply, you’ll probably have better luck. Until you know how your particular combination of components will behave, it’s probably safest to leave it running at its default. You can always come back and change it later on, once you have everything stabilized.

My particular board and power supply have proven themselves to be a reliable pair, so I’m going to turn it up to the maximum of 1000 MHz. I’ve also turned the “GPU Memory” down to 32 since I don’t plan to use the graphical desktop very often. In fact, after this initial configuration, the only part of this course that will require the desktop is CrashPlan. I could probably go even lower than 32, but this setting has worked well for me.

Note: If you’re using a Pi 3 or Zero, then the overclocking option will be grayed out and unavailable.

Interfaces

Starting with the November 2016 release of Raspbian, SSH has been disabled by default in an effort to tighten security. If you’re intending to go headless and connect remotely, you might want to turn SSH back on. If you think you’re only going to use the full remote desktop, then you can leave SSH turned off. It’s up to you, but I always enable SSH and give the Pi user a new password.

If you want to be able to use the remote desktop functionality, then make sure you enable VNC. We’ll come back to this in a later post.

Finished

Click the “OK” button on the Configuration dialog, and you’ll be prompted to reboot the system:

Click “Yes” to reboot the Raspberry Pi and apply the new settings.

Second Boot

When the Raspberry Pi reboots this time, it will be ready to use. It doesn’t have a lot of programs installed so far, but it’s a starting point. If you disabled “Logon as user ‘pi'” like you should have, then you’ll need to type the user’s password when prompted.

If you disabled booting to the desktop, then you should see a login prompt similar to this:

RPHS login:

Log in using the user name “pi”, and the password you set up earlier, or “raspberry” if you skipped that step… which you shouldn’t have.

A Quick Primer on “sudo”

In a Linux system, the main administrative user is called “root”, and root can do pretty much anything it wants, no questions asked. The “pi” user is a specially-privileged account, capable of doing things an ordinary user account can’t, but it isn’t as all-powerful as root.

You’ll see the command “sudo” used a lot in this series. It stands for “substitute user do”, and lets you run other commands as if you were a different user. If you don’t specify a particular user to impersonate, the “root” user is assumed. Only user accounts that have been specifically allowed can perform actions as the root user. The pi user is one of those accounts. Using “sudo”, the pi user can perform individual actions as if it were root.

Unlike the Windows world, which occasionally displays a “User Account Control” (UAC) prompt to make sure you really mean it before letting you do something potentially dangerous, Linux systems take a different approach, and require you to say up-front that you know what you’re doing by running those potentially dangerous commands as root by using the sudo command.

You could also log in as root, or give “pi” root privileges, but that’s way more dangerous. The use of “sudo” acts to stop you from doing dangerous things without at least acknowledging that you know they are dangerous.

Wrapping up

You’ve reached the first major milestone. The Raspberry Pi is now a fully-functioning computer. It has been localized, personalized, and brought up to date with the latest software releases. Now is the ideal time to take a backup.

Shut down the Raspberry Pi “politely” by using the menu, or with the following command at a terminal window.

sudo shutdown –h now

This tells the compute to shut down “hard”, and to do it immediately. There are a few different shutdown options, including the ability to schedule a shutdown for a specified time and date in the future. This is the command you’ll use most often, though. After the screen turns off, wait a few more seconds for the Raspberry Pi to finish up any last-minute tasks, make sure all the lights have stopped blinking, unplug the power, and transfer the SD card to your regular computer.

You can use any number of different programs to create a backup. The first, and most common, is Win32DiskImager. This is the program used earlier to write the Raspbian OS image to the SD card in the first place. Use Win32 Disk Imager, “Read” the contents of the SD card into a backup file for safekeeping.

I have been using a different program lately called HDD Raw Copy. It functions a lot like Win32DiskImager, but has the ability to compress the resulting image file as it goes. The raw image files created by Win32DiskImager are the full size of the SD card being backed up, even though the SD card contains a lot of empty space right now. A compressed image file will be much smaller, allowing you to keep more of them around.

I personally keep backups of each step in this build process so that when something changes or breaks, and I’m trying to update this series for my readers, I can jump right to the step I need. If you only intend to keep around one recent backup for safety, then compressing it won’t help you nearly as much as it helps me.

The steps for using HDD Raw Copy are pretty similar to Win32DiskImager. First, select the source of the data to be backed up. That would be your SD card.

Click the “Continue” button, and select the destination for the backup. This would be a file somewhere on your hard drive. Double-click the “FILE” option from the list, and pick somewhere to put the file.

Make sure to pick “Compressed Image” from the “Save as type” drop-down to save space. HDD Raw Copy can create raw .img files just like Win32DiskImager, but my 8GB card was backed up into an image that’s only 4GB.

Click “Continue” again to get to the next screen which will show a summary of the proposed backup operation. This is your last chance to double-check the settings. Once you are satisfied, click “Start” and wait for the backup to complete. When it is finished, exit HDD Raw Copy, eject the SD card, and return it to your Raspberry Pi.

I’ll recommend making these checkpoint saves after pretty much each post in this series. You can refer back to this post for the instructions if needed.

What’s next?

In the next post, we’ll configure the network so that your Pi stays put, allowing you to find and connect to it later on.

Note: This post is part of a series. Each post builds on the previous ones. If you are just trying to add one thing to an existing system that was not built following this series, then I cannot promise that these instructions will work for you, although they probably will. If you’ve started from something other than a non-NOOBS Raspbian image, then you’ll probably need to adjust for that.

Please refer to the series Introduction for a list of all the different posts in the series.

Self-Promotion: I have recorded this series as a screencast for Pluralsight:
(http://www.pluralsight.com/courses/raspberry-pi-home-server)
If you have a Pluralsight subscription, please consider watching it. Reading the instructions is one thing, but watching it done demystifies the whole process.

Thank you!

What’s a Raspberry Pi?

The Raspberry Pi (http://amzn.to/2ecP7Zi) is a Linux computer about the size of a deck of cards (Uno cards if you put it in a case), and it’s not the prettiest computer in the world. The early models didn’t even really have a “front”. Because it’s so small, there were ports sticking off every side of the board. The newer models (B+, 2B, Zero, 3) have consolidated the ports so that cables are only sticking out of one or two sides now.

The Raspberry Pi was designed as an affordable computer to teach students computer programming with. Before long, it found a following in the “maker” community for its ability to act as an affordable, low-power brain for a wide variety of electronics projects. With its low cost, power consumption, and size, it has been used for a surprisingly large number of projects.

There have been several different models for sale. The $25 Model A+ has 256 MB of RAM, and 1 USB port. The $35 Model B has 512 MB of RAM, 2 USB ports, and a built-in Ethernet port. The B+ builds on the B and while it doesn’t change any of the key attributes of the system like RAM or CPU speed, it doubles the number of USB ports to four, moves some of the connectors around into a little nicer arrangement, and uses a bit less power. There was an earlier, 256 MB version of the Model B as well, but that was completely superseded by the 512 MB version. The Raspberry Pi 2 introduced a quad-core processor to the mix, keeping the same physical layout as the B+.

Next came the Raspberry Pi Zero, which only costs $5 (Usually), but has some serious limitations in both memory, speed, and connectivity. It’s a great choice for building small projects, or for fulfilling the Raspberry Pi’s original goal of teaching children to code, but it isn’t quite suitable for building this project.

Then, the Raspberry Pi Model 3 increased the overall speed of the system, and added built-in WiFi and Bluetooth.

Finally, the Raspberry Pi Zero W added WiFi and Bluetooth to the Zero.

I built my first Raspberry Pi Home Server with a Model B. These days I would recommend at least a Model 2, but you’ll get the best results using a Model 3.

But I don’t know Linux

Yeah… that’s pretty normal. Let’s face it, Linux is not an OS you’d put in front of your grandmother or non-technical spouse. For people used to Windows or Mac machines, Linux is going to be very unfamiliar. You generally don’t get to just right-click on things and pick “Properties” to set stuff up, and when you plug in new hardware, Linux doesn’t automatically download drivers from the internet and set them up for you. The desktop environment is getting pretty nice these days, though.

For those of us that were around for the first wave of home computers in the 80’s, and remember tweaking out the settings in autoexec.bat to get the memory settings just right so that we could play a certain game, this will be like a wave of nostalgia. For the younger folks, this may seem very strange and foreign.

Linux is an enchanted land full of magic words that you have to get exactly right in order for them to work. You have to stand in the right spot, at the right time of night, under exactly the right moon, and you’d better be sure that the chicken you’re about to sacrifice is a lefty, or you’ll bring doom and destruction on all the land.

Okay, it’s not quite that bad, but if you are uncomfortable with the idea of following instructions that tell you to type an obscure string of commands and parameters with little or no explanation as to what they mean, you’ll need to get over that fear. The internet is your friend, and you can look up how to do just about anything, but there are a lot of things to configure, and people aren’t always going to tell you why something has to be phrased the way it is.

In this series, I’ll try to explain the meaning of the obscure commands as best as I can, at least the ones that I understand well enough.

What hardware do I need?

Unless you bought some kind of bundle, your Raspberry Pi will need a few accessories before you can do anything with it. The Raspberry Pi doesn’t come with a power supply, or any kind of storage. When you buy a Pi, that’s all you get. A single-board computer, and nothing else.

You’ll need to supply your own monitor, keyboard, and mouse. The monitor can be anything with an HDMI or composite video input. There are adapters available to go from HDMI to VGA if that’s all you have available, but they can be pricey. Any USB keyboard and mouse will do.

Nearly everything in this series will be done from the command line, but the current Raspbian OS boots to the desktop initially, so you’re going to need a mouse as well, but only for the initial setup, and when configuring CrashPlan later on.

The power supply can be almost any Micro-USB phone charger, although it must provide at least 700mA for 1st generation Pis, 1800mA for the Pi 2, or 2500mA for the 3. Check here for a complete table of recommendations. Since the quality of the power supply can directly affect the stability of the system, I’d recommend buying a high quality supply like the official Raspberry Pi power supply. They’re only about $12, and will save you frustration later. I have some leftover chargers from old cell phones that claim to be full 1-Amp supplies, but can’t even power the caps-lock light on my keyboard. The better your supply, the better your experience will be. Trust me, it’s worth it.

Note: In general, you should try to avoid plugging things into the Raspberry Pi while it is running. Linux is more than capable of hot-plugging peripherals, but with such a small power supply, the Raspberry Pi has tendency to “brown out” and reboot when you plug things in. Your basic flash drive is probably safe, but plugging in a game controller might power-cycle the computer. You’ve been warned.

Storage comes in the form of an SD card. The larger the card, the more stuff it will hold, so you might be tempted to get the largest card you can. As part of this series, I’ll show you how to boot the Raspberry Pi from an external hard drive, so in the long run anything over 8GB will just go to waste, and will take longer to back up. For that reason, I usually stick to an 8GB card while I do the build. Similarly, once you’re booting from the hard drive, the speed of the card won’t matter anymore, but faster cards will make some of the initial setup noticeably faster. If you’re not planning to boot from the hard drive, then feel free to go a little larger on the SD card.

Finally, you may want to buy a case of some kind. There are plenty to choose from, and there are even plans you can print out and fold up to make a case out of card paper. Many people seem to like making cases from Lego bricks as well. I’d recommend a decent protective plastic case with keyhole slots on the back so that the whole thing can be mounted somewhere when you’re finished. You could also Velcro or tie-wrap the Raspberry Pi to the side of the hard drive if you want.

What software do I need?

The Raspberry Pi has no built-in operating system of any kind. It expects to find its OS on the SD card when it powers up, so you’ll need to get an OS onto the SD card before it’s going to do anything.

Note: Older versions of these posts talked about using the Raspberry Pi Foundation’s “NOOBS” image, but that really does cause more problems than it solves when building a server, so I have removed that information. If you really want to read that stuff, then you can dig around using the Internet Archive’s Wayback Machine:

https://web.archive.org/web/*/melgrubb.wordpress.com

These days, I only recommend getting a plain, Raspbian image from the Raspberry Pi Foundation’s download page, and writing it directly to the card using a program like Win32 Disk Imager or HDD Raw Copy

You might also want to go get the SD Association’s “SD Formatter” tool. This will help you reclaim the unallocated space on your SD card in the event that you want to use it for something else. When you write a 4GB image to a 16GM SD card, you can’t just stick it in a Windows machine, reformat it and expect to get all the space back. Windows is going to see an SD card with a 4GB partition, and 12GB of unallocated space which it will totally ignore. When you tell Windows to format the drive, it’s just going to format the 4GB partition, and leave the other 12GB alone. SD Formatter will format the whole card and get all your space back.

You can accomplish the same effect using something like diskpart on Windows or fdisk on Linux, and you’ll need them if things go really wrong, but in most cases, SD Formatter is all you need. See the supplemental post “SD Cards Gone Bad” for more information on using diskpart to recover corrupted SD cards.

What’s next?

That’s it for the introduction. In the next post, we’ll start building the initial OS image, and get the machine up and running.

Back in 2014, I wrote a series of posts on creating a complete home server using a Raspberry Pi. There were plenty of articles out there on the web that covered the same individual elements, and even a few that covered larger chunks of the entire process, but they were all either incomplete, or lacked the detail I wanted, so I started creating my own. The idea was that it would be a single series of articles written in a consistent style, at an appropriate level for a beginner, and would do their best to actually explain what the commands were doing as much as possible rather than just giving a “do this” list. I was explaining this process to myself as much as anyone else at the time.

Since that time, the Raspberry Pi world has changed several times. We’ve seen the release of several new Pi models (Model B+, Pi 2, Zero, and now the Pi 3), two major OS releases (Jessie and Stretch), and a new desktop environment (Pixel). I’ve spent a lot of time building and re-building essentially the same server over and over again each time something changed and broke my instructions. I also recorded this entire process from end to end as a course for PluralSight, and streamlined my personal approach to building the server along the way.

I feel that the time has come for me to stop patching my original blog articles and create a new set of posts, updated with all of the things I’ve learned since the original series was published, including what I think is a better ordering of the steps. For instance, I’ve found that moving the OS to the hard drive early on in the series became a burden since it was no longer possible to take a simple SD card backup of the system. As a result, I’ve moved that step closer to the end, and added a new section on making usable backups after the move has been made.

This also gives me a chance to declare “comment bankruptcy”, and start over with a clean set of reader comments that won’t be as clogged up with old information that’s no longer relevant.

The new posts are largely a copy/paste/update/clarify process. Most of the body text has been carried forward from the original posts, but I’m reviewing it top to bottom, fixing errors I somehow never noticed before, and adding clarity where I can. In addition, I’ll be adding a Troubleshooting section to each article addressing problems as they arise. I found that updating my existing instructions didn’t help those who had already built a server, but encountered problems after upgrading individual components.When things stop working, I want people to be able to come back and find the answers they need.

Lastly, I’m not going to number the individual steps this time around. This will allow me to rearrange the recommended order later on, and to insert or replace steps as needed. I also feel this will provide a better mechanism for alternative routes through the series.

Hopefully this version of the series will last for a while, and I won’t have to do this kind of overhaul again anytime soon. The original articles will remain, but with a note at the top redirecting users to the updated versions.

Introduction

This series originally started as my own notes on building a complete home server using a Raspberry Pi. I found a lot of other posts out there on the internet, but they were all in varying levels of detail, and were often out of date. As I have found since publishing my original series, keeping up with the Raspberry Pi world is hard. It changes very fast. Every time major changes happen, I brace myself for a flood of comments letting me know that one piece of the series or another has stopped working, and I have to play detective again to figure out how to adjust to the latest change.

Not only does the OS keep changing, but the individual components keep getting their own updates as well. Any one of these could receive an update at any time, but I try to stay on top of it as best I can. Fortunately for me, each component is fairly independent of the others, so I usually don’t have to revise everything at once. If something doesn’t work for you, please be sure to read through the comments. This series has received great support from the community, and often questions in the comments are answered by someone else before I even get home to look into them.

Article Index

• Introduction (You’re reading it now)
• Meet the Raspberry Pi
• Installing the OS
• Network Configuration
• Remote Desktop
• Adding a Hard Drive
• Samba File Sharing
• MiniDLNA
• OpenVPN
• Resilio Sync
• Network UPS Tools
• Booting From the Hard Drive
• Backing up the OS from the Hard Drive
• Transmission
• CrashPlan-less CrashPlan
• CrashPlan (Link to original post)
• Web Administration
• Syncthing
• LAMP (Link to original post)
• WordPress (Link to original post)
• When SD Cards Go Bad (Link to original post)
• Trimming the Fat
• Creating a home page
• Adding an “Emergency Stop” button

Acknowledgements

A lot of credit goes to the Instructables article that got me started: http://www.instructables.com/id/Ultimate-Pi-Based-Home-Server

It’s a good article, but it’s extremely out of date now. Also, I wanted to start from a standard Raspbian image rather than TurnKey, and I didn’t agree with some of the choices and ordering. For instance, I felt that basic file sharing should be tackled before building out a private cloud, and that the LAMP stack should be installed all at once before any other components that depend on it to prevent their installations from jumping the gun and installing bits of it on their own.