State of the Blog

I wanted to post an update here to let people know what’s going on with the blog, and the Raspberry Pi Home Server series in particular. I realized a while ago that the Pi world moves incredible fast, and my articles keep getting out of date. This affects some articles more than others.

The worst one is the CrashPlan installation. Code42 does not support ARM-based devices, and in the last year or so seems to have really cranked up the frequency of their releases. With each release, CrashPlan auto-updates itself, destroying all of the necessary tweaks that were done to make it work in the first place. There’s no good solution to this problem. There are, however, some kinda bad but they’ll have to do solutions. I’ve heard mixed reviews about the success or failure of the different approaches, and have been trying to crowdsource a consensus out of my comments, which now dwarf the articles they are attached to.

I’ve also just recently had a hardware failure. The SD card in my own server croaked, and while I DO have backups, Raspbian has moved significantly onward since I first wrote these articles. I’m taking this as an opportunity to rebuild from scratch, document it all again, and release an updated set of articles with new ordering, advice, and screenshots. This is going to take a little while, but I’m working on it.

It’s time to declare comment bankruptcy and publish the RPHS v2 series, and it starts right now with me rebuilding my own server from scratch, and documenting it all like I did the first time.

In the meantime, the old articles are still perfectly valid, and should get you where you need to be. Just make sure you read the Updates sections that now appear on pretty much all of the articles. I think my plan now is to do less updating in-place, and publish new versions of the articles instead. This will let me occasionally reset the long tail of comments each article has grown, and start over with a fresh set for each update. It’s worth a shot.

Posted in Computers and Internet | 2 Comments

Raspberry Pi Home Server: Changes

A lot of exciting changes are happening, and at least one major disappointment. There’s a new Raspbian image out(https://www.raspberrypi.org/downloads/raspbian), and it brings a new desktop along with it (https://www.raspberrypi.org/blog/introducing-pixel). The Pixel desktop is a Pi-optimized LXDE variant, and it looks pretty much like what you’re used to if you’ve built a Pi since the beginning of the year. There are some notable changes, though.

Chromium Browser: This is the open-source version of Chrome, and while we’ve always had the ability to install it ourselves, it’s there by default now. Not only that, but it’s got some plug-ins preconfigured to make the web more Pi-friendly. In particular, it’ll force YouTube to send videos in a format that the Pi can decode more easily using hardware acceleration. That doesn’t really mean much to a server, but it’s a welcome upgrade.

Built-in VNC: That whole chapter I wrote about x11vnc? Yeah, you can forget that now. You now have a working remote desktop included in Raspbian out of the box. Just download the RealVNC Viewer and connect. It might be possible to use other viewers like UltraVNC, but it’ll take some configuration. The RealVNC viewer is small, portable (no need to install it), and works immediately. I’ve made the switch.

And now for the bad news. The folks at Code42 have been pushing updates to CrashPlan at an increasing rate, and every time CrashPlan updates itself, it breaks all of the patching and tweaking we had to do to get it running. At first, this was merely annoying. Now it’s become unmaintainable. I can’t trust my backup machine if it can be brought down at any time by a silent, compulsory update. I’m working on a new blog post about alternative approaches to using CrashPlan at home, and I’m hoping that the Pi can still be a part of that setup.

In the meantime, if you want to continue holding the Pi’s hand, and diving into the guts of CrashPlan roughly once a week, be my guest, but I won’t be putting any more time into maintaining the CrashPlan instructions unless I hear that Code42 has either started supporting ARM-based devices directly, or at least given us an alternative to stop the constant updates from undoing everything.

Posted in Computers and Internet | 4 Comments

Backing up a Pi that boots from the hard drive

I get this question a lot, and figured it’s about time to address it. Once you’ve finished a Raspberry Pi Home Server, and have it booting from the hard drive, how are you supposed to take backups? After all, you can’t just pop out the SD card and just run it through Win32DiskImager anymore.

If your main computer is a Mac or Linux machine, then you can simply shut down the Pi, hook it up to your main computer, and use the “dd” command to make a copy of the OS partition in the same way you wrote the SD card in the first place, but with the “if” and “of” parameters reversed.

If your main computer is a Windows machine, then one answer is to create a secondary, minimal SD card or the Pi. You can use Raspbian Lite for this, since you won’t need anything but the command line. Then, you just shut down the Pi, drag it back over to your desk and hook it back up to the monitor and keyboard, swap the SD cards, boot up using the new SD card, and use the “dd” command to write a backup of the OS partition to the Data partition on the same drive, or mount a flash drive, and back up to there.

But wait… there’s a much simpler option. You can just attach the hard drive to your Windows computer, and use Win32DiskImager to take a backup of the whole hard drive. You’ll get a snapshot of the complete working state of the system this way. If the drive attached to the Pi is very large, then I realize this may not be an option, but let me tell you about my current setup.

My Raspberry Pi Home Server has two drives these days. The first is a smallish SSD (like 60GB small) that I have long since “traded up” from. It still has two partitions on it, a 16GB OS partition, and a Data partition, but that’s not where most of my actual stuff goes. I use the Data partition on this first drive for things like the swap file, MiniDLNA’s database, and CrashPlan’s metadata, but my actual music, movies, pictures, and CrashPlan backup files are all on a second hard drive, which is the 2TB RAID. In this way, I’ve completely separated my “stuff” from the system. The storage drive is blissfully unaware of the Pi’s existence. I still format this drive using NTFS, so I can just plug it into my regular computer and get to my stuff.

By the way, I don’t really use Win32DiskImager anymore, either. These days I use HDDRawCopy, which has the added benefit of writing out compressed image files, so instead of creating a 60GB file and then Zipping it up to get my hard drive space back, I end up with a file that’s already compressed in the first place.

Every now and then, after doing system upgrades, I’ll shut the whole thing down, hook the primary drive up to my regular computer, and create a new image. This image contains not only the OS, but the current state of the Data partition as well. If I’ve added music or movies since the last back up, then MiniDLNA will have to play catch-up when the system starts up, but eventually, it all sorts itself out.

But what if you only have a single, very large drive hooked up to the Pi? There are many programs available that will let you back up individual partitions. I don’t have a personal favorite here, but I would welcome recommendations in the comments.

I’ve heard good things about Paragon Backup & Recovery, and Macrium Reflect Free, but I don’t know if backing up and restoring individual partitions is included in their free versions or not. Also, I think the free version of Paragon is limited to backing up and restoring drives that are 2TB or less. I’m not sure about Macrium. Perhaps I’ll find the time to do some testing on these free solutions and update this post a bit later on.

Posted in Computers and Internet | 3 Comments

CrashPi Zero?

So I just went to my local MicroCenter, and picked up a Raspberry Pi Zero for $5. It’s pretty much everything the model “A” had, but with 512MB of memory. You could also say that it’s like a model “B” without the network jack. Networking is really the only thing this model is missing, and if you are planning to use a USB hub, or just a wireless network adapter, then you should be all set.

The way I have the CrashPi set up right now with a powered USB hub running the whole thing, I realize that I could replace the current B+ with a Zero, and everything would continue to work just fine. The only new piece of hardware needed is a Micro-USB to regular USB adapter, sometimes called an “OTG” (USB On-The-Go) adapter, so I picked one of those up as well. Due to the new design, I also had to use a Mini-HDMI adapter which I originally bought for our DSLR camera. I have used the adapter precisely zero times for that purpose, so it’s nice to see it finally getting some use.

I just burned a new Raspbian Jessie image to the card and plugged it in. According to my “Charger Doctor”, it’s only drawing 130 milliamps with the desktop running, so the power consumption is considerably less than previous models. The possibility of battery-powering this new version is very real. The composite video output is still there in the form of some unpopulated solder pads on the board, so you’ll have to provide your own jack. At this point, I’m thinking it should be very possible to buy one of those “TV Games” joysticks, and entirely replace the guts with a Pi Zero, making it the most portable DIY emulation system ever.

If it weren’t for the lack of networking, I’d be ready to replace my single Raspberry Pi Home Server with a cluster of Pi Zeros, each running just one or two of the normal server functions. You could have one that just runs VPN, another that’s just for the file server, and another that’s just doing CrashPlan. The functions would be more isolated, and you’d STILL be cheaper than a normal Pi. Adding in USB hubs and wireless adapters would raise the cost of course, but it’s amazing what you can find on Amazon.

Oh yes. There will be updates coming.

Posted in Computers and Internet, Home Server, Raspberry Pi | 3 Comments

CrashPi – An off-Site backup for the whole family

In Part 15 of the Raspberry Pi Home Server series, I mentioned the possibility of building a “CrashPi”, a simplified version of the full server running only CrashPlan, and some basic supporting functions such as X11VNC for maintenance. I hadn’t actually built one, though. It was just an idea… until now.

Today, I’m introducing a dedicated CrashPi into my home network. The original idea of the CrashPi was for it to be placed at a friend or relative’s house to act as my off-site backup, keeping the home videos, photos, and documents safe in case of a disaster like a house fire. I hadn’t quite gotten around to asking to mooch from the in-laws broadband, though.

Then, just last week, my wife suggested that I could just put it in the garage, which is not attached to our house. The odds of the house and the garage being lost in anything short of a meteor strike are pretty low, so I’ve decided to build a real CrashPi, and to use CrashPlan’s “friend codes” to act as a true off-site backup for my college-age twins, who are now off on their own. No matter where they go now, the CrashPi in my garage is their secure backup.

Gathering the parts

I have a few older Raspberry Pis sitting around since the 2s came out, so I’m going to use my old B+, which has more RAM than my original B. This extra memory will be very helpful for CrashPlan, which can be rather resource hungry. Offloading CrashPlan to a separate server means that my main home server will have less to do now, which will also help with its stability.

The complete list of parts is as follows (with Amazon links):

  1. Raspberry Pi Model B+ (http://amzn.to/24K2Fiz)
  2. Power Supply (http://amzn.to/1p93m58)
  3. 8GB Micro SD card (http://amzn.to/1p93kKu)
  4. TP-Link TL-WN722N WiFi adapter (http://amzn.to/21TlD40)
  5. CyberPower SX550G UPS (http://amzn.to/1pt06RO)
  6. 5TB Seagate “Expansion” drive (http://amzn.to/1pt04cN)
  7. BUD Industries NBF-32018 ABS NEMA Enclosure (http://amzn.to/1pt0aRs)
  8. C14 Power Inlet (http://amzn.to/1pt09NE)
  9. Panel-mount USB cable (http://amzn.to/1pt0fol)
  10. Powered USB Hub (http://amzn.to/21TmbXG or similar)
  11. Adhesive anchors (http://amzn.to/1p94SE5)

Note: Not all the items actually came from Amazon. Most of the parts (hubs, power supplies, cables, WiFi dongle, etc.) came from my local MicroCenter, but not everyone has a toy store like that close to their house, so I’m providing Amazon links so you can at least see the parts I’m talking about. My hub is actually made by Inland, but Amazon doesn’t carry it, so I picked something similar.

The Pi, power supply, and SD card I already had on hand. The other items I bought new for this project. I have a smaller WiFi adapter (Tenda W311M) that I’ve used inside the house, but wanted a little more range to reach the garage, so I sprung for the dual band. I know how Pis hate having their power yanked, so I picked up another $45 UPS to keep things maintenance-free. The hard drive was the biggest expense on this project. All together, it came to a little over $200, but I think it will be worth it to keep the whole family’s stuff safe, and especially the kids. One has already had a laptop stolen and lost all of her work for the semester up to that point, so the cost seemed worth it to me.

I won’t repeat all the individual steps here, but I’ll refer back to the original posts, and explain any differences.

Install the OS

installed the OS as usual using the current Raspbian image (not NOOBS). I left the overclocking off this time, favoring stability over performance for the CrashPi. I didn’t add any additional users, either.

Going Headless

Next, I set everything up for headless operation, installing Webmin and X11VNC exactly the same way as I did for the main server.

Attaching a Hard Drive

I attached the hard drive, mounted it at “/mnt/backups”, and created a “public” share through Samba. I did not, however, choose to boot from the hard drive in this case. I did set up a 1GB swap file on the external drive, and configured the Pi to use that for swap space rather than the file on the SD card. This will prolong the life of the SD card. I opted for NTFS as the file system again so that I can easily attach the drive to my regular computer if needed. “ext4” is still your best choice for performance and reduced overhead, though.

CrashPlan

Of course, the whole point of this project is to run CrashPlan, so that went on next, with the backups going to “/mnt/backups/crashplan”. I have signed into CrashPlan using the same credentials as my regular computer and Raspberry Pi Home Server. I signed each of the kids up with their own accounts, though, and used the friend code from the CrashPi instance to use as a destination.

It’s important to note that the friend codes are installation-specific, so the code from the CrashPi is different than the code from my regular computer or the existing server. Using the friend code from the CrashPi means that my kids have the ability to backup to the CrashPi, and only the CrashPi, so I don’t need to worry about them using up all of my primary server’s space.

Also, since each kid has their own account, they get their own notification emails about how up-to-date their backups are, rather than all of the emails going to me. Using their own accounts also means that their stuff is safe from prying eyes, even mine. They own the encryption of their individual backups, not me. The downside is that if they forget the password to their CrashPlan account, I can’t help them and they’ll have to start all over again.

NUT

Finally, I installed NUT, and configured the CrashPi to shut itself down in the event of a power failure. I didn’t set up email notifications about power events or anything like that because my main home server is already telling me about those.

Building the enclosure

Here’s where the project starts to become its own thing. I ordered the BUD box from Amazon based on it’s interior dimensions. It’s a bit deeper than I had anticipated, but that worked out well as it allows me to simply lay the hard drive down on the bottom of the enclosure rather than mounting it to the back of the box as I originally had in mind.

A metal box would have let me mount a bare hard drive directly to it, and would have acted as a heat sink, but I went with a plastic box so as not to interfere with the wireless signal. A metal box would have required an external WiFi antenna. I tried to build a system with as little custom fabrication as possible so that all the pieces can be easily swapped out if needed.

I could have run cords out of a simple hole in the box, but since this computer will be mounted in my dusty garage, I wanted to leave no holes where dust or spiders could get in and mess things up. For that reason, I wanted to keep the number of holes to a minimum (zero). I got the power inlet and panel-mount USB port so that once they were in place, I could seal around the openings with some RTV sealant I already had around. This should keep everything clean, and keep any insects out.

WP_20150922_17_08_54_ProWP_20150922_17_08_42_Pro

I cut most of the cord off of a small, three-outlet power strip (from Big Lots), soldered the leads directly to the power inlet, and enclosed the joints with some heat-shrink tubing. Then I mounted all the components to their respective walls using tie-wraps and adhesive anchors. The end result looks like this:

WP_20151124_08_55_46_Pro

On the right is the USB hub and power strip (behind the hub). On the back of the box is the Pi itself, and just resting on the bottom of the box is the drive. There’s still room in this box for me to add another drive or two if needed, so I should be set for quite a while.

As you can see, the Pi is powered by its own hub. The hub is plugged into one of the Pi’s regular USB ports, but the Pi’s own power cord plugs right back into the hub again. You might not expect it, but this works just fine because the micro-USB jack on the Pi doesn’t even use the data lines, it only uses the power lines. You don’t have to worry about the Pi “talking to itself” or anything.

The drive I chose has its own power supply, but luckily it’s been implemented properly and doesn’t “back-power” the hub (and thereby the Pi), so when I turn the hub off, the whole system goes down. Naturally, I log in remotely and “sudo shutdown -h now” first.

Finally, here’s a picture of the whole thing mounted to the wall in my garage:

WP_20151124_08_56_11_Pro.jpg

The single cord coming from the right side of the UPS (The battery-backed side) is running everything in the server box. The USB cable coming from the bottom of the UPS runs to the panel-mount USB port, which is then plugged directly into the Pi internally. The two plugs on the left have nothing to do with the server. They are for a couple tool chargers, and the UPS just provided a convenient, surge-protected place to plug them in.

It’s not the world’s most attractive “server rack” for sure, but it’s weather-proof and kind of off-site. It’s now the destination for all of my backups, including backing up the data from my regular Raspberry Pi Home Server. I’ve set up CrashPlan on my server to back up it’s drive full of stuff (Media, mostly) to the CrashPi, so now I have a redundant backup of all of the family photos, videos, and music. These were already stored on a two-drive RAID, but now they’re even better protected in the event of a disaster.

Posted in Computers and Internet, Home Server, Raspberry Pi | 4 Comments

Raspbian Jessie released

Raspbian “Jessie” just went official, and that means that those of you following along with my instructions are going to notice some serious changes. The first and most obvious is that Jessie boots straight to the desktop by default, and raspi-config has been superseded augmented with a desktop GUI tool.

I have built up a new server using the Jessie image, and will add updates to the beginning of each individual post. I’ll be updating the posts in-place in the near future, but I’ve simply run out of weekend for now.

Posted in Computers and Internet, Home Server, Raspberry Pi | Leave a comment

The Raspberry Pi Home Server series is now on Pluralsight

I’ve been working on this for the last couple of months, and I’m excited to announce that I’ve turned this series into a course for Pluralsight. The course is available at:

http://www.pluralsight.com/courses/raspberry-pi-home-server

For those that don’t know already, Pluralsight is a subscription-based online training site. Don’t worry, I’m not going to take the blog posts away. I would never do that to you. Sometimes you just want to SEE something being done instead of just reading about it, though. You can follow along with the bouncing ball, and build your own server along with me.

The course is divided up in much the same way as these blog posts, with each module adding on one more piece of the puzzle, although the order is a little different, and some items (like Transmission) weren’t included.

If you have a Pluralsight subscription, please consider watching the course. If it does well, I might be able to do more of them.

Posted in Computers and Internet | 8 Comments