The home lab

After reading about others home lab environments, I was inspired to write about mine. It’s nothing too fancy but maybe somebody out there will find it interesting.

I’ve flipped-flopped several times between various operating systems and hypervisors trying to find the solution that best fits my needs. I’ve tried everything from pure Debian (kvm/libvirt, lxc), and Ubuntu (kvm/libvirt, lxd), to FreeBSD 11 (jails, bhyve), and SmartOS (zones, kvm).

At this point, I’ve settled on SmartOS, using project-fifo for my management layer and scheduler. I briefly tried out Joyent’s Triton, but it requires an entire machine be dedicated to running the head node with (at a minimum) 64GB of DRAM. Further, while it does provide an operations portal for configuring the system, the end-user portal requires you to sign-up for a support contract.

Why? SmartOS provides many benefits including running completely from DRAM. This is nice because I don’t have devote entire disk(s) for running the OS. The OS boots from a simple USB key where it loads its configuration and imports the ZFS datasets responsible for running your VM’s (zones/kvm). I won’t dive into the many benefits of ZFS here, but if you’re like me and value your data, you’ll appreciate it.

Because it boots from a USB key and the config is a single file, upgrades are extremely easy, just insert another USB key with an updated image and reboot. Boom.

So let’s get into the physical gear.

Servers:
* 1x Dell R720xd = 64GB RAM, 24x 300G 10K SAS + 1 SSD L2ARC
* 2x Dell R610 = 64GB RAM, 6x 250G SSD

Firewall/Router (not pictured):
* PCEngines APU2 = 2GB RAM, 30GB mSATA, 3 intel (igb) gigabit ports

Switch:
* Quanta LB4M 48port GB switch +2 10GB ports

Networking:
* Intel x520-da1 10Gbe (ixgbe) NIC on seach server
* Broadcom quad port 1gbe nic (bnx)
* Each server has two links to the switch. The first is a 1gbe link for the admin_nic and the second is a 10gbe for the trunk_nic which connects my zones/VMs to their respective VLANS.

I’ll layout the various VM’s I have running and why in a later post, or I’ll simply update this one at a later date.