Deploying Kubernetes on SmartOS
Some things are just explained better through video. Here I’m deploying Kubernetes on SmartOS with the help of bhyve.
Bhyve pfSense 2.4 no console menu
I ran into an annoying issue today while trying to install pfsense 2.4.2 in a bhyve VM using the ISO installer. Everything went swimmingly until post-install when pfsense finished startup and never provided the expected pfSense console. All it would show is bootup complete.
I went through and confirmed /etc/ttys was configured properly and added console=comconsole to /boot/loader.conf. However, it still wouldn’t work. I’d get all the typical startup info but it still wouldn’t drop to the pfSense console.
To fix this, I ended up having to install pfSense using the memstick serial installer.
In case you aren’t already using vm-bhyve, here’s how it went down from start to finish:
1. Install and initial setup:
pkg install -y vm-bhyve grub2-bhyve zfs create -o mountpoint=/bhyve zroot/bhyve sysrc vm_enable="YES" sysrc vm_dir="zfs:zroot/bhyve" vm init
I manage network bridges myself, so I’ll just import them into vm-bhyve so it can use them.
vm switch import wan bridge0 vm switch import mgmt bridge1 fetch -o /tmp/pf-memstick-serial.img.gz https://nyifiles.pfsense.org/mirror/downloads/pfSense-CE-memstick-serial-2.4.2-RELEASE-amd64.img.gz gunzip /tmp/pf-memstick-serial.img.gz
2. Create pfsense VM:
cd /bhyve/.templates cat > pfsense.conf <<EOF loader="bhyveload" cpu=2 memory=512M network0type="virtio-net" network0switch="wan" network1type="virtio-net" network1switch="mgmt" disk0type="virtio-blk" disk0name="disk0.img" EOF vm create -t pfsense -s20G pf1
3. Temporarily reconfigure the VM to use the memstick installer.
Basically, we just need to add another disk (the installer image) and make sure it’s first to boot.
cd /bhyve/pf1/ cp /tmp/ cp pf1.conf pf1.orig.conf cat >pf1.conf<<EOF loader="bhyveload" cpu=2 memory=512M network0_type="virtio-net" network0_switch="wan" network1_type="virtio-net" network1_switch="mgmt" disk0_type="virtio-blk" disk0_name="/tmp/pf-memstick-serial.img" disk1_type="virtio-blk" disk1_name="disk0.img" EOF vm start pf1 vm console pf1
Walk through install process and when finished DON’T reboot. Simply disconnect from the console (~ + ctrl-d) and shut the vm down. If we let it reboot, it’ll just reboot back into the intaller since it’s still configured as the first disk.
vm stop pf1 mv /bhyve/pf1/pf1.orig.conf /bhyve/pf1/pf1.conf vm start pf1
After it was all said and done, I checked what the memstick installer inserts into /boot/loader.conf to make it work. I’m guessing this is the key, and what I should’ve added to the loader config when I tried using the ISO installer initially.
boot_multicons="YES" boot_serial="YES" console="comconsole,vidconsole" comconsole_speed="115200"
I didn’t try using the ISO installer and adding the above as I just wanted to get up and running, but it’d be interesting to see if it would do the trick.
Hope this helps some of you!