Bhyve pfSense 2.4 no console menu

I ran into an annoying issue today while trying to install pfsense 2.4.2 in a bhyve VM using the ISO installer. Everything went swimmingly until post-install when pfsense finished startup and never provided the expected pfSense console. All it would show is bootup complete.

I went through and confirmed /etc/ttys was configured properly and added console=comconsole to /boot/loader.conf. However, it still wouldn’t work. I’d get all the typical startup info but it still wouldn’t drop to the pfSense console.

To fix this, I ended up having to install pfSense using the memstick serial installer.

In case you aren’t already using vm-bhyve, here’s how it went down from start to finish:

1. Install and initial setup:

pkg install -y vm-bhyve grub2-bhyve
zfs create -o mountpoint=/bhyve zroot/bhyve
sysrc vm_enable="YES"
sysrc vm_dir="zfs:zroot/bhyve"
vm init

I manage network bridges myself, so I’ll just import them into vm-bhyve so it can use them.

vm switch import wan bridge0
vm switch import mgmt bridge1
fetch -o /tmp/pf-memstick-serial.img.gz https://nyifiles.pfsense.org/mirror/downloads/pfSense-CE-memstick-serial-2.4.2-RELEASE-amd64.img.gz
gunzip /tmp/pf-memstick-serial.img.gz

2. Create pfsense VM:

cd /bhyve/.templates
cat > pfsense.conf <<EOF
loader="bhyveload"
cpu=2
memory=512M
network0type="virtio-net"
network0switch="wan"
network1type="virtio-net"
network1switch="mgmt"
disk0type="virtio-blk"
disk0name="disk0.img"
EOF
vm create -t pfsense -s20G pf1

3. Temporarily reconfigure the VM to use the memstick installer.
Basically, we just need to add another disk (the installer image) and make sure it’s first to boot.

cd /bhyve/pf1/
cp /tmp/
cp pf1.conf pf1.orig.conf
cat >pf1.conf<<EOF
loader="bhyveload"
cpu=2
memory=512M
network0_type="virtio-net"
network0_switch="wan"
network1_type="virtio-net"
network1_switch="mgmt"
disk0_type="virtio-blk"
disk0_name="/tmp/pf-memstick-serial.img"
disk1_type="virtio-blk"
disk1_name="disk0.img"
EOF
vm start pf1
vm console pf1

Walk through install process and when finished DON’T reboot. Simply disconnect from the console (~ + ctrl-d) and shut the vm down. If we let it reboot, it’ll just reboot back into the intaller since it’s still configured as the first disk.

vm stop pf1
mv /bhyve/pf1/pf1.orig.conf /bhyve/pf1/pf1.conf
vm start pf1

After it was all said and done, I checked what the memstick installer inserts into /boot/loader.conf to make it work. I’m guessing this is the key, and what I should’ve added to the loader config when I tried using the ISO installer initially.

boot_multicons="YES"
boot_serial="YES"
console="comconsole,vidconsole"
comconsole_speed="115200"

I didn’t try using the ISO installer and adding the above as I just wanted to get up and running, but it’d be interesting to see if it would do the trick.

Hope this helps some of you!