To do this, you must be running at least FreeBSD 10.3-RELEASE – if you’re running an older version of FreeBSD, you should update! Also, ensure that you have a CPU that supports hardware virtualisation. Most new server CPUs seem to work just fine.
First, we need to install some packages to make things much easier:
pkg install vm-bhyve grub2-bhyve
Now we need to load some kernel modules on boot to allow us to virtualise and bridge networking… add the following to /boot/loader.conf:
You need to reboot to activate the modules, or you can load them now:
kldload if_bridge if_tap nmdm vmm
And we need to add some configuration to /etc/rc.conf:
Now we will setup some storage for the hypervisor – For this, we’ll assume ‘zroot’ zfs pool has enough capacity:
zfs create -o mountpoint=/vms zroot/vms
cp /usr/local/share/examples/vm-bhyve/* /vms/.templates/
Now we can create a virtual switch (bridge) and bridge it to our network interface. Change igb0 to your network interface below:
vm switch create public
vm switch add public igb0
Now, fetch an installation ISO for a FreeBSD guest:
vm iso ftp://ftp.uk.freebsd.org/pub/FreeBSD/releases/amd64/amd64/ISO-IMAGES/10.3/FreeBSD-10.3-RELEASE-amd64-disc1.iso
Now edit the default template /vms/.templates/default.conf (actually every .conf file in the folder) and change the disk config:
This will make vms use zvols instead of disk image files. Now we’re all prepared.
To create a test 16GB FreeBSD vm:
vm create -s 16G testvm
vm -f install testvm FreeBSD-10.3-RELEASE-amd64-disc1.iso
Select ‘vt100’ when asked for console type, and install as normal. Once finished, login to the new vm and power it off (use ‘poweroff’) – control will return to the host.
Here’s some useful commands to manage VMs…
To list all your VMs:
To start a VM called testvm:
vm start testvm
To stop a VM (gracefully via ACPI) called testvm:
vm stop testvm
To forcibly power off a VM called testvm:
vm poweroff testvm
To press reset on a VM called testvm:
vm reset testvm
To connect to the serial console on a VM called testvm:
vm console testvm
(to disconnect, send ~. to exit the console session)
To permanently destroy a VM called testvm (must be turned off first):
vm destroy testvm
To edit the configuration of a VM called testvm (e.g. to change the MAC address):
vm configure testvm
To increase the size of the disk on a VM called testvm to 32GB:
zfs set volsize=32G zroot/vms/testvm/disk0
To create an Ubuntu guest using a previously downloaded ISO called ubuntu.iso:
vm create -t ubuntu -s 16G testubuntu
vm install testubuntu ubuntu.iso
To make Ubuntu boot unaided each time, edit the configuration file for the VM just created (in /vms/testubuntu/testubuntu.conf) and add:
You can also add the above to the ubuntu template file for it to be automatically added to ubuntu VMs.
Windows guests take a lot more effort as there is no GUI console. This also means debugging a broken windows install is almost impossible so I would discourage its use for production. However, this is how to prepare to install Windows VMs:
fetch -o /vms/.config/BHYVE_UEFI.fd http://people.freebsd.org/%7Egrehan/bhyve_uefi/BHYVE_UEFI_20151002.fd
pkg install git p7zip cdrtools-devel
git clone https://github.com/nahanni/bhyve-windows-unattend-xml
This fetches a special UEFI loader, a collection of automated unattended Windows installation scripts, and the VirtIO drivers to allow storage to work.
Now copy a Windows installation ISO to /vms/work/win.iso and then:
7z x ../win.iso
cp /vms/work/bhyve-windows-unattend-xml/files/win2012r2_AutoUnattend.xml AutoUnattend.xml
This extracts the Windows ISO and copies an automated unattended installation script to it. There are several AutoUnattend.xml files so choose the one matching the OS you’re installing… we’re doing a Windows 2012R2 install here.
You can edit the AutoUnattend.xml file to set a product key if required (not required for recent Windows OS)
Now we copy the VirtIO drivers into the temporary work directory and build a new ISO image:
tar xf /vms/work/virtio-win-0.1.96.iso
-b boot/etfsboot.com -no-emul-boot -c BOOT.CAT \
-iso-level 4 -J -l -D \
-N -joliet-long \
-relaxed-filenames -v \
-V Custom -udf \
-boot-info-table -eltorito-alt-boot -eltorito-platform 0xEF \
-eltorito-boot efi/microsoft/boot/efisys_noprompt.bin \
-o /vms/.iso/win2k12r2.iso .
You now have a win2k12r2.iso file ready to build a windows VM:
vm create -t windows -s 64G wintest
vm install wintest win2k12r2.iso
This will start the unattended installation of windows. It can take up to around 25 minutes. You can monitor its progress by looking at the logfile:
tail -f /vms/wintest/vm-bhyve.log
You need to wait for it to restart twice (so three “starting bhyve” messages) – you can then get the IP from the server:
vm console wintest
and press “i” to view the IP address – this only seems to work on win2k12r2 and above. You may need to check your DHCP server logs otherwise.
You can now RDP to the VM using Administrator and Test123 as the password.
Again, you should shutdown the VM and start it using the vm start wintest command.
To make VMs automatically start on boot, you can add them to /etc/rc.conf in the vm_list variable, for example to start wintest and testvm: