Daily Archives: May 30, 2016

OpenVPN Setup in FreeBSD (with NAT for IPv4 and IPv6)

It seems people always use Linux when it comes to setting up an OpenVPN server for internet access, so here’s how to do it on FreeBSD.

You’ll need at least FreeBSD 10.3, but here’s how to do it.

Our interface will be igb0, our IPv4 public IP will be 192.0.2.10 and our IPv6 public IP will be 2001:DB8::10 – these will both be shared via NAT.

First we need to install some packages…

pkg install bash easy-rsa openvpn

Now we need to load some kernel drivers, add to /boot/loader.conf:

net.inet.ip.fw.default_to_accept=”1″
aesni_load=”YES”
crypto_load=”YES”
if_bridge_load=”YES”
if_tap_load=”YES”
ipfw_load=”YES”
ipfw_nat_load=”YES”

You will need to reboot to activate them, but you can do that at the end of the process.

We need to configure pf to NAT our IPv6 IPs, create /etc/pf.conf with these contents:

v6_wan_if=”igb0″
v6_wan_ip=”2001:DB8::10″

no nat on $v6_wan_if inet6 from $v6_wan_ip to any
nat on $v6_wan_if inet6 from any to any -> $v6_wan_ip

and we need to activate the pf config in /etc/rc.conf by adding:

pf_enable=”YES”
pf_rules=”/etc/pf.conf”
pf_program=”/sbin/pfctl”
pf_flags=””

We’re going to handle IPv4 nat using ipfw instead of pf (this may not be the best of things, but it works well) – we’ll also do some firewalling at the same time.

Create a file called /usr/local/etc/rc.d/000.ipfw.sh with the following contents:

#!/bin/sh

case “$1” in
‘start’)
/sbin/ipfw -f flush

/sbin/ipfw nat 4 config log ip 192.0.2.10

/sbin/ipfw add 00100 allow ipv6-icmp from :: to ff02::/16 // allow DAD
/sbin/ipfw add 00110 allow ipv6-icmp from fe80::/10 to fe80::/10 // allow RS RA NS NA Redirects
/sbin/ipfw add 00120 allow ipv6-icmp from fe80::/10 to ff02::/16 // allow RS RA NS NA Redirects
/sbin/ipfw add 00130 allow ipv6-icmp from any to any icmp6types 1 // allow destination unreachables
/sbin/ipfw add 00140 allow ipv6-icmp from any to any icmp6types 2,135,136 // allow NS/NA/toobig ICMPs

/sbin/ipfw add 00500 nat 4 ip4 from any to any via igb0 // NATv4

/sbin/ipfw add 00700 check-state

/sbin/ipfw add 01000 allow all from any to any via lo0 // allow loopback
/sbin/ipfw add 01010 deny all from any to 127.0.0.0/8
/sbin/ipfw add 01020 deny all from 127.0.0.0/8 to any
/sbin/ipfw add 01030 deny all from any to ::1
/sbin/ipfw add 01040 deny all from ::1 to any

/sbin/ipfw add 02000 allow ipv6-icmp from :: to ff02::/16 // allow DAD
/sbin/ipfw add 02010 allow ipv6-icmp from fe80::/10 to fe80::/10 // allow RS RA NS NA Redirects
/sbin/ipfw add 02020 allow ipv6-icmp from fe80::/10 to ff02::/16 // allow RS RA NS NA Redirects
/sbin/ipfw add 02030 allow ipv6-icmp from any to any icmp6types 1 // allow destination unreachables
/sbin/ipfw add 02040 allow ipv6-icmp from any to any icmp6types 2,135,136 // allow NS/NA/toobig ICMPs

/sbin/ipfw add 03000 allow tcp from any to any established
/sbin/ipfw add 03100 allow all from any to any frag
/sbin/ipfw add 03200 allow tcp from me to any setup
/sbin/ipfw add 03300 allow udp from me to any 53 keep-state
/sbin/ipfw add 03400 allow udp from me to any 123 keep-state

;;
‘stop’)
;;
*)
echo “Please specify ‘start’ or ‘stop'”
;;
esac

and ensure it runs on boot:

chmod a+x /usr/local/etc/rc.d/000.ipfw.sh

Now… onto OpenVPN configuration.

Create a file called /usr/local/etc/openvpn/openvpn.conf with the following contents:

script-security 2
local 192.0.2.10
port 1194
proto udp
dev tun
ca /usr/local/etc/openvpn/keys/ca.crt
cert /usr/local/etc/openvpn/keys/vpn.crt
key /usr/local/etc/openvpn/keys/vpn.key  # This file should be kept secret
crl-verify /usr/local/etc/openvpn/keys/crl.pem
dh /usr/local/etc/openvpn/keys/dh.pem
tun-ipv6
ifconfig-pool-persist /usr/local/etc/openvpn/ipp.txt
server 172.31.255.0 255.255.255.0
server-ipv6 fc00:da::/64
push “redirect-gateway def1 bypass-dhcp”
push “dhcp-option DNS 8.8.8.8”
push “route-ipv6 ::/1”
push “route-ipv6 8000::/1”
client-config-dir /usr/local/etc/openvpn/ccd
keepalive 10 30
comp-lzo
persist-key
persist-tun
status /var/log/openvpn-status.log
log-append /var/log/openvpn.log
verb 4

You’ll notice we’re using fc00:da::/64 as the IPv6 prefix for VPN clients, this is a range reserved for local usage so will not conflict with any globally reachable IP addresses.

We need to configure OpenVPN’s certificate authority now:

easyrsa init-pki
easyrsa build-ca
(enter password, set Common Name to VPN hostname)
easyrsa gen-dh
easyrsa gen-req vpn nopass
(set Common Name to VPN hostname)
easyrsa sign server vpn
(answer ‘yes’ and use password from CA step above)
easyrsa gen-crl
(enter password from CA step above)

Now we need to copy some files into the OpenVPN working directory:

mkdir /usr/local/etc/openvpn/ccd
cd /usr/local/share/easy-rsa/pki
mkdir -p /usr/local/etc/openvpn/keys
cp -p ca.crt crl.pem dh.pem index* serial* private/vpn.key issued/vpn.crt /usr/local/etc/openvpn/keys/

We’re now ready to add some users… repeat the following for each new client you want to create:

easyrsa gen-req client1 nopass
easyrsa sign client client1
cd /usr/local/share/easy-rsa/pki
cp -p private/client1.key issued/client1.crt /usr/local/etc/openvpn/keys/

and also create a file for each called /usr/local/etc/openvpn/ccd/client1 (where client1 is the username) containing:

iroute 10.0.0.0 255.0.0.0
iroute 192.168.0.0 255.255.0.0
iroute 172.16.0.0 255.240.0.0

Now all we need to do is generate the configure file(s) for each client to use.

Create a file called /usr/local/etc/openvpn/client1.ovpn containing:

client
dev tun
remote 192.0.2.10 1194 udp
resolv-retry infinite
nobind
persist-key
persist-tun
tun-ipv6
mute-replay-warnings
<ca>
(contents of /usr/local/etc/openvpn/keys/ca.crt)
</ca>
<key>
(contents of /usr/local/etc/openvpn/keys/client1.key)
</key>
<cert>
(contents of /usr/local/etc/openvpn/keys/client1.crt)
</cert>
route-delay
comp-lzo
verb 3
mute 20

The file needs to contain the contents of three files, including the lines starting “—–”

Copy this .ovpn file to the client device’s config directory – it should also work with the Android OpenVPN client.

Finally, we need to tell OpenVPN to start when FreeBSD starts by adding the following to your /etc/rc.conf file:

openvpn_enable=”YES”

Now, simply reboot and everything should work fine!

If you need to revoke a user (e.g. if they lose their device, or they become compromised), then run the following commands:

easyrsa revoke client1
easyrsa gen-crl
cp /usr/local/share/easy-rsa/pki/crl.pem /usr/local/etc/openvpn/keys/
killall -USR1 openvpn

This will revoke the client from connecting without having to re-generate all other client certificates.

Using FreeBSD as a Hypervisor (using bhyve and vm-bhyve to manage them)

To do this, you must be running at least FreeBSD 10.3-RELEASE – if you’re running an older version of FreeBSD, you should update!  Also, ensure that you have a CPU that supports hardware virtualisation.  Most new server CPUs seem to work just fine.

First, we need to install some packages to make things much easier:

pkg install vm-bhyve grub2-bhyve

Now we need to load some kernel modules on boot to allow us to virtualise and bridge networking… add the following to /boot/loader.conf:

if_bridge_load=”YES”
if_tap_load=”YES”
nmdm_load=”YES”
vmm_load=”YES”

You need to reboot to activate the modules, or you can load them now:

kldload if_bridge if_tap nmdm vmm

And we need to add some configuration to /etc/rc.conf:

vm_enable=”YES”
vm_dir=”zfs:zroot/vms”
vm_list=””
vm_delay=”5″

Now we will setup some storage for the hypervisor – For this, we’ll assume ‘zroot’ zfs pool has enough capacity:

zfs create -o mountpoint=/vms zroot/vms
vm init
cp /usr/local/share/examples/vm-bhyve/* /vms/.templates/

Now we can create a virtual switch (bridge) and bridge it to our network interface.  Change igb0 to your network interface below:

vm switch create public
vm switch add public igb0

Now, fetch an installation ISO for a FreeBSD guest:

vm iso ftp://ftp.uk.freebsd.org/pub/FreeBSD/releases/amd64/amd64/ISO-IMAGES/10.3/FreeBSD-10.3-RELEASE-amd64-disc1.iso

Now edit the default template /vms/.templates/default.conf (actually every .conf file in the folder) and change the disk config:

disk0_name=”disk0″
disk0_dev=”sparse-zvol”

This will make vms use zvols instead of disk image files.  Now we’re all prepared.

To create a test 16GB FreeBSD vm:

vm create -s 16G testvm
vm -f install testvm FreeBSD-10.3-RELEASE-amd64-disc1.iso

Select ‘vt100’ when asked for console type, and install as normal.  Once finished, login to the new vm and power it off (use ‘poweroff’) – control will return to the host.

Here’s some useful commands to manage VMs…

To list all your VMs:

vm list

To start a VM called testvm:

vm start testvm

To stop a VM (gracefully via ACPI) called testvm:

vm stop testvm

To forcibly power off a VM called testvm:

vm poweroff testvm

To press reset on a VM called testvm:

vm reset testvm

To connect to the serial console on a VM called testvm:

vm console testvm

(to disconnect, send ~. to exit the console session)

To permanently destroy a VM called testvm (must be turned off first):

vm destroy testvm

To edit the configuration of a VM called testvm (e.g. to change the MAC address):

vm configure testvm

To increase the size of the disk on a VM called testvm to 32GB:

zfs set volsize=32G zroot/vms/testvm/disk0

To create an Ubuntu guest using a previously downloaded ISO called ubuntu.iso:

vm create -t ubuntu -s 16G testubuntu
vm install testubuntu ubuntu.iso

To make Ubuntu boot unaided each time, edit the configuration file for the VM just created (in /vms/testubuntu/testubuntu.conf) and add:

grub_run_dir=”/grub”
grub_run_file=”grub.cfg”

You can also add the above to the ubuntu template file for it to be automatically added to ubuntu VMs.

Windows guests take a lot more effort as there is no GUI console.  This also means debugging a broken windows install is almost impossible so I would discourage its use for production.  However, this is how to prepare to install Windows VMs:

fetch -o /vms/.config/BHYVE_UEFI.fd http://people.freebsd.org/%7Egrehan/bhyve_uefi/BHYVE_UEFI_20151002.fd
pkg install git p7zip cdrtools-devel
mkdir /vms/work
cd /vms/work
git clone https://github.com/nahanni/bhyve-windows-unattend-xml
fetch https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/archive-virtio/virtio-win-0.1.96/virtio-win-0.1.96.iso

This fetches a special UEFI loader, a collection of automated unattended Windows installation scripts, and the VirtIO drivers to allow storage to work.

Now copy a Windows installation ISO to /vms/work/win.iso and then:

mkdir tmp
cd tmp
7z x ../win.iso
cp /vms/work/bhyve-windows-unattend-xml/files/win2012r2_AutoUnattend.xml AutoUnattend.xml

This extracts the Windows ISO and copies an automated unattended installation script to it.  There are several AutoUnattend.xml files so choose the one matching the OS you’re installing… we’re doing a Windows 2012R2 install here.

You can edit the AutoUnattend.xml file to set a product key if required (not required for recent Windows OS)

Now we copy the VirtIO drivers into the temporary work directory and build a new ISO image:

mkdir virtio
cd virtio
tar xf /vms/work/virtio-win-0.1.96.iso
cd /vms/work/tmp
mkisofs \
-b boot/etfsboot.com -no-emul-boot -c BOOT.CAT \
-iso-level 4 -J -l -D \
-N -joliet-long \
-relaxed-filenames -v \
-V Custom -udf \
-boot-info-table -eltorito-alt-boot -eltorito-platform 0xEF \
-eltorito-boot efi/microsoft/boot/efisys_noprompt.bin \
-no-emul-boot \
-o /vms/.iso/win2k12r2.iso .

You now have a win2k12r2.iso file ready to build a windows VM:

vm create -t windows -s 64G wintest
vm install wintest win2k12r2.iso

This will start the unattended installation of windows.  It can take up to around 25 minutes.  You can monitor its progress by looking at the logfile:

tail -f /vms/wintest/vm-bhyve.log

You need to wait for it to restart twice (so three “starting bhyve” messages) – you can then get the IP from the server:

vm console wintest

and press “i” to view the IP address – this only seems to work on win2k12r2 and above.  You may need to check your DHCP server logs otherwise.

You can now RDP to the VM using Administrator and Test123 as the password.

Again, you should shutdown the VM and start it using the vm start wintest command.

To make VMs automatically start on boot, you can add them to /etc/rc.conf in the vm_list variable, for example to start wintest and testvm:

vm_list=”wintest testvm”

Enjoy.