To follow on from my post about full disk encryption (well almost), this is how to do the same but with a ZFS filesystem.  Like the other post, your /boot folder (which contains your kernel and modules) will not be encrypted, but the rest of your filesystem will be.

One disadvantage of this method is that you have to enter a passphrase for EACH disk in your ZFS system each boot.  Encryption inside ZFS will appear at some point – but until then this will suffice !

Boot from any FreeBSD 9 install medium (except bootonly), and choose Live CD at the install menu.

For the purposes of this article, I will assume that you’re using 4 disks (da0, da1, da2, da3), a 10GB /boot (this will be mirrored on each of the 4 disks), and the remaining space as a raidz1 (roughly similar to RAID5) ZFS filesystem.  The contents will be encrypted with 256-bit AES-XTS encryption with a 4 kb random data partial key and a secondary passphrase (required to type on each boot).

If your CPU supports the AESNI flag, the crypto(4) framework will utilise this too.

First we need to remove any existing GPT or MBR partition tables on each of the disks (ignore any ‘invalid argument’ messages):

gpart destroy -F da0
gpart destroy -F da1
gpart destroy -F da2
gpart destroy -F da3

Now we need to create a new GPT partition table on each disk:

gpart create -s gpt da0
gpart create -s gpt da1
gpart create -s gpt da2
gpart create -s gpt da3

We will now create a 64kb boot partition (this contains the boot loader only, so is safe and required to be unencrypted):

gpart add -s 128 -t freebsd-boot da0
gpart add -s 128 -t freebsd-boot da1
gpart add -s 128 -t freebsd-boot da2
gpart add -s 128 -t freebsd-boot da3

Next, we will create the /boot partition – you can adjust the sizes here if you need, but i’d suggest not shrinking it too much or you’ll get into problems when doing OS upgrades later… Note: this is mirrored not striped across the disks for maximum resilience – so will use 10GB on each disk for 10GB total usable space.

gpart add -s 10G -t freebsd-zfs da0
gpart add -s 10G -t freebsd-zfs da1
gpart add -s 10G -t freebsd-zfs da2
gpart add -s 10G -t freebsd-zfs da3

Finally, we will assign the remaining space on each disk to the root ZFS partition.  This will be encrypted before we build ZFS on top of it.

gpart add -t freebsd-zfs da0
gpart add -t freebsd-zfs da1
gpart add -t freebsd-zfs da2
gpart add -t freebsd-zfs da3

Now that we’ve created daXp1 (bootloader), daXp2 (/boot partition), daXp3 (root partition) – we need to write the boot loader code to each disk:

gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da0
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da1
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da2
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da3

Ok, next we will build a ramdisk to mount on /boot/zfs – this helps us mount things temporarily…

mdconfig -a -t malloc -s 128m -u 2
newfs -O2 /dev/md2
mount /dev/md2 /boot/zfs

Now we will load the modules required for ZFS and encryption:

kldload opensolaris
kldload zfs
kldload geom_eli

Next, we build a ZFS mirror for the /boot partition and mount it temporarily (to house the encryption key) – ignore any mention of unable to mount:

zpool create bootdir mirror /dev/da0p2 /dev/da1p2 /dev/da2p2 /dev/da3p2
zpool set bootfs=bootdir bootdir
mkdir /boot/zfs/bootdir
zfs set mountpoint=/boot/zfs/bootdir bootdir
zfs mount bootdir

Now we generate a random 4kb encryption key that will form (along with passphrase) the encryption key for the disk:

dd if=/dev/random of=/boot/zfs/bootdir/encryption.key bs=4096 count=1

We have everything we need to start encrypting the disks now… Enter your passphrase twice for each init phase and once again for each attach phase below:

geli init -b -B /boot/zfs/bootdir/da0p3.eli -e AES-XTS -K /boot/zfs/bootdir/encryption.key -l 256 -s 4096 /dev/da0p3
geli init -b -B /boot/zfs/bootdir/da1p3.eli -e AES-XTS -K /boot/zfs/bootdir/encryption.key -l 256 -s 4096 /dev/da1p3
geli init -b -B /boot/zfs/bootdir/da2p3.eli -e AES-XTS -K /boot/zfs/bootdir/encryption.key -l 256 -s 4096 /dev/da2p3
geli init -b -B /boot/zfs/bootdir/da3p3.eli -e AES-XTS -K /boot/zfs/bootdir/encryption.key -l 256 -s 4096 /dev/da3p3
geli attach -k /boot/zfs/bootdir/encryption.key /dev/da0p3
geli attach -k /boot/zfs/bootdir/encryption.key /dev/da1p3
geli attach -k /boot/zfs/bootdir/encryption.key /dev/da2p3
geli attach -k /boot/zfs/bootdir/encryption.key /dev/da3p3

Now that we have encrypted and mounted the partitions, we can build a ZFS root filesystem on top of it like so:

zpool create zroot raidz1 /dev/da0p3.eli /dev/da1p3.eli /dev/da2p3.eli /dev/da3p3.eli
zfs set mountpoint=/boot/zfs/zroot zroot
zfs mount zroot
zfs unmount bootdir
mkdir /boot/zfs/zroot/bootdir
zfs set mountpoint=/boot/zfs/zroot/bootdir bootdir
zfs mount bootdir

Note we unmounted the old boot mirror and re-mounted it within the root filesystem.  This will be used later to copy the kernel and modules into.

Ok, now we create all our ZFS mounts with various options as follows:

zfs set checksum=fletcher4 zroot
zfs create -o compression=on -o exec=on -o setuid=off zroot/tmp
chmod 1777 /boot/zfs/zroot/tmp
zfs create zroot/usr
zfs create zroot/usr/home
cd /boot/zfs/zroot; ln -s /usr/home home
zfs create -o compression=lzjb -o setuid=off zroot/usr/ports
zfs create -o compression=off -o exec=off -o setuid=off zroot/usr/ports/distfiles
zfs create -o compression=off -o exec=off -o setuid=off zroot/usr/ports/packages
zfs create zroot/var
zfs create -o compression=lzjb -o exec=off -o setuid=off zroot/var/crash
zfs create -o exec=off -o setuid=off zroot/var/db
zfs create -o compression=lzjb -o exec=on -o setuid=off zroot/var/db/pkg
zfs create -o exec=off -o setuid=off zroot/var/empty
zfs create -o compression=lzjb -o exec=off -o setuid=off zroot/var/log
zfs create -o compression=gzip -o exec=off -o setuid=off zroot/var/mail
zfs create -o exec=off -o setuid=off zroot/var/run
zfs create -o compression=lzjb -o exec=on -o setuid=off zroot/var/tmp
chmod 1777 /boot/zfs/zroot/var/tmp

Now we’re ready to install FreeBSD onto the new ZFS partitions.  We’re going to install the base, all sources and a generic kernel – this takes some time so please be patient…

cd /boot/zfs/zroot
unxz -c /usr/freebsd-dist/base.txz | tar xpf –
unxz -c /usr/freebsd-dist/kernel.txz | tar xpf –
unxz -c /usr/freebsd-dist/src.txz | tar xpf –

Now we can set /var/empty to readonly:

zfs set readonly=on zroot/var/empty

And now we’re ready to chroot into the installed system to setup the configuration:

chroot /boot/zfs/zroot

Now that the base system and kernel are installed, we can move our /boot folder to it’s final place on the ZFS unencrypted mirror and do a little housekeeping:

cd /
mv boot bootdir/
ln -fs bootdir/boot
mv bootdir/encryption.key bootdir/boot/
mv bootdir/*.eli bootdir/boot/

We need to setup an initial /etc/rc.conf which will mount all ZFS filesystems on boot:

echo ‘zfs_enable=”YES”‘ > /etc/rc.conf
touch /etc/fstab

And an initial /boot/loader.conf that will load ZFS, encryption and settings for encrypted disks on boot:

echo ‘vfs.zfs.prefetch_disable=”1″‘ > /boot/loader.conf
echo ‘vfs.root.mountfrom=”zfs:zroot”‘ >> /boot/loader.conf
echo ‘zfs_load=”YES”‘ >> /boot/loader.conf
echo ‘aesni_load=”YES”‘ >> /boot/loader.conf
echo ‘geom_eli_load=”YES”‘ >> /boot/loader.conf
echo ‘geli_da0p3_keyfile0_load=”YES”‘ >> /boot/loader.conf
echo ‘geli_da0p3_keyfile0_type=”da0p3:geli_keyfile0″‘ >> /boot/loader.conf
echo ‘geli_da0p3_keyfile0_name=”/boot/encryption.key”‘ >> /boot/loader.conf
echo ‘geli_da1p3_keyfile0_load=”YES”‘ >> /boot/loader.conf
echo ‘geli_da1p3_keyfile0_type=”da1p3:geli_keyfile0″‘ >> /boot/loader.conf
echo ‘geli_da1p3_keyfile0_name=”/boot/encryption.key”‘ >> /boot/loader.conf
echo ‘geli_da2p3_keyfile0_load=”YES”‘ >> /boot/loader.conf
echo ‘geli_da2p3_keyfile0_type=”da2p3:geli_keyfile0″‘ >> /boot/loader.conf
echo ‘geli_da2p3_keyfile0_name=”/boot/encryption.key”‘ >> /boot/loader.conf
echo ‘geli_da3p3_keyfile0_load=”YES”‘ >> /boot/loader.conf
echo ‘geli_da3p3_keyfile0_type=”da3p3:geli_keyfile0″‘ >> /boot/loader.conf
echo ‘geli_da3p3_keyfile0_name=”/boot/encryption.key”‘ >> /boot/loader.conf

The above settings tell the OS which encryption keyfile to use for each disk partition.

Now you can set your root password:

passwd root

And configure your timezone:

tzsetup

And setup a dummy /etc/mail/aliases file to prevent sendmail warnings:

cd /etc/mail
make aliases

Now you can configure any additional settings you require (such as adding new users, configuring networking or setting sshd to run on boot) – when you’re done, we need to exit the chroot:

exit

Now, we need to make sure the bootloader can read our ZFS pool cache (or it wont mount our ZFS disks on boot):

cd /boot/zfs
cp /boot/zfs/zpool.cache /boot/zfs/zroot/boot/zfs/zpool.cache

Finally, we need to unmount all the ZFS filesystems and configure their final mountpoints…

zfs unmount -a
zfs set mountpoint=legacy zroot
zfs set mountpoint=/tmp zroot/tmp
zfs set mountpoint=/usr zroot/usr
zfs set mountpoint=/var zroot/var
zfs set mountpoint=/bootdir bootdir

Now we can ‘reboot’ and remove the media while the computer reboots.  Do this as soon as you can.

The computer should reboot into a ZFS-based filesystem, booted from a software RAID array on fully protected disks with all but /boot partition encrypted.  Note: it will ask you to enter a passphrase for each disk parition used above (4 times) – you should take care to enter the correct passwords as it will treat any passwords missed as a failed disk (you get 3 attempts at each password)

Once it’s booted, you can login and run sysinstall to configure other options like networking and startup programs (like SSH!)

The only point to note is that when you do an OS upgrade, during the “mergemaster” stage, it will complain that /boot is a symlink not a directory.  Simply tell it to ignore/do nothing and it will install the files as normal.

Enjoy!

The default console in FreeBSD is 80 x 25 geometry in whatever default resolution it can choose… but sometimes you want more.

So here’s how to set a new resolution…

First, you need to load the VESA driver into the kernel (if it’s not already loaded) – ignore any errors…

kldload vesa

Next, you need to query the console to find out what modes are available, to do this type (as root):

vidcontrol -i mode < /dev/console

This will output all the available modes for your card+monitor.  An example of one of the lines on my card is as follows:

280 (0x118) 0x0000001f G 1024x768x32 D   8×16  0xa0000 64k 64k 0xf3000000 3072k

(mode 280 is 1024 x 768 in 32-bit colour and uses a 8×16 font)

This is the mode I want to use, and I want to increase my geometry from 80×25 to 132×60, so I can type the following command to one-time set it:

vidcontrol -g 132×60 MODE_280 < /dev/console

If you want to use this as the default every boot, add the following line to your /etc/rc.conf file:

allscreens_flags=”-g 132×60 MODE_280″

And also load the VESA driver on boot, by adding the following line to your /boot/loader.conf file:

vesa_load=”YES”

That’s all there is to it.

This article will tell you how to fully encrypt your hard disk in FreeBSD 9.x.  When I say ‘fully’, I mean as close as possible.  It will leave the bootloader and /boot folder unencrypted, but everything else will be encrypted (including your swap space).  Basically, all your data is encrypted and that’s the point…

Boot from any FreeBSD 9 install medium (except bootonly), and choose Live CD at the installer menu.

For this article, I will assume you’re using the /dev/ada0 disk, a 10GB /boot, a 4GB swap and remaining disk encrypted.  The contents will be encrypted using AES-XTS 256-bit encryption  with a 4 kilobit random data partial key and a passphrase (required to type on boot).  This method requires no external data (no USB sticks, no bootable CDs to boot the OS) – but does not offer two factor authentication which is better than this method.  For general encryption needs, this method is more than sufficient.

Note that more recent CPUs support AESNI flag for offloading. As GELI uses the crypto(4) framework, the OS will utilise this function of your CPU to assist the encryption to decrease CPU load.

First, we need to remove any existing GPT or MBR partition tables on the disk – ignore any ‘invalid argument’ messages you get at this stage:

gpart destroy -F ada0

Now we need to initialise a new GPT partition table, as follows:

gpart create -s gpt ada0

We will now create a 64kb boot partition (this contains the boot loader only, so is safe and required to be unencrypted):

gpart add -s 128 -t freebsd-boot ada0

Next, we will create the /boot partition – you can adjust the sizes here if you need, but i’d suggest not shrinking it too much or you’ll get into problems when doing OS upgrades later…

gpart add -s 10G -t freebsd-ufs ada0

Now for a swap partition.  Again, you can adjust the size if needed.  This will be encrypted during boot with a one-time 256bit key.

gpart add -s 4G -t freebsd-swap ada0

Finally, we assign the remaining data to a partition.  This will form the entire disk (excluding /boot) and will be encrypted shortly.

gpart add -t freebsd-ufs ada0

OK, so we’ve created… ada0p1 (bootloader), adap2 (unencrypted /boot partition), adap3 (swap partition) and adap4 (encrypted disk partition).  We need to write the boot loader to the disk now:

gpart bootcode -b /boot/pmbr -p /boot/gptboot -i 1 ada0

Now we need to format the /boot partition:

newfs -O2 -U -m 0 -j /dev/ada0p2

And temporarily mount it as /mnt:

mount /dev/ada0p2 /mnt

Now we will create a 4kb random data file that will form part of the encryption key:

dd if=/dev/random of=/mnt/encryption.key bs=4096 count=1

Now we’re in a position to encrypt the main disk.  This part will ask for a passphrase twice to complete:

kldload geom_eli
geli init -a HMAC/SHA256 -b -B /mnt/ada0p4.eli -e AES-XTS -K /mnt/encryption.key -l 256 -s 4096 /dev/ada0p4
geli attach -k /mnt/encryption.key /dev/ada0p4

You will receive some messages on the console about checksum mismatches – this is normal and please ignore them.

If you have time, I recommend writing the entire disk with random data to initialise the checksums.  This is a VERY time consuming step – you can skip it if you wish, but it is recommended:

dd if=/dev/random of=/dev/ada0p4.eli bs=1m

Next we will unmount the old parition so we can mount the new “root” after formatting  (and re-mount the /boot partition too):

umount /mnt
newfs -O2 -U -j -m 6 /dev/ada0p4.eli
mount /dev/ada0p4.eli /mnt
mkdir /mnt/bootdir
mount /dev/ada0p2 /mnt/bootdir

OK, we’re ready to install the OS files onto the disk now… We will install the base, kernel, src and ports tarballs as follows:

cd /mnt
unxz -c /usr/freebsd-dist/base.txz | tar xpf –
unxz -c /usr/freebsd-dist/kernel.txz | tar xpf –
unxz -c /usr/freebsd-dist/src.txz | tar xpf –
unxz -c /usr/freebsd-dist/ports.txz | tar xpf –

Note: this can take a while (especially the ports extraction) so please be patient.  If you’d like to see some kind of progress, change the “xpf” to “xvpf” and it will scroll the files to the screen as they are extracted.

Now we have to move the /boot folder to the unencrypted partition (it’s really not much use if it’s encrypted!) – we will also move the keyfile and backup file into the /boot folder:

mv boot bootdir/
ln -fs bootdir/boot
mv encryption.key ada0p4.eli bootdir/boot/

Now we need to prepare a few things in the installed OS – so we will chroot into the folder:

chroot /mnt

We need to tell the boot loader to load kernel modules for decryption, and also tell it about the keyfile for the partition… edit the file /boot/loader.conf and enter the following:

vfs.root.mountfrom=”ufs:/dev/ada0p4.eli”
aesni_load=”YES”
geom_eli_load=”YES”
geli_ada0p4_keyfile0_load=”YES”
geli_ada0p4_keyfile0_type=”ada0p4:geli_keyfile0″
geli_ada0p4_keyfile0_name=”/boot/encryption.key”

Now we need to tell the system to encrypt our swap space using a one-time key on each boot (note: this prevents system dumps from working)… edit /etc/rc.conf and enter:

geli_swap_flags=”-e AES-XTS -l 256 -s 4096 -d”

Next we need to tell the system our mountpoints… edit the file /etc/fstab and enter:

# Device                   Mountpoint FStype Options Dump Pass#
/dev/ada0p4.eli   /                   ufs    rw      0    0
/dev/ada0p2         /bootdir   ufs    rw      1    1
/dev/ada0p3.eli   none           swap   sw      0    0

Now we need to initialise a few things… let’s start by setting the root password:

passwd root

And configuring your timezone:

tzsetup

And initialise the sendmail aliases file:

cd /etc/mail
make aliases

You can do any other system setup you need now, such as adding users, configuring SSH or networking…  when you’re done:

exit

Now we’re done, we can reboot…

reboot

On boot, you will see a prompt for:

Enter passphrase for ada0p4:

Note, however, that devices are still being detected while this occurs so it may scroll off the screen (usually while detecting USB devices) – this doesn’t affect your ability to enter the passphrase, but can be confusing if you’re not expecting it!

Once the system is up and running, you can use it as normal.

The only point to note is that when you do an OS upgrade, during the “mergemaster” stage, it will complain that /boot is a symlink not a directory.  Simply tell it to ignore/do nothing and it will install the files as normal.

You can use portupgrade to keep ports up to date in FreeBSD.  It is best to do this often to avoid as many problems as possible.

It sometimes causes a problem and you should be prepared to resolve any problems it may cause.  You have been warned!

First, if you have not installed portupgrade, you can install it from packages or ports.  To install it from packages, use:

pkg_add -r portupgrade

Or from ports, use:

cd /usr/ports/ports-mgmt/portupgrade
make install distclean

Assuming you have portupgrade installed, you should first upgrade your ports collection.  I have detailed how to do this in this post.

Once the ports tree is updated (it takes a while), you can check which ports/packages require upgrading with the following command:

pkg_version -v | grep -v up-to-date

This will show you 1 line per package that needs upgrading.  On the left, the installed package and version – and on the right the new version.

To upgrade all ports/packages (this takes time as it will download and compile everything), use the following command:

portupgrade -rv \*

Depending on your shell, you may or may not require the escape character ( \ ) before the *.

Beware that the uninstall of some ports will shutdown daemons but the installer will not restart them.  A common example is the MySQL server port.  A simple solution is to reboot the server once done.

Any problems will be reported at the end of the process.  As I explained above, you may need to resolve some issues manually and this can be complex!

Sometimes, it’s preferrably to use a swap file instead of a swap partition in FreeBSD.  It can be useful if you boot from a ZFS raidz1/2/mirror root, or if you just need to add additional swap space to an existing server.

Whatever your reasons for wanting to do it, here’s how to do it using FreeBSD 8.x and 9.x

I will create a 4GB swap file, but you can create any size you want by adjusting the figure in the dd command.

First, we need to create a swapfile of the desired size:

dd if=/dev/zero of=/swapfile.dat bs=1m count=4096

This will create /swapfile.dat with 4096 x 1MB blocks (4GB) – next change the permissions on it as follows:

chmod 0600 /swapfile.dat

And finally, add the following line to your /etc/rc.conf file:

swapfile=”/swapfile.dat”

Upon next reboot, your server will create a virtual node (using mdconfig) to the swapfile and add it as swapspace.

To activate it without rebooting, you can run:

/etc/rc.d/addswap start

It should be noted that swapfiles wont be used for kernel crashdumps.  These need to be partitions in order to work.

This is how to make freebsd boot from a ZFS volume (whether it be raid0, raid5 or raid6).  There are rumours that a future installer will support ZFS – but this was not available for the 9.0-RELEASE, so we have to do this manually.

If you’re using FreeBSD 8.x then follow the guide at https://www.dan.me.uk/blog/2010/02/08/booting-from-zfs-raid0156-in-freebsd/

First, grab yourself a copy of DVD1 iso or the memory stick image and boot from it.  No other boot image will work – it MUST be the DVD or memory stick image!

Once the installer loads up, choose ‘Live CD’ and login with ‘root’

For my example, i’m going to build a RAID5 array on disks da0 da1 and da2.

First, we need to remove any existing GPT partition info from the disks – ignore the ‘invalid argument’ message if you get it at this stage:

gpart destroy da0
gpart destroy da1
gpart destroy da2

Now we need to initialise the GPT partitions on each disk:

gpart create -s gpt da0
gpart create -s gpt da1
gpart create -s gpt da2

We will now make a boot (64KB) and ZFS (remaining space) partition on each disk in turn:

gpart add -s 128 -t freebsd-boot da0
gpart add -s 128 -t freebsd-boot da1
gpart add -s 128 -t freebsd-boot da2

gpart add -t freebsd-zfs -l disk0 da0
gpart add -t freebsd-zfs -l disk1 da1
gpart add -t freebsd-zfs -l disk2 da2

And now we have to install the protected MBR boot code into all the drives:

gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da0
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da1
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da2

Now that we’ve configured the disks, we need to load the ZFS kernel modules from the CD so that we can build ZFS volumes:

kldload opensolaris
kldload zfs

And create a ZFS pool.  If you want a RAID6 volume, choose raidz2 instead of raidz1 here.  If you want a mirror, use mirror or if you want RAID0 (or single disk) just omit the raidz1 completely:

zpool create zroot raidz1 /dev/gpt/disk0 /dev/gpt/disk1 /dev/gpt/disk2
zpool set bootfs=zroot zroot

Ok, now we’ve made our ZFS pool (and it complained about not being able to mount as /zroot) – we need to mount it, then make all our filesystems on it… this is complicated, but here we go:

mdconfig -a -t malloc -s 128m -u 2
newfs -O2 /dev/md2
mount /dev/md2 /boot/zfs
zfs set mountpoint=/boot/zfs/zroot zroot
zfs mount zroot
zfs set checksum=fletcher4 zroot
zfs create -o compression=on -o exec=on -o setuid=off zroot/tmp
chmod 1777 /boot/zfs/zroot/tmp
zfs create zroot/usr
zfs create zroot/usr/home
cd /boot/zfs/zroot; ln -s /usr/home home
zfs create -o compression=lzjb -o setuid=off zroot/usr/ports
zfs create -o compression=off -o exec=off -o setuid=off zroot/usr/ports/distfiles
zfs create -o compression=off -o exec=off -o setuid=off zroot/usr/ports/packages
zfs create zroot/var
zfs create -o compression=lzjb -o exec=off -o setuid=off zroot/var/crash
zfs create -o exec=off -o setuid=off zroot/var/db
zfs create -o compression=lzjb -o exec=on -o setuid=off zroot/var/db/pkg
zfs create -o exec=off -o setuid=off zroot/var/empty
zfs create -o compression=lzjb -o exec=off -o setuid=off zroot/var/log
zfs create -o compression=gzip -o exec=off -o setuid=off zroot/var/mail
zfs create -o exec=off -o setuid=off zroot/var/run
zfs create -o compression=lzjb -o exec=on -o setuid=off zroot/var/tmp
chmod 1777 /boot/zfs/zroot/var/tmp

Now we’re ready to install FreeBSD onto the new ZFS partitions.  We’re going to install the base, all sources and a generic kernel – this takes some time so be patient…

cd /boot/zfs/zroot
unxz -c /usr/freebsd-dist/base.txz | tar xpf –
unxz -c /usr/freebsd-dist/kernel.txz | tar xpf –
unxz -c /usr/freebsd-dist/src.txz | tar xpf –

Now we need to set /var/empty to readonly:

zfs set readonly=on zroot/var/empty

And now we’re ready to configure the installation.  To make things easier, we will chroot into the environment:

chroot /boot/zfs/zroot

We need to setup an initial /etc/rc.conf which will mount all ZFS filesystems:

echo ‘zfs_enable=”YES”‘ > /etc/rc.conf
touch /etc/fstab

And an initial /boot/loader.conf that will load the ZFS modules and set our root mountpoint:

echo ‘vfs.zfs.prefetch_disable=”1″‘ > /boot/loader.conf
echo ‘vfs.root.mountfrom=”zfs:zroot”‘ >> /boot/loader.conf
echo ‘zfs_load=”YES”‘ >> /boot/loader.conf

Now you can set your root password:

passwd root

And configure your timezone:

tzsetup

And setup a dummy aliases file for sendmail to keep it quiet 😉

cd /etc/mail
make aliases

You can do other configuration here, like adding a user etc – but when you’re done we can exit the environment:

exit

Now, we need to export our ZFS configuration (and reimport it) so we can save out the cache file:

cd /boot/zfs
zpool export zroot && zpool import zroot
cp /boot/zfs/zpool.cache /boot/zfs/zroot/boot/zfs/zpool.cache

This is the tricky part, we need to unmount the ZFS partitions and re-assign their mountpoints for the root filesystems:

zfs unmount -a
zfs set mountpoint=legacy zroot
zfs set mountpoint=/tmp zroot/tmp
zfs set mountpoint=/usr zroot/usr
zfs set mountpoint=/var zroot/var

Now we can ‘reboot’ and remove the media while the computer reboots.  Do this as soon as you can.

The computer should reboot into a ZFS-based filesystem, booted from a software RAID array on fully protected disks.

Once it’s booted, you can login and run sysinstall to configure other options like networking and startup programs (like SSH!)

Enjoy!