Daily Archives: January 22, 2012

Upgrading software (ports/packages) in FreeBSD

You can use portupgrade to keep ports up to date in FreeBSD.  It is best to do this often to avoid as many problems as possible.

It sometimes causes a problem and you should be prepared to resolve any problems it may cause.  You have been warned!

First, if you have not installed portupgrade, you can install it from packages or ports.  To install it from packages, use:

pkg_add -r portupgrade

Or from ports, use:

cd /usr/ports/ports-mgmt/portupgrade
make install distclean

Assuming you have portupgrade installed, you should first upgrade your ports collection.  I have detailed how to do this in this post.

Once the ports tree is updated (it takes a while), you can check which ports/packages require upgrading with the following command:

pkg_version -v | grep -v up-to-date

This will show you 1 line per package that needs upgrading.  On the left, the installed package and version – and on the right the new version.

To upgrade all ports/packages (this takes time as it will download and compile everything), use the following command:

portupgrade -rv \*

Depending on your shell, you may or may not require the escape character ( \ ) before the *.

Beware that the uninstall of some ports will shutdown daemons but the installer will not restart them.  A common example is the MySQL server port.  A simple solution is to reboot the server once done.

Any problems will be reported at the end of the process.  As I explained above, you may need to resolve some issues manually and this can be complex!

Using a swap file instead of swap partition in FreeBSD 8.x/9.x

Sometimes, it’s preferrably to use a swap file instead of a swap partition in FreeBSD.  It can be useful if you boot from a ZFS raidz1/2/mirror root, or if you just need to add additional swap space to an existing server.

Whatever your reasons for wanting to do it, here’s how to do it using FreeBSD 8.x and 9.x

I will create a 4GB swap file, but you can create any size you want by adjusting the figure in the dd command.

First, we need to create a swapfile of the desired size:

dd if=/dev/zero of=/swapfile.dat bs=1m count=4096

This will create /swapfile.dat with 4096 x 1MB blocks (4GB) – next change the permissions on it as follows:

chmod 0600 /swapfile.dat

And finally, add the following line to your /etc/rc.conf file:

swapfile=”/swapfile.dat”

Upon next reboot, your server will create a virtual node (using mdconfig) to the swapfile and add it as swapspace.

To activate it without rebooting, you can run:

/etc/rc.d/addswap start

It should be noted that swapfiles wont be used for kernel crashdumps.  These need to be partitions in order to work.

Booting from ZFS RAID0/1/5/6 in FreeBSD 9.0-RELEASE

This is how to make freebsd boot from a ZFS volume (whether it be raid0, raid5 or raid6).  There are rumours that a future installer will support ZFS – but this was not available for the 9.0-RELEASE, so we have to do this manually.

If you’re using FreeBSD 8.x then follow the guide at https://www.dan.me.uk/blog/2010/02/08/booting-from-zfs-raid0156-in-freebsd/

First, grab yourself a copy of DVD1 iso or the memory stick image and boot from it.  No other boot image will work – it MUST be the DVD or memory stick image!

Once the installer loads up, choose ‘Live CD’ and login with ‘root’

For my example, i’m going to build a RAID5 array on disks da0 da1 and da2.

First, we need to remove any existing GPT partition info from the disks – ignore the ‘invalid argument’ message if you get it at this stage:

gpart destroy da0
gpart destroy da1
gpart destroy da2

Now we need to initialise the GPT partitions on each disk:

gpart create -s gpt da0
gpart create -s gpt da1
gpart create -s gpt da2

We will now make a boot (64KB) and ZFS (remaining space) partition on each disk in turn:

gpart add -s 128 -t freebsd-boot da0
gpart add -s 128 -t freebsd-boot da1
gpart add -s 128 -t freebsd-boot da2

gpart add -t freebsd-zfs -l disk0 da0
gpart add -t freebsd-zfs -l disk1 da1
gpart add -t freebsd-zfs -l disk2 da2

And now we have to install the protected MBR boot code into all the drives:

gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da0
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da1
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da2

Now that we’ve configured the disks, we need to load the ZFS kernel modules from the CD so that we can build ZFS volumes:

kldload opensolaris
kldload zfs

And create a ZFS pool.  If you want a RAID6 volume, choose raidz2 instead of raidz1 here.  If you want a mirror, use mirror or if you want RAID0 (or single disk) just omit the raidz1 completely:

zpool create zroot raidz1 /dev/gpt/disk0 /dev/gpt/disk1 /dev/gpt/disk2
zpool set bootfs=zroot zroot

Ok, now we’ve made our ZFS pool (and it complained about not being able to mount as /zroot) – we need to mount it, then make all our filesystems on it… this is complicated, but here we go:

mdconfig -a -t malloc -s 128m -u 2
newfs -O2 /dev/md2
mount /dev/md2 /boot/zfs
zfs set mountpoint=/boot/zfs/zroot zroot
zfs mount zroot
zfs set checksum=fletcher4 zroot
zfs create -o compression=on -o exec=on -o setuid=off zroot/tmp
chmod 1777 /boot/zfs/zroot/tmp
zfs create zroot/usr
zfs create zroot/usr/home
cd /boot/zfs/zroot; ln -s /usr/home home
zfs create -o compression=lzjb -o setuid=off zroot/usr/ports
zfs create -o compression=off -o exec=off -o setuid=off zroot/usr/ports/distfiles
zfs create -o compression=off -o exec=off -o setuid=off zroot/usr/ports/packages
zfs create zroot/var
zfs create -o compression=lzjb -o exec=off -o setuid=off zroot/var/crash
zfs create -o exec=off -o setuid=off zroot/var/db
zfs create -o compression=lzjb -o exec=on -o setuid=off zroot/var/db/pkg
zfs create -o exec=off -o setuid=off zroot/var/empty
zfs create -o compression=lzjb -o exec=off -o setuid=off zroot/var/log
zfs create -o compression=gzip -o exec=off -o setuid=off zroot/var/mail
zfs create -o exec=off -o setuid=off zroot/var/run
zfs create -o compression=lzjb -o exec=on -o setuid=off zroot/var/tmp
chmod 1777 /boot/zfs/zroot/var/tmp

Now we’re ready to install FreeBSD onto the new ZFS partitions.  We’re going to install the base, all sources and a generic kernel – this takes some time so be patient…

cd /boot/zfs/zroot
unxz -c /usr/freebsd-dist/base.txz | tar xpf –
unxz -c /usr/freebsd-dist/kernel.txz | tar xpf –
unxz -c /usr/freebsd-dist/src.txz | tar xpf –

Now we need to set /var/empty to readonly:

zfs set readonly=on zroot/var/empty

And now we’re ready to configure the installation.  To make things easier, we will chroot into the environment:

chroot /boot/zfs/zroot

We need to setup an initial /etc/rc.conf which will mount all ZFS filesystems:

echo ‘zfs_enable=”YES”‘ > /etc/rc.conf
touch /etc/fstab

And an initial /boot/loader.conf that will load the ZFS modules and set our root mountpoint:

echo ‘vfs.zfs.prefetch_disable=”1″‘ > /boot/loader.conf
echo ‘vfs.root.mountfrom=”zfs:zroot”‘ >> /boot/loader.conf
echo ‘zfs_load=”YES”‘ >> /boot/loader.conf

Now you can set your root password:

passwd root

And configure your timezone:

tzsetup

And setup a dummy aliases file for sendmail to keep it quiet 😉

cd /etc/mail
make aliases

You can do other configuration here, like adding a user etc – but when you’re done we can exit the environment:

exit

Now, we need to export our ZFS configuration (and reimport it) so we can save out the cache file:

cd /boot/zfs
zpool export zroot && zpool import zroot
cp /boot/zfs/zpool.cache /boot/zfs/zroot/boot/zfs/zpool.cache

This is the tricky part, we need to unmount the ZFS partitions and re-assign their mountpoints for the root filesystems:

zfs unmount -a
zfs set mountpoint=legacy zroot
zfs set mountpoint=/tmp zroot/tmp
zfs set mountpoint=/usr zroot/usr
zfs set mountpoint=/var zroot/var

Now we can ‘reboot’ and remove the media while the computer reboots.  Do this as soon as you can.

The computer should reboot into a ZFS-based filesystem, booted from a software RAID array on fully protected disks.

Once it’s booted, you can login and run sysinstall to configure other options like networking and startup programs (like SSH!)

Enjoy!