Monthly Archives: May 2009

CARP in FreeBSD (HSRP/VRRP alternative)

Leave a Reply

CARP can be used for several things, but I’m going to concentrate on a HSRP/VRRP alternative in this post (having a hot standby on a seperate computer).

To enable CARP support, you need to add the following line in your kernel config file (in /usr/src/sys/<arch>/conf/):

device carp

Once you have built and installed your new kernel, you can configure a CARP interface.  To use carp, you need a non-carp IP on your interface on each computer and then a second shared IP.  For this purpose we will use 192.168.0.1 as the shared IP and 192.168.0.2 as the fixed non-carp IP on the fxp0 interface.

Edit your /etc/rc.conf file as follows:

ifconfig_fxp0=”inet 192.168.0.2 netmask 255.255.255.0″

cloned_interfaces=”carp0″
ifconfig_carp0=”vhid 1 advskew 100 pass p4ssw0rd 192.168.0.1/24″

To explain the carp0 line in more detail…

The vhid contains the virtual interface ID.  There seems to be some confusion online whether this should be identical or different on machines.  Personally, I use the same vhid on primary and backup hosts.

The advskew value specifies which should be primary and backup.  A lower figure is a higher priority so the master should be a lower number than the secondary.

The pass value is the authentication value.  This needs to match between primary and backup machines.  The password is sent plaintext over the interface, so is easily snooped unfortunately.

Finally, the shared IP is listed.  This needs to match an existing subnet on the machine or the carp interface will not be configured.

An example of a backup to the above machine (using 192.168.0.3 as its non-carp IP) would have the following config:

ifconfig_fxp0=”inet 192.168.0.3 netmask 255.255.255.0″

cloned_interfaces=”carp0″
ifconfig_carp0=”vhid 1 advskew 200 pass p4ssw0rd 192.168.0.1/24″

As you can see, it is identical except fxp0 has a different IP, and the advskew is higher making the priority lower.

To check on the status, you run ifconfig carp0 – it should look similar to the following:

carp0: flags=49<UP,LOOPBACK,RUNNING> metric 0 mtu 1500
inet 192.168.0.1 netmask 0xffffff00
carp: MASTER vhid 1 advbase 1 advskew 100

That should be all you need to configure carp on a FreeBSD machine.

Alternative to null-routing in FreeBSD (using IPFW)

Leave a Reply

Instead of using null-routing, you can use IPFW to block the traffic (the advantages include being able to set the ICMP response type).  My favourite is to use “Communication prohibited by filter” response.

If you wanted to block 192.168.0.1 in this way, you would use:

/sbin/ipfw add 01000 unreach filter-prohib ip from 192.168.0.1 to me

You can also adapt the above to only include certain types of traffic which is where it is more flexible than null-routing.

Null Routing in FreeBSD

2 Replies

Ok Karl… Null routing – real easy 🙂

If you wanted to null route a single IP (192.168.0.1), you would run (as root):

/sbin/route add 192.168.0.1 127.0.0.1 -blackhole

If you wanted to null route a block of IPs (192.168.0.0/24), then use:

/sbin/route add -net 192.168.0.0/24 127.0.0.1 -blackhole

If you would rather generate a “Destination Host Unreachable” ICMP response instead of blackholing the traffic, replace -blackhole with -reject

RTG FreeBSD Startup Script

Leave a Reply

Ok Karl, here you go… Once you have installed RTG on your server from FreeBSD ports and configured it, you can add this script to your /usr/local/etc/rc.d/ folder.  Make sure it runs AFTER your mysql server starts or it will be useless!

#!/bin/sh

case “$1” in
‘start’)
rm /tmp/rtgpoll.pid
/usr/local/bin/rtgpoll -c /usr/local/etc/rtg/rtg.conf -t /usr/local/etc/rtg/targets.cfg &
;;
‘stop’)
;;
*)
echo “Please specify ‘start’ or ‘stop'”
;;
esac

Interface Renaming in FreeBSD

4 Replies

Ok, this can be a really useful feature.  Renaming network interfaces in freebsd to something more useful.

This requires FreeBSD 7.  We are going to rename fxp0 to net0.  Edit /etc/rc.conf and add:

ifconfig_fxp0_name=”net0″
ifconfig_net0=”inet 10.0.0.1 netmask 255.0.0.0″

Yes – it’s that simple 🙂

Cisco FastEtherChannel in FreeBSD

1 Reply

To configure Cisco EtherChannel in FreeBSD 7 is relatively simple.  We will assume interfaces fxp0 and fxp1 are to be configured into a 200mbps Cisco EtherChannel.

First, we have to load the netgraph modules to support this.  Edit /boot/loader.conf and add:

netgraph_load=”YES”
ng_socket_load=”YES”
ng_fec_load=”YES”

Now, the rest of the configuration is done in /etc/rc.conf as follows:

ifconfig_fxp0=”up promisc”
ifconfig_fxp1=”up promisc”

fec_interfaces=”fec0″
fecconfig_fec0=”fxp0 fxp1″

ifconfig_fec0=”inet 10.0.0.1 netmask 255.0.0.0″

Now, this also requires configuration on the Cisco Switch they plug into.  In my configuration, this is a Cisco 3550-24-EMI configured as follows:

interface Port-channel1
description fec0
switchport access vlan 100
switchport mode access
switchport nonegotiate
!
interface FastEthernet0/1
description fec0 [1/2]
switchport access vlan 100
switchport mode access
switchport nonegotiate
channel-group 1 mode on
!
interface FastEthernet0/2
description fec0 [2/2]
switchport access vlan 100
switchport mode access
switchport nonegotiate
channel-group 1 mode on

I’ve been running this configuration for over a year now with no problems at all, and have tested greater than 100mbps speed to it.

Failover Network Interfaces in FreeBSD

Leave a Reply

Ok, so I wanted a way for my FreeBSD server to be unaffected during a switch failure.  In the past, I had used etherchannel to aggregate multiple ports together – but they had always been to the same switch (and involved switch configuration).  In FreeBSD 7, it’s pretty simple to setup a basic link-state failover connection.

In my configuration, I am using an Intel Pro/1000T Server card as my primary link (interface name “em0”), and a Broadcom Gigabit Ethernet onboard card as my backup link (interface name “bge0”).

To setup failover,we need to load the driver.  Edit /boot/loader.conf and add:

if_lagg_load=”YES”

Now we can do everything in /etc/rc.conf with the following configuration:

ifconfig_em0=”up”
ifconfig_bge0=”up”
cloned_interfaces=”lagg0″
ifconfig_lagg0=”laggproto failover laggport em0 laggport bge0″
ifconfig_lagg0_alias0=”inet 10.0.0.1 netmask 255.0.0.0″

The first interface listed for laggport becomes the primary.  When we check ifconfig, we can see:

bge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
media: Ethernet autoselect (100baseTX <full-duplex>)
status: active
lagg: laggdev lagg0

em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
media: Ethernet autoselect (100baseTX <full-duplex>)
status: active
lagg: laggdev lagg0

lagg0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
inet 10.0.0.1 netmask 0xff000000 broadcast 10.0.0.255
media: Ethernet autoselect
status: active
laggproto failover
laggport: bge0 flags=0<>
laggport: em0 flags=5<MASTER,ACTIVE>

If you unplug the cable to em0, the server handles the failover automatically to bge0 with the loss of only a couple of packets – more than acceptable!

FreeBSD vmware tools in FreeBSD 8.x

3 Replies

You can install vmware tools under ESX – but I like to edit part of it.

NOTE: if you’re running FreeBSD 9, you need to follow the guide here instead.

To install, connect with your VM Infrastucture Client, edit the CD Drive, select ‘Datastore ISO file’, and Browse.  Open up the ‘vmimages’ folder and select ‘freebsd.iso’.  Make sure to check the ‘Connected’ checkbox before clicking OK.

In freebsd, as root, do the following:

mount /cdrom
cd /tmp
gunzip -c /cdrom/vmware<tab> | tar xvf –
umount /cdrom
cd vmware<tab>
./vmware-install<tab>

This will start the vmware installer… follow the instructions and select all the defaults.  Once it is complete, you can remove the folder in /tmp/

NOTE: vmtools requires perl to be installed and also “compat6x” port/package if you are using FreeBSD 7 – do this beforehand!

now… edit /usr/local/etc/rc.d/vmware-tools.sh and search for “–background”.  This should be around line 626.  At the end of that line, add:

–halt-command “/sbin/shutdown -p now”

then restart the vmware tools with:

/usr/local/etc/rc.d/vmware-tools.sh restart

This will ensure that, when you use ‘shutdown guest’, the freebsd virtual machine will power off.  Without this update, it will shutdown but not power off.

FreeBSD with ESX

Leave a Reply

If you run FreeBSD under ESX (unsupported, but works) and you use a type of storage that is susceptable to slowdowns (in my case a SAN with a failed hard drive forced to do RAID5 reconstruction), you may find your FreeBSD virtual machines panic and crash.

This is caused by the SCSI disk timeout being reached on the root device.  You can spot these by looking in your log for SCSI timeout messages.

To get around this, we can increase the number of retries before it fails (hopefully giving the storage device time to catch up).  This is done with the following command:

sysctl kern.cam.da.retry_count=120

You can set this to happen each boot, by editing the file /etc/sysctl.conf and adding:

kern.cam.da.retry_count=120