Sheepdog on EL6

AN!Wiki :: How To :: Sheepdog on EL6

Warning: This is essentially just a compilation of notes I made while playing with NTT's Sheepdog. It may or may not become a complete tutorial someday.

Installing, configuring and using Sheepdog in EL6.

Base Cluster Config

Note: Replace bindnetaddr's value with the IP address of the interface you want the cluster communication on (BCN, generally).

vim /etc/corosync/corosync.conf

# Please read the corosync.conf 5 manual page
compatibility: whitetank
totem {
  version: 2
  secauth: off
  threads: 0
  # Note, fail_recv_const is only needed if you're 
  # having problems with corosync crashing under 
  # heavy sheepdog traffic. This crash is due to 
  # delayed/resent/misordered multicast packets. 
  # fail_recv_const: 5000
  interface {
    ringnumber: 0
    bindnetaddr: 192.168.3.{3..7}
    mcastaddr: 226.94.1.1
    mcastport: 5405
  }
}
logging {
  fileline: off
  to_stderr: no
  to_logfile: yes
  to_syslog: yes
  # the pathname of the log file
  logfile: /var/log/cluster/corosync.log
  debug: off
  timestamp: on
  logger_subsys {
    subsys: AMF
    debug: off
  }
}
amf {
  mode: disabled
}

On one machine, in the first terminal run;

corosync-keygen

This will ask for entropy. Typing at the keyboard is tedious, so open a second terminal to the node and then do other things (download files, dd something, etc). Once complete, you will have /etc/corosync/authkey. Pull this to all other nodes.

rsync -av root@an-node03:/etc/corosync/authkey /etc/corosync/

Install

Done as root (ya, I know...) on CentOS 6.0;

Install dependencies;

yum -y install nss-devel zlib-devel

Corosync from source;

yum -y remove corosync corosynclib corosynclib-devel
cd ~
git clone git://github.com/corosync/corosync.git
cd corosync
git checkout -b flatiron origin/flatiron
./autogen.sh
./configure --enable-nss
make install
cd ~

QEMU (needs >= v0.13);

cd ~
yum -y remove libvirt
git clone git://git.sv.gnu.org/qemu.git
cd qemu
./configure
make install
cd ~

Sheepdog (we don't build the RPMs for missing dependencies reasons);

cd ~
git clone git://github.com/collie/sheepdog.git
cd sheepdog
./autogen.sh
./configure
make install
cd ~

Using Sheepdog

You need a "store" directory. I use /sheepdog formatted on /dev/sda5 as an ext4 partition.

mkfs.ext4 /dev/sda5
mkdir /sheepdog
tune2fs -l /dev/sda5 | grep "Filesystem UUID" | sed -e "s/^Filesystem UUID:\s*\(.*\)/UUID=\L\1\E\t\/sheepdog\text4\tdefaults,user_xattr\t1 3/" >> /etc/fstab
mount /sheepdog
mount

/dev/sda2 on / type ext3 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
tmpfs on /dev/shm type tmpfs (rw)
/dev/sda1 on /boot type ext3 (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
/dev/sda5 on /sheepdog type ext4 (rw,user_xattr)

Note the (rw,user_xattr), this is important.

Now create the store.

 

Any questions, feedback, advice, complaints or meanderings are welcome.
Alteeve's Niche!	Enterprise Support: Alteeve Support	Community Support
© Alteeve's Niche! Inc. 1997-2024		Anvil! "Intelligent Availability®" Platform
legal stuff: All info is provided "As-Is". Do not use anything here unless you are willing and able to take responsibility for your own actions.