Fencing KVM Virtual Servers
{(howto_header}}
It's difficult to image a production use-case for clusters built on virtual machines. As a learning and test bed though, clusters of virtual machines can be invaluable.
Clusters of VMs still require fencing, just the same. Generally this is accomplished by using a fence agent that calls the hypervisor and asks it to force the target VM off.
Each hypervisor will have it's own methods for doing this. In this tutorial, we will discuss fencing virtual machines running on the KVM hypervisor using the fence_virtd daemon and the fence_xvm guest fence agent.
Note: This tutorial was based on Clusterlabs.org's "Guest Fencing" tutorial. Thanks to Andrew Beekhof for writing this and helping me sort out some problems I ran into. |
Steps
Setting up fencing requires two steps;
- Configure the host's fence_virtd to listen for fence requests from guests.
- Configure the guests to talk to the host's daemon using their fence_xvm fence agents.
There is no need for the guest and host operating systems to match. So the specific steps for each of these two steps will vary somewhat depending on which OS you are using.
Host Configuration
This tutorial will use Fedora 18 as the host operating system.
Fedora 18
Configuring the host is a three step process;
- Install the needed components.
- Create a key.
- Configure the fence_virtd daemon.
Fedora 18 Host; Installing Components
The first step is to install the fence_virtd components;
yum install fence-virt fence-virtd fence-virtd-multicast fence-virtd-libvirt
Loaded plugins: langpacks, presto, refresh-packagekit
Resolving Dependencies
--> Running transaction check
---> Package fence-virt.x86_64 0:0.3.0-5.fc18 will be installed
---> Package fence-virtd.x86_64 0:0.3.0-5.fc18 will be installed
---> Package fence-virtd-libvirt.x86_64 0:0.3.0-5.fc18 will be installed
---> Package fence-virtd-multicast.x86_64 0:0.3.0-5.fc18 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
================================================================================
Package Arch Version Repository Size
================================================================================
Installing:
fence-virt x86_64 0.3.0-5.fc18 fedora 38 k
fence-virtd x86_64 0.3.0-5.fc18 fedora 30 k
fence-virtd-libvirt x86_64 0.3.0-5.fc18 fedora 13 k
fence-virtd-multicast x86_64 0.3.0-5.fc18 fedora 19 k
Transaction Summary
================================================================================
Install 4 Packages
Total download size: 101 k
Installed size: 173 k
Is this ok [y/N]: y
Downloading Packages:
(1/4): fence-virt-0.3.0-5.fc18.x86_64.rpm | 38 kB 00:00
(2/4): fence-virtd-0.3.0-5.fc18.x86_64.rpm | 30 kB 00:00
(3/4): fence-virtd-libvirt-0.3.0-5.fc18.x86_64.rpm | 13 kB 00:00
(4/4): fence-virtd-multicast-0.3.0-5.fc18.x86_64.rpm | 19 kB 00:00
--------------------------------------------------------------------------------
Total 187 kB/s | 101 kB 00:00
Running Transaction Check
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : fence-virtd-0.3.0-5.fc18.x86_64 1/4
Installing : fence-virtd-multicast-0.3.0-5.fc18.x86_64 2/4
Installing : fence-virtd-libvirt-0.3.0-5.fc18.x86_64 3/4
Installing : fence-virt-0.3.0-5.fc18.x86_64 4/4
Verifying : fence-virtd-multicast-0.3.0-5.fc18.x86_64 1/4
Verifying : fence-virt-0.3.0-5.fc18.x86_64 2/4
Verifying : fence-virtd-0.3.0-5.fc18.x86_64 3/4
Verifying : fence-virtd-libvirt-0.3.0-5.fc18.x86_64 4/4
Installed:
fence-virt.x86_64 0:0.3.0-5.fc18
fence-virtd.x86_64 0:0.3.0-5.fc18
fence-virtd-libvirt.x86_64 0:0.3.0-5.fc18
fence-virtd-multicast.x86_64 0:0.3.0-5.fc18
Complete!
Fedora 18 Host; Create The Key
For security reasons, we want to use a file with a secret key in it. This will ensure that only nodes in our cluster can send requests to terminate VMs to our fence_virtd daemon.
This key is stored in a file called /etc/cluster/fence_xvm.key file. You can use any string you want, including simple words or traditional passwords. In our case though, we will use dd to read in 512 bytes from the kernel's /dev/random (pseudo-)random number generator to create a nice, strong and random key.
dd if=/dev/random of=/etc/cluster/fence_xvm.key bs=512 count=1
0+1 records in
0+1 records out
128 bytes (128 B) copied, 0.000143983 s, 889 kB/s
If you try to look at the contents of this, you will find that it does not render as ASCII text very well. This is expected and not a problem.
Fedora 18 Host; Configure the Daemon
There is a nice configuration mode for fence_virtd that makes configuring it very simple. In this example, we will use all default values.
Note: We will use multicast for communication between the guests and the host. The multicast address does not have to match the multicast address used by your cluster. |
fence_virtd -c
Module search path [/usr/lib64/fence-virt]:
Available backends:
libvirt 0.1
Available listeners:
multicast 1.2
Listener modules are responsible for accepting requests
from fencing clients.
Listener module [multicast]:
The multicast listener module is designed for use environments
where the guests and hosts may communicate over a network using
multicast.
The multicast address is the address that a client will use to
send fencing requests to fence_virtd.
Multicast IP Address [225.0.0.12]:
Using ipv4 as family.
Multicast IP Port [1229]:
Setting a preferred interface causes fence_virtd to listen only
on that interface. Normally, it listens on all interfaces.
In environments where the virtual machines are using the host
machine as a gateway, this *must* be set (typically to virbr0).
Set to 'none' for no interface.
Interface [virbr0]:
The key file is the shared key information which is used to
authenticate fencing requests. The contents of this file must
be distributed to each physical host and virtual machine within
a cluster.
Key File [/etc/cluster/fence_xvm.key]:
Backend modules are responsible for routing requests to
the appropriate hypervisor or management layer.
Backend module [libvirt]:
Configuration complete.
=== Begin Configuration ===
backends {
libvirt {
uri = "qemu:///system";
}
}
listeners {
multicast {
port = "1229";
family = "ipv4";
interface = "virbr0";
address = "225.0.0.12";
key_file = "/etc/cluster/fence_xvm.key";
}
}
fence_virtd {
module_path = "/usr/lib64/fence-virt";
backend = "libvirt";
listener = "multicast";
}
=== End Configuration ===
Replace /etc/fence_virt.conf with the above [y/N]? y
Note that it will ask if you want to replace the /etc/fence_virt.conf, even though it doesn't exist.
Done!
Fedora 18 Host; Starting the Daemon
There is no init.d or systemd tool for starting or stopping the fence_virtd daemon. So to start it, simple call it at the command line;
fence_virtd
Nothing will be displayed, but we can check that it started well by checking it's exit code and using pidof to check for it's process ID.
echo $?
1
pidof fence_virtd
3244
Note: The process ID you see will almost certainly be different from the one above. |
That's it!
Fedora 18 Host; Restarting fence_virtd
If you ever want to stop fence_virtd, in order to load an updated configuration for example, you can use kill to do so.
kill $(pidof fence_virtd)
If you re-run the pidof command, you will see that no process ID is returned any more.
pidof fence_virtd
Now start it back up again;
fence_virtd
pidof fence_virtd
29210
Fedora 18 Host; Testing the Daemon
To test that the daemon is listening, you can run fence_xvm and ask it to list the clients. We have not configured any guests yet, so it should return immediately with no output.
fence_xvm -o list
Had there been a problem, you would see an error like:
fence_xvm -o list
Timed out waiting for response
Operation failed
Guest Configuration
This tutorial will use Fedora 19 (beta TC5) as the guest VMs.
Fedora 19
blah
Any questions, feedback, advice, complaints or meanderings are welcome. | |||
Alteeve's Niche! | Enterprise Support: Alteeve Support |
Community Support | |
© Alteeve's Niche! Inc. 1997-2024 | Anvil! "Intelligent Availability®" Platform | ||
legal stuff: All info is provided "As-Is". Do not use anything here unless you are willing and able to take responsibility for your own actions. |