|
|
| Line 1: |
Line 1: |
| {{howto_header}}
| | == Setup SSH == |
|
| |
|
| {{warning|1=This tutorial is incomplete, flawed and generally sucks at this time. Do not follow this and expect anything to work. In large part, it's a dumping ground for notes and little else. This warning will be removed when the tutorial is completed.}}
| | Same as [[2-Node_Red_Hat_KVM_Cluster_Tutorial#Setting_up_SSH|before]]. |
|
| |
|
| {{warning|1=This tutorial is built on '''''a guess''''' of what [[Red Hat]]'s Enterprise Linux 7 will offer, based on what the author sees happening in [[Fedora]] upstream. [[Red Hat]] never confirms what a future release will contain until it is actually released. As such, this tutorial may turn out to be inappropriate for the final release of [[RHEL]] 7. In such a case, the warning above will remain in place until the tutorial is updated to reflect the final release.}}
| | == Populating And Pushing ~/.ssh/known_hosts == |
|
| |
|
| This is the third '''AN!Cluster''' tutorial built on [[Red Hat]]'s Enterprise Linux 7. It improves on the [[Red Hat Cluster Service 2 Tutorial|RHEL 5, RHCS stable 2]] and [[2-Node Red Hat KVM Cluster Tutorial|RHEL 6, RHCS stable3]] tutorials.
| | Same as [[2-Node_Red_Hat_KVM_Cluster_Tutorial#Populating_And_Pushing_~/.ssh/known_hosts|before]]. |
| | |
| As with the previous tutorials, the end goal of this tutorial is a 2-node cluster providing a platform for high-availability virtual servers. It's design attempts to remove all single points of failure from the system. Power and networking are made fully redundant in this version, along with minimizing the node failures which would lead to service interruption. This tutorial also covers the [[AN!Utilities]]; [[AN!Cluster Dashboard]], [[AN!Cluster Monitor]] and [[AN!Safe Cluster Shutdown]].
| |
| | |
| As it the previous tutorial, [[KVM]] will be the hypervisor used for facilitating virtual machines. The old <span class="code">[[cman]]</span> and <span class="code">[[rgmanager]]</span> tools are replaced in favour of <span class="code">[[pacemaker]]</span> for resource management.
| |
| | |
| = Before We Begin =
| |
| | |
| This tutorial '''does not''' require prior cluster experience, but it does expect familiarity with Linux and a low-intermediate understanding of networking. Where possible, steps are explained in detail and rationale is provided for why certain decisions are made.
| |
| | |
| '''For those with cluster experience''';
| |
| | |
| Please be careful not to skip too much. There are some major and some subtle changes from previous tutorials.
| |
| | |
| = OS Setup =
| |
| | |
| {{warning|1=I used Fedora 18 at this point, obviously things will change, possibly a lot, once RHEL 7 is released.}}
| |
| | |
| == Install ==
| |
| | |
| Not all of these are required, but most are used at one point or another in this tutorial.
| |
|
| |
|
| <source lang="bash"> | | <source lang="bash"> |
| yum install bridge-utils gpm man net-tools rsync vim wget
| | ssh root@an-c03n01.alteeve.ca |
| </source> | | </source> |
| | | <source lang="text"> |
| If you want to use your mouse at the node's terminal, run the following;
| | The authenticity of host 'an-c03n01.alteeve.ca (10.20.30.1)' can't be established. |
| | | RSA key fingerprint is 7b:dd:0d:aa:c5:f5:9e:a6:b6:4d:40:69:d6:80:4d:09. |
| <source lang="bash"> | | Are you sure you want to continue connecting (yes/no)? |
| systemctl enable gpm.service
| |
| systemctl start gpm.service
| |
| </source> | | </source> |
|
| |
|
| == Setting the Hostname == | | Type <span class="code">yes</span> |
|
| |
|
| Fedora 18 is '''very''' different from [[EL6]].
| | <source lang="text"> |
| | | Are you sure you want to continue connecting (yes/no)? yes |
| {{note|1=The '<span class="code">--pretty</span>' line currently doesn't work as there is [https://bugzilla.redhat.com/show_bug.cgi?id=895299 a bug (rhbz#895299)] with single-quotes.}}
| |
| {{note|1=The '<span class="code">--static</span>' option is currently needed to prevent the '<span class="code">.</span>' from being removed. See [https://bugzilla.redhat.com/show_bug.cgi?id=896756 this bug (rhbz#896756)].}}
| |
| | |
| Use a format that works for you. For the tutorial, node names are based on the following;
| |
| * A two-letter prefix identifying the company/user (<span class="code">an</span>, for "Alteeve's Niche!")
| |
| * A sequential cluster ID number in the form of <span class="code">cXX</span> (<span class="code">c01</span> for "Cluster 01", <span class="code">c02</span> for Cluster 02, etc)
| |
| * A sequential node ID number in the form of <span class="code">nYY</span>
| |
| | |
| In my case, this is my third cluster and I use the company prefix <span class="code">an</span>, so my two nodes will be;
| |
| * <span class="code">an-c03n01</span> - node 1
| |
| * <span class="code">an-c03n02</span> - node 2
| |
| | |
| Folks who've read my earlier tutorials will note that this is a departure in naming. I find this method spans and scales much better. Further, it the simply required in order to use the [[AN!CDB|AN! Cluster Dashboard]].
| |
| | |
| <source lang="bash"> | |
| hostnamectl set-hostname an-c03n01.alteeve.ca --static
| |
| hostnamectl set-hostname --pretty "Alteeve's Niche! - Cluster 01, Node 01"
| |
| </source>
| |
| | |
| If you want the new host name to take effect immediately, you can use the traditional <span class="code">hostname</span> command:
| |
| | |
| <source lang="bash">
| |
| hostname an-c03n01.alteeve.ca
| |
| </source>
| |
| | |
| '''Alternatively'''
| |
| | |
| If you have trouble with those commands, you can directly edit the files that contain the host names.
| |
| | |
| The host name is stored in <span class="code">/etc/hostname</span>:
| |
| | |
| <source lang="bash">
| |
| echo an-c03n01.alteeve.ca > /etc/hostname
| |
| cat /etc/hostname
| |
| </source> | | </source> |
| <source lang="text"> | | <source lang="text"> |
| an-c03n01.alteeve.ca | | Warning: Permanently added 'an-c03n01.alteeve.ca,10.20.30.1' (RSA) to the list of known hosts. |
| | Last login: Thu Feb 14 15:18:33 2013 from 10.20.5.100 |
| </source> | | </source> |
|
| |
|
| The "pretty" host name is stored in <span class="code">/etc/machine-info</span> as the unquoted value for the <span class="code">PRETTY_HOSTNAME</span> value.
| | You will now be logged into the <span class="code">an-c03n01</span> node, which in this case is the same machine on a new session in the same terminal. |
|
| |
|
| <source lang="bash">
| |
| vim /etc/machine-info
| |
| </source>
| |
| <source lang="text"> | | <source lang="text"> |
| PRETTY_HOSTNAME=Alteeves Niche! - Cluster 01, Node 01
| | [root@an-c03n01 ~]# |
| </source>
| |
| | |
| If you can't get the <span class="code">hostname</span> command to work for some reason, you can reboot to have the system read the new values.
| |
| | |
| == Optional - Video Problems ==
| |
| | |
| On my servers, [[Fedora]] 18 doesn't detect or use the video card properly. To resolve this, I need to add <span class="code">nomodeset</span> to the kernel line when installing and again after the install is complete.
| |
| | |
| Once installed
| |
| | |
| Edit the <span class="code">/etc/default/grub</span> and append <span class="code">nomodeset</span> to the end of the <span class="code">GRUB_CMDLINE_LINUX</span> variable.
| |
| | |
| <source lang="bash">
| |
| vim /etc/default/grub
| |
| </source>
| |
| <source lang="bash">
| |
| GRUB_TIMEOUT=5
| |
| GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"
| |
| GRUB_DEFAULT=saved
| |
| GRUB_CMDLINE_LINUX="nomodeset rd.md=0 rd.lvm=0 rd.dm=0 $([ -x /usr/sbin/rhcrashkernel-param ] && /usr/sbin/rhcrashkernel-param || :) rd.luks=0 vconsole.keymap=us nomodeset"
| |
| GRUB_DISABLE_RECOVERY="true"
| |
| GRUB_THEME="/boot/grub2/themes/system/theme.txt"
| |
| </source>
| |
| | |
| Save that. and then rewrite the [[grub2]] configuration file.
| |
| | |
| <source lang="bash">
| |
| grub2-mkconfig -o /boot/grub2/grub.cfg
| |
| </source>
| |
| | |
| Next time you reboot, you should get a stock 80x25 character display. It's not much, but it will work on esoteric video cards or weird monitors.
| |
| | |
| == What Security? ==
| |
| | |
| This section will be re-added at the end. For now;
| |
| | |
| <source lang="bash">
| |
| setenforce 0
| |
| sed -i 's/SELINUX=.*/SELINUX=disabled/' /etc/selinux/config
| |
| systemctl disable firewalld.service
| |
| systemctl stop firewalld.service
| |
| </source>
| |
| | |
| == Network ==
| |
| | |
| We want static, named network devices. Follow this;
| |
| | |
| * [[Changing Ethernet Device Names in EL7 and Fedora 15+]]
| |
| | |
| Then, use these configuration files;
| |
| | |
| Build the bridge;
| |
| | |
| <source lang="bash">
| |
| vim /etc/sysconfig/network-scripts/ifcfg-ifn-vbr1
| |
| </source>
| |
| <source lang="bash">
| |
| # Internet-Facing Network - Bridge
| |
| DEVICE="ifn-vbr1"
| |
| TYPE="Bridge"
| |
| BOOTPROTO="none"
| |
| IPADDR="10.255.10.1"
| |
| NETMASK="255.255.0.0"
| |
| GATEWAY="10.255.255.254"
| |
| DNS1="8.8.8.8"
| |
| DNS2="8.8.4.4"
| |
| DEFROUTE="yes"
| |
| </source>
| |
| | |
| Now build the bonds;
| |
| | |
| <source lang="bash">
| |
| vim /etc/sysconfig/network-scripts/ifcfg-ifn-bond1
| |
| </source>
| |
| <source lang="bash">
| |
| # Internet-Facing Network - Bond | |
| DEVICE="ifn-bond1"
| |
| BRIDGE="ifn-vbr1"
| |
| BOOTPROTO="none"
| |
| NM_CONTROLLED="no"
| |
| ONBOOT="yes"
| |
| BONDING_OPTS="mode=1 miimon=100 use_carrier=1 updelay=120000 downdelay=0 primary=ifn1"
| |
| </source>
| |
| | |
| <source lang="bash">
| |
| vim /etc/sysconfig/network-scripts/ifcfg-sn-bond1
| |
| </source>
| |
| <source lang="bash">
| |
| # Storage Network - Bond
| |
| DEVICE="sn-bond1"
| |
| BOOTPROTO="none"
| |
| NM_CONTROLLED="no"
| |
| ONBOOT="yes"
| |
| BONDING_OPTS="mode=1 miimon=100 use_carrier=1 updelay=120000 downdelay=0 primary=sn1"
| |
| IPADDR="10.10.10.1"
| |
| NETMASK="255.255.0.0"
| |
| </source>
| |
| | |
| <source lang="bash">
| |
| vim /etc/sysconfig/network-scripts/ifcfg-bcn-bond1
| |
| </source>
| |
| <source lang="bash">
| |
| # Back-Channel Network - Bond
| |
| DEVICE="bcn-bond1"
| |
| BOOTPROTO="none"
| |
| NM_CONTROLLED="no"
| |
| ONBOOT="yes"
| |
| BONDING_OPTS="mode=1 miimon=100 use_carrier=1 updelay=120000 downdelay=0 primary=bcn1"
| |
| IPADDR="10.20.10.1"
| |
| NETMASK="255.255.0.0"
| |
| </source>
| |
| | |
| Now tell the interfaces to be slaves to their bonds;
| |
| | |
| Internet-Facing Network;
| |
| | |
| <source lang="bash">
| |
| vim /etc/sysconfig/network-scripts/ifcfg-ifn1
| |
| </source>
| |
| <source lang="bash">
| |
| # Internet-Facing Network - Link 1
| |
| DEVICE="ifn1"
| |
| NM_CONTROLLED="no"
| |
| BOOTPROTO="none"
| |
| ONBOOT="yes"
| |
| SLAVE="yes"
| |
| MASTER="ifn-bond1"
| |
| </source>
| |
| | |
| <source lang="bash">
| |
| vim /etc/sysconfig/network-scripts/ifcfg-ifn2
| |
| </source>
| |
| <source lang="bash">
| |
| # Back-Channel Network - Link 2
| |
| DEVICE="ifn2"
| |
| NM_CONTROLLED="no"
| |
| BOOTPROTO="none"
| |
| ONBOOT="yes"
| |
| SLAVE="yes"
| |
| MASTER="ifn-bond1"
| |
| </source>
| |
| | |
| Storage Network;
| |
| | |
| <source lang="bash">
| |
| vim /etc/sysconfig/network-scripts/ifcfg-sn1
| |
| </source>
| |
| <source lang="bash">
| |
| # Storage Network - Link 1
| |
| DEVICE="sn1"
| |
| NM_CONTROLLED="no"
| |
| BOOTPROTO="none"
| |
| ONBOOT="yes"
| |
| SLAVE="yes"
| |
| MASTER="sn-bond1"
| |
| </source>
| |
| | |
| <source lang="bash">
| |
| vim /etc/sysconfig/network-scripts/ifcfg-sn2
| |
| </source>
| |
| <source lang="bash">
| |
| # Storage Network - Link 1
| |
| DEVICE="sn2"
| |
| NM_CONTROLLED="no"
| |
| BOOTPROTO="none"
| |
| ONBOOT="yes"
| |
| SLAVE="yes"
| |
| MASTER="sn-bond1"
| |
| </source> | | </source> |
|
| |
|
| Back-Channel Network
| | You can logout by typing <span class="code">exit</span>. |
|
| |
|
| <source lang="bash"> | | <source lang="bash"> |
| vim /etc/sysconfig/network-scripts/ifcfg-bcn1
| | exit |
| </source>
| |
| <source lang="bash">
| |
| # Back-Channel Network - Link 1
| |
| DEVICE="bcn1"
| |
| NM_CONTROLLED="no"
| |
| BOOTPROTO="none"
| |
| ONBOOT="yes"
| |
| SLAVE="yes"
| |
| MASTER="bcn-bond1"
| |
| </source>
| |
| | |
| <source lang="bash">
| |
| vim /etc/sysconfig/network-scripts/ifcfg-bcn2
| |
| </source>
| |
| <source lang="bash">
| |
| # Storage Network - Link 1
| |
| DEVICE="bcn2"
| |
| NM_CONTROLLED="no"
| |
| BOOTPROTO="none"
| |
| ONBOOT="yes"
| |
| SLAVE="yes"
| |
| MASTER="bcn-bond1"
| |
| </source>
| |
| | |
| Now restart the network, confirm that the bonds and bridge are up and you are ready to proceed.
| |
| | |
| == Setup The hosts File ==
| |
| | |
| You can use [[DNS]] if you prefer. For now, lets use <span class="code">/etc/hosts</span> for node name resolution.
| |
| | |
| <source lang="bash">
| |
| vim /etc/hosts
| |
| </source> | | </source> |
| <source lang="text"> | | <source lang="text"> |
| 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
| | logout |
| ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
| | Connection to an-c03n01.alteeve.ca closed. |
| | |
| # AN!Cluster 01, Node 01
| |
| 10.255.10.1 an-c01n01.ifn
| |
| 10.10.10.1 an-c01n01.sn
| |
| 10.20.10.1 an-c01n01.bcn an-c01n01 an-c01n01.alteeve.ca
| |
| 10.20.11.1 an-c01n01.ipmi
| |
| | |
| # AN!Cluster 01, Node 02
| |
| 10.255.10.2 an-c01n02.ifn
| |
| 10.10.10.2 an-c01n02.sn
| |
| 10.20.10.2 an-c01n02.bcn an-c01n02 an-c01n02.alteeve.ca
| |
| 10.20.11.2 an-c01n02.ipmi
| |
| | |
| # Foundation Pack
| |
| 10.20.2.7 an-p03 an-p03.alteeve.ca
| |
| </source> | | </source> |
|
| |
|
| == Setup SSH ==
| | Now we have to repeat the steps for all the other variations on the names of the hosts. This is annoying and tedious, sorry. |
| | |
| Same as [[2-Node_Red_Hat_KVM_Cluster_Tutorial#Setting_up_SSH|before]].
| |
| | |
| == Installing Programs ==
| |
| | |
| From this point forward, this tutorial borrows heavily from [http://www.clusterlabs.org/doc/en-US/Pacemaker/1.1/html/Clusters_from_Scratch/ Pacemaker 1.1, Clusters from Scratch].
| |
|
| |
|
| <source lang="bash"> | | <source lang="bash"> |
| yum install pacemaker corosync pcs
| | ssh root@an-c03n01 |
| </source>
| | ssh root@an-c03n01.bcn |
| | | ssh root@an-c03n01.sn |
| == Configure Corosync ==
| | ssh root@an-c03n01.ifn |
| | | ssh root@an-c03n02.alteeve.ca |
| More on [[Multicast]].
| | ssh root@an-c03n02 |
| | | ssh root@an-c03n02.bcn |
| Pick a Multicast address and ensure that it doesn't conflict with any other multicast group used on your subnet. Pay particular attention to the ambiguous bits and how multicast IPs overlap with one another.
| | ssh root@an-c03n02.sn |
| | | ssh root@an-c03n02.ifn |
| For this tutorial, we will use multicast group <span class="code">239.255.1.1</span> and port <span class="code">5405</span>
| |
| | |
| <source lang="bash">
| |
| rpm -Uvh http://people.redhat.com/cfeist/pcs/pcs-0.9.26-1.fc18.noarch.rpm
| |
| </source>
| |
| <source lang="bash">
| |
| </source> | | </source> |
|
| |
|
| Line 347: |
Line 62: |
| <source lang="text"> | | <source lang="text"> |
| </source> | | </source> |
| <source lang="diff">
| |
| </source>
| |
|
| |
| {{footer}}
| |