Changing the ethX to Ethernet Device Mapping in EL6: Difference between revisions

From Alteeve Wiki
Jump to navigation Jump to search
(No difference)

Revision as of 22:22, 3 February 2011

 AN!Wiki :: How To :: Changing the ethX to Ethernet Device Mapping in EL6

Note: This paper is written for Red Hat Enterprise Linux 6, but should be directly adaptable to derivative distributions like CentOS 6 and Scientific Linux 6.

Overview

It is very difficult to configure which physical network cards get what network device names during installation . This document will show you how to change the ethX device name assigned to each physical network interface using MAC addresses.

This is desirable in cases where specific network cards need to be used on specific networks, and you are interested in maintaining common ethX names across servers. A practical example is found in the Red Hat Cluster Service 3 Tutorial.

For the Impatient

Assuming that NetworkManager is already disabled.

Stop the existing network. This is critical as you must stop the network using it's existing configuration before making changes. If you skip this step you may need to restart your server when you are done.

/etc/init.d/network stop

Edit each network configuration file and comment out the HWADDR="..." and, if it exists, the UUID="..." lines.

vim /etc/sysconfig/network-scripts/ifcfg-eth*

Edit the udev network rules file and change the NAME="..." entries to the ethX name you want for the relevant entries.

vim /etc/udev/rules.d/70-persistent-net.rules

Reload udev and start networking.

start_udev
/etc/init.d/network start

Example

I find it easier to follow instructions when I have an example to follow, so let me provide one here:

I've got the same hardware in the same configuration in both of my storage nodes. To keep things simple, I want to make sure that both nodes use the same ethX name for the network devices on each of the given networks.

Desired Mapping

Let's assume that you want the following configuration. The comments are meant to help you follow the devices as they move and are of no inherent value.

  • eth0
    • Type: # Realtek onboard NIC
    • MAC: 90:E6:BA:71:82:EA
  • eth1
    • Type: # D-Link PCIe gigabit, center
    • MAC: 00:21:91:19:96:53
  • eth2
    • Type: # Intel PCI gigabit, left-most
    • MAC: 00:0E:0C:59:46:E4

Initial Mapping

When the Fedora install finished, you may have gotten this:

  • eth0
    • Type: # D-Link PCIe gigabit, center
    • MAC: 00:21:91:19:96:53
  • eth1
    • Type: # Realtek onboard NIC
    • MAC: 90:E6:BA:71:82:EA
  • eth2
    • Type: # Intel PCI gigabit, left-most
    • MAC: 00:0E:0C:59:46:E4

The Problem

In the above example, eth0 and eth1 are reversed. The eth2 interface is where we want it, so we leave it alone.

The Fix

Traditionally, the ethX to physical ethernet device was linked via the HWADDR="..." argument in each NIC's ifcfg-ethX configuration file. With the introduction of udev, the HWADDR="..." is no longer needed. Instead, this mapping is now made in the 70-persistent-net.rules udev configuration file.

So the trick then is to remove or comment-out the HWADDR="..." and UUID="..." argument(s) in each NIC's ifcfg-ethX configuration file, then, change the NAME="ethX" in the udev configuration file to match the MAC address you want.

Stopping the Network

First, stop the network. This is important because if you change the MAC address to ethX mapping while the network is still up, the init.d script will fail to bring down the network devices and you will need to reboot for the changes to take effect.

/etc/init.d/network stop
Shutting down interface eth0:                              [  OK  ]
Shutting down interface eth1:                              [  OK  ]
Shutting down loopback interface:                          [  OK  ]

Note: The eth2 interface is not yet set to start with the system, so didn't need to be stopped.

Interface Configuration

Note: Though technically fine, I personally find the default configuration of the network interfaces to be sloppy and inconsistent. I like to take this step to also clean up these files and set them to the values I want. In this step I will set the interfaces to match an-node01 from the Red Hat Cluster Service 3 Tutorial. Feel free to adapt the configurations to suit your needs and networks.

Go to the /etc/sysconfig/network-scripts/ directory and then edit the ifcfg-eth* network configuration files. Below are examples of the interface configurations "before" and "after" the changes. Note that the only edit that really matters is that the HWADDR="..." lines are commented out. In my case, there were no UUID="..." entries.

Network Config File Syntax

For an exhaustive list of available options in ifcfg-ethX files, please see the Fedora documentation.

For a fully up to date, if somewhat sparse, list of available options, please consult the sysconfig documentation shipped with your distribution. For Red Hat Enterprise Linux 6.0, this can be found at /usr/share/doc/initscripts-9.03.17/sysconfig.txt under the Files in /etc/sysconfig/network-scripts/ section. Specifically, look at the /etc/sysconfig/network-scripts/ifcfg-<interface-name> sub-section.

/etc/sysconfig/network-scripts/ifcfg-eth0

First up, eth0.

vim /etc/sysconfig/network-scripts/ifcfg-eth0

The file original looked like this:

DEVICE="eth0"
BOOTPROTO="dhcp"
DHCP_HOSTNAME="an-node01.alteeve.com"
HWADDR="00:21:91:19:96:53"
NM_CONTROLLED="yes"
ONBOOT="yes"

I like to keep the MAC address in the file to act as a reference for when I am working with the files. For this reason, I change the HWADDR="..." entry despite commenting it out. I also like to add a NAME="..." entry and a descriptive comment at the head of the file. Again though, all of this is optional. All that really matters is that HWADDR and UUID are gone.

# Realtek onboard NIC "blue" network
#HWADDR=90:E6:BA:71:82:EA
DEVICE="eth0"
NAME="Internet Facing Network eth0"
ONBOOT="yes"
NM_CONTROLLED="no"
TYPE=Ethernet
BOOTPROTO=none
IPV4_FAILURE_FATAL=yes
IPV6INIT=no
DEFROUTE=yes
IPADDR=192.168.1.71
NETMASK=255.255.255.0
GATEWAY=192.168.1.254
DNS1=192.139.81.117
DNS2=192.139.81.1
DOMAIN=alteeve.com

/etc/sysconfig/network-scripts/ifcfg-eth1

As with eth0, all that matters is that HWADDR and UUID are gone. As before though, I like to use this time to clean up the file.

vim /etc/sysconfig/network-scripts/ifcfg-eth0

The file originally looked like this:

DEVICE="eth1"
BOOTPROTO="dhcp"
DHCP_HOSTNAME="an-node01.alteeve.com"
HWADDR="90:E6:BA:71:82:EA"
NM_CONTROLLED="yes"
ONBOOT="yes"

As before, I updated and the commented-out the HWADDR="..." line to act as a reference.

# D-Link PCIe gigabit, center "green" network
#HWADDR=00:21:91:19:96:53
DEVICE="eth1"
NAME="Storage Network eth1"
ONBOOT="yes"
NM_CONTROLLED="no"
TYPE=Ethernet
BOOTPROTO=none
IPV4_FAILURE_FATAL=yes
IPV6INIT=no
DEFROUTE=no
IPADDR=192.168.2.71
NETMASK=255.255.255.0

/etc/sysconfig/network-scripts/ifcfg-eth2

Technically speaking, the HWADDR="..." doesn't need to be removed, if it agrees with the entry in /etc/udev/rules.d/70-persistent-net.rules. For this reason, given this example, there is no strict need to edit this file. However, being somewhat OCDish, I like to clean the file up to match the others. Also, in my case, this device was not set to start with the system. This is a great time to sort that out.

vim /etc/sysconfig/network-scripts/ifcfg-eth2

The original was quite minimal.

DEVICE="eth2"
HWADDR="00:0E:0C:59:46:E4"
NM_CONTROLLED="yes"
ONBOOT="no"

Now it's in-line with the other two configuration files.

# Intel PCI gigabit, left-most "black" network
#HWADDR=00:0E:0C:59:46:E4
DEVICE="eth2"
NAME="Back Channel Network eth2"
ONBOOT="yes"
NM_CONTROLLED="no"
TYPE=Ethernet
BOOTPROTO=none
IPV4_FAILURE_FATAL=yes
IPV6INIT=no
DEFROUTE=no
IPADDR=192.168.3.71
NETMASK=255.255.255.0

Modifying The udev Configuration

The user-space device manager, udev, maintains a list of configuration files that help ensure that hardware to /dev (software) mappings stay consistent, even when hardware is hot swapped or moved between physical interfaces. For networking hardware, this mapping is recorded in the 70-persistent-net.rules file.

Here, all that needs to be done is for the NAME="..." entry to be changed to be the ethX device you want associated with the given MAC address seen in the corresponding ATTR{address}=="..." entry.

Note: The entries do not need to be in any particular order.

Edit the 70-persistent-net.rules.

vim /etc/udev/rules.d/70-persistent-net.rules

The complete file originally was as follows.

# This file was automatically generated by the /lib/udev/write_net_rules
# program, run by the persistent-net-generator.rules rules file.
#
# You can modify it, as long as you keep each rule on a single
# line, and change only the value of the NAME= key.

# PCI device 0x10ec:0x8168 (r8169) (custom name provided by external tool)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="90:e6:ba:71:82:ea", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"

# PCI device 0x8086:0x100e (e1000) (custom name provided by external tool)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:0e:0c:59:46:e4", ATTR{type}=="1", KERNEL=="eth*", NAME="eth2"

# PCI device 0x1186:0x4b00 (sky2) (custom name provided by external tool)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:21:91:19:96:53", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"

The only change was to the first and third entries.

# This file was automatically generated by the /lib/udev/write_net_rules
# program, run by the persistent-net-generator.rules rules file.
#
# You can modify it, as long as you keep each rule on a single
# line, and change only the value of the NAME= key.

# PCI device 0x10ec:0x8168 (r8169) (custom name provided by external tool)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="90:e6:ba:71:82:ea", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"

# PCI device 0x8086:0x100e (e1000) (custom name provided by external tool)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:0e:0c:59:46:e4", ATTR{type}=="1", KERNEL=="eth*", NAME="eth2"

# PCI device 0x1186:0x4b00 (sky2) (custom name provided by external tool)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:21:91:19:96:53", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"

Making the Changes Take Effect

First, you need to reload udev so that it updates the mapping in memory. This is done using the start_udev command.

start_udev
Starting udev:                                             [  OK  ]

With udev updated, the final step is to start network back up.

/etc/init.d/network start
Bringing up loopback interface:                            [  OK  ]
Bringing up interface eth0:                                [  OK  ]
Bringing up interface eth1:                                [  OK  ]
Bringing up interface eth2:                                [  OK  ]

Use ifconfig to verify that the network cards are properly assigned to the specified ethX names.

ifconfig
eth0      Link encap:Ethernet  HWaddr 90:E6:BA:71:82:EA  
          inet addr:192.168.1.71  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::92e6:baff:fe71:82ea/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2478 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1301 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:272436 (266.0 KiB)  TX bytes:245173 (239.4 KiB)
          Interrupt:28 Base address:0x6000 

eth1      Link encap:Ethernet  HWaddr 00:21:91:19:96:53  
          inet addr:192.168.2.71  Bcast:192.168.2.255  Mask:255.255.255.0
          inet6 addr: fe80::221:91ff:fe19:9653/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:53 errors:0 dropped:0 overruns:0 frame:0
          TX packets:56 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:12144 (11.8 KiB)  TX bytes:10693 (10.4 KiB)
          Interrupt:18 

eth2      Link encap:Ethernet  HWaddr 00:0E:0C:59:46:E4  
          inet addr:192.168.3.71  Bcast:192.168.3.255  Mask:255.255.255.0
          inet6 addr: fe80::20e:cff:fe59:46e4/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:83 errors:0 dropped:0 overruns:0 frame:0
          TX packets:47 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:22960 (22.4 KiB)  TX bytes:8191 (7.9 KiB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:8 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:480 (480.0 b)  TX bytes:480 (480.0 b)

Confirm that the HWaddr ... are where you want them.

Done!

Caveat!

NOTE: If you are using a VLAN, the device facing the VLAN can not have the HWADDR=... value set! Technically speaking, the HWADDR="..." entries may exist, as noted in the eth2 section above. However, if this is left in for the VLAN interface, the OS will not see the physical device and will fail to bring up the interface at all.

Thanks

  • Thanks to fenris02 and mutk on the #fedora IRC channel for helping me sort out the udev component.
  • Thanks to Fiver- and inode0 on the #rhel IRC channel for helping me find the sysconfig documentation.

 

Any questions, feedback, advice, complaints or meanderings are welcome.
Alteeve's Niche! Enterprise Support:
Alteeve Support
Community Support
© Alteeve's Niche! Inc. 1997-2024   Anvil! "Intelligent Availability®" Platform
legal stuff: All info is provided "As-Is". Do not use anything here unless you are willing and able to take responsibility for your own actions.