Networking in Xen 3.x: Difference between revisions

From Alteeve Wiki
Jump to navigation Jump to search
No edit summary
No edit summary
Line 15: Line 15:
== ethX ==
== ethX ==


Replace 'X' in 'ethX' with a number reflecting a given ethernet device. For example, 'eth0' is the first ethernet device, 'eth1' the second and so on.
Normally, this is the name of a "main" ethernet device. However, under Xen, this does '''''NOT''''' reflect the real hardware once the <span class="code">xend</span> daemon has started!


This is the name of a "main" ethernet device. However, under Xen, this does '''''NOT''''' reflect the real hardware once the Xen daemon has started!
Under Xen, what '<span class="code">ethX</span>' refers to depends on where you are. Under <span class="code">dom0<span>, '<span class="code">ethX</span>' is a virtual ethernet device which has a copy of the real ethernet device's MAC and IP addresses. On <span class="code">domU</span>, '<span class="code">ethX</span>' is a virtual ethernet device connecting to a matching '<span class="code">vifX.Y</span> device created on <span class="code">dom0</span> by Xen when <span class="code">domU</span> was started. See the '''vifX.Y''' definition below.


Once Xen is up, what 'ethX' actually refers to depends on where you are. Under '''dom0''', 'ethX' is a virtual ethernet device which has a copy of the real ethernet device's MAC and IP addresses. On '''domU''', 'ethX' is a virtual ethernet device connecting to a matching '''vifX.Y''' device created on '''dom0''' by Xen when '''domU''' was started. See the '''vifX.Y''' definition below.
Replace '<span class="code">X</span>' in '<span class="code">ethX</span>' with a number reflecting a given ethernet device. For example, '<span class="code">eth0</span>' is the first ethernet device, '<span class="code">eth1</span>' the second and so on.


== pethX ==
== pethX ==


Replace 'X' in 'pethX' with a number reflecting a given ethernet device. For example, 'peth0' is the first ethernet device, 'peth1' the second and so on.
The <span class="code">pethX</span> device(s) exist only on <span class="code">dom0</span>.  


'''pethX''' devices exist only on '''dom0'''.  
When <span class="code">xend</span> starts, it renames the "real" '<span class="code">ethX</span>' to '<span class="code">pethX</span>' in order to create a virtual copy of it. This is done to facilitate bridging and routing. So then, <span class="code">pethX</span> devices are in fact the interfaces to the real hardware ethernet devices.


When xend starts, it renames the "real" 'ethX' to 'pethX' in order to create a virtual copy of it. This is done to facilitate bridging and routing. So then, '''pethX''' devices are in fact the interfaces to the real hardware ethernet devices.
Replace '<span class="code">X</span>' in '<span class="code">pethX</span>' with a number reflecting a given ethernet device. For example, '<span class="code">peth0</span>' is the first ethernet device, '<span class="code">peth1</span>' the second and so on.


== vethX ==
== vethX ==


Replace 'X' in 'vethX' with a number reflecting a given virtual ethernet device. For example, 'veth0' is the first virtual ethernet device, 'veth1' the second and so on.
The <span class="code">vethX</span> device(s) exist only on <span class="code">dom0</span> and only for a short time during <span class="code">xend</span> startup.  


'''vethX''' devices exist only on '''dom0'''.  
During the Xen startup process, <span class="code">ethX</span> is brought down and it's MAC and IP addresses are copied to <span class="code">vethX</span>. Once done, <span class="code">ethX</span> is renamed to <span class="code">pethX</span> to move it out of the way. Then <span class="code">vethX</span> is renamed to <span class="code">ethX</span> to take the place of the <span class="code">pethX</span> interface. As such, <span class="code">vethX</span> devices exist for only a short time. You should never need to worry about or factor in these devices into your networking plans.


During the Xen startup process, '''ethX''' is brought down and it's MAC and IP addresses are copied to '''vethX'''. Once done, '''ethX''' is renamed to '''pethX''' to move it out of the way. Then '''vethX''' is renamed to '''ethX''' to take the place of the '''pethX''' interface. As such, '''vethX''' devices exist for only a short time. You should never need to worry about or factor in these devices into your networking plans.
Replace '<span class="code">X</span>' in '<span class="code">vethX</span>' with a number reflecting a given virtual ethernet device. For example, '<span class="code">veth0</span>' is the first virtual ethernet device, '<span class="code">veth1</span>' the second and so on.


== vifX.Y ==
== vifX.Y ==


'''vifX.Y''' devices exist only on '''dom0'''.  
The <span class="code">vifX.Y</span> devices exist only on <span class="code">dom0</span>.  


'''Note'''; in this section, 'X' matches the '''domU''' number, 'Y' matches a given '''ethX''' device on '''dom0''' and 'Z' matches a given '''ethX''' device inside a '''domU'''. I know this is a little odd, but you need to get this clear in order to see how things connect here.
'''Note'''; in this section, '<span class="code">X</span>' matches the <span class="code">domU</span> number, '<span class="code">Y</span>' matches a given <span class="code">ethX</span> device on <span class="code">dom0</span> and '<span class="code">Z</span>' matches a given <span class="code">ethX</span> device inside a <span class="code">domU</span>. I know this is a little odd, but you need to get this clear in order to see how things connect here.


These are Xen's virtual interfaces used to connect an '''ethX''' device on '''dom0''' to an '''ethZ''' device on a '''domU'''. Please be aware that there need not be any correlation between '''eth0''' on '''dom0''' and '''eth0''' on '''domU'''! This is why 'X' and 'Z' are used separately here.
These are Xen's virtual interfaces used to connect an <span class="code">ethX</span> device on <span class="code">dom0</span> to a <span class="code">pethZ</span> device on a <span class="code">domU</span>. Please be aware that there need not be any correlation between <span class="code">eth0</span> on <span class="code">dom0</span> and <span class="code">eth0</span> on <span class="code">domU</span>! This is why '<span class="code">X</span>' and '<span class="code">Z</span>' are used separately here.


Specifically, when Xen starts a '''domU''', that virtual machine's ID# plus a given '''dom0''' '''ethX''' device number are used to create one or more virtual interfaces inside '''dom0'''. For example, if you want to connect '''eth0''' on '''dom0''' to '''eth0''' on '''dom4''', you will do so via '''vif4.0''' where '4' matches the '''domU''' ID# and '0' matches the ethernet device number in '''dom0'''. The actual routing goes through '''xenbr0'''. If this is still confusing, take a look at the network map below.
Specifically, when Xen starts a <span class="code">domU</span>, that virtual machine's ID# plus a given <span class="code">dom0</span> <span class="code">ethX</span> device number are used to create one or more virtual interfaces inside <span class="code">dom0</span>. For example, if you want to connect <span class="code">eth0</span> on <span class="code">dom0</span> to <span class="code">eth0</span> on <span class="code">dom4</span>, you will do so via <span class="code">vif4.0</span> where '<span class="code">4</span>' matches the <span class="code">domU</span> ID# and '<span class="code">0</span>' matches the ethernet device number in <span class="code">dom0</span>. The actual routing goes through <span class="code">xenbr0</span>. If this is still confusing, take a look at the network map below.


== xenbrX ==
== xenbrX ==


'''xenbrX''' devices exist only on '''dom0'''.
The <span class="code">xenbrX</span> device(s) exist only on <span class="code">dom0</span>.


Xen uses software bridges to route traffic within it's virtual network. These can be thought of as a "switch" your would find in any normal network. They serve to connect a given ethernet device on '''dom0''' to a given ethernet device in a '''domU''' virtual machine. A different bridge is created for each '''ethX''' device on '''dom0'''.
Xen uses software bridges to route traffic within it's virtual network. These can be thought of as a "switch" your would find in any normal network. They serve to connect a given ethernet device on <span class="code">dom0</span> to a given ethernet device in a <span class="code">domU</span> virtual machine. A different bridge is created for each <span class="code">ethX</span> device on <span class="code">dom0</span>.


== Network Map - Putting it all together ==
== Network Map - Putting it all together ==


This map is a sample of a four-[[VM]] setup. You will notice that '''eth1''' is not connected to any '''domU''''s because it is used for the DRBD channel. All virtual machines have a connection to '''eth0''' because that is the virtual machine network's "back channel". In practice, there is no internet connection on '''eth0''', and thus on the '''xenbr0''' bridge, except through the firewall, '''dom1'''. The '''dom6''' shows a link to '''eth2''', which would give it a direct connection to the Internet. This isn't really done in the CanadaEquity network, but it helps show how the '''vifX.Y''' naming convention works.
This map is a sample of a four-[[VM]] setup in a single Xen host. You will notice that <span class="code">eth1</span> is not connected to any <span class="code">domU</span>'s because, in this case, it is used for the DRBD channel. All virtual machines have a connection to <span class="code">eth0</span> because that is the virtual machine network's "back channel". In practice, there is no internet connection on <span class="code">eth0</span>, and thus on the <span class="code">xenbr0</span> bridge, except through the firewall, <span class="code">dom1</span>. The <span class="code">dom6</span> shows a link to <span class="code">eth2</span>, which would give it a direct connection to the Internet.


<source lang="text">
   _________________________________________________________________
   _________________________________________________________________
  | dom0                                          _______________  |
  | dom0                                          _______________  |
Line 97: Line 98:
  |                                              |_______________| |
  |                                              |_______________| |
  |_________________________________________________________________|
  |_________________________________________________________________|
</source>


[[Sample Xen 2-node Cluster Network Map|This]] is one an example of networking inside a two node cluster.
[[Sample Xen 2-node Cluster Network Map|This]] is one an example of networking inside a two node cluster.
= Network Changes When Xen Starts =
Before the '''xend''' server starts,


{{footer}}
{{footer}}

Revision as of 20:47, 10 May 2010

 AN!Wiki :: How To :: Networking in Xen 3.x

This section is based on this link. There are a lot of other good topics on Xen networking and general Xen topics here.

Definitions

dom0

The "real" OS on a Xen server is not real at all, but in fact just another virtual machine with special access to the hardware. This first virtual machine is known as dom0.

domU

Any virtual machines created on a Xen server other than the special dom0 is called a domU server. Generally this is used in documentation to describe any particular virtual machine. When a specific virtual machine is to be referenced, it is called dom1, dom2, ..., domN. The number of the virtual machine is important as it relates to the underlying network setup. The given number is not user assignable, but is set when the virtual machine is created. Please note that, even if a virtual machine is destroyed, it's number is not recycled. Thus, you may well have something like 'dom0', 'dom1', 'dom3', 'dom5' if, at some point, you created a 'dom2' and 'dom4' and then destroyed them.

ethX

Normally, this is the name of a "main" ethernet device. However, under Xen, this does NOT reflect the real hardware once the xend daemon has started!

Under Xen, what 'ethX' refers to depends on where you are. Under dom0, 'ethX' is a virtual ethernet device which has a copy of the real ethernet device's MAC and IP addresses. On domU, 'ethX' is a virtual ethernet device connecting to a matching 'vifX.Y device created on dom0 by Xen when domU was started. See the vifX.Y definition below.

Replace 'X' in 'ethX' with a number reflecting a given ethernet device. For example, 'eth0' is the first ethernet device, 'eth1' the second and so on.

pethX

The pethX device(s) exist only on dom0.

When xend starts, it renames the "real" 'ethX' to 'pethX' in order to create a virtual copy of it. This is done to facilitate bridging and routing. So then, pethX devices are in fact the interfaces to the real hardware ethernet devices.

Replace 'X' in 'pethX' with a number reflecting a given ethernet device. For example, 'peth0' is the first ethernet device, 'peth1' the second and so on.

vethX

The vethX device(s) exist only on dom0 and only for a short time during xend startup.

During the Xen startup process, ethX is brought down and it's MAC and IP addresses are copied to vethX. Once done, ethX is renamed to pethX to move it out of the way. Then vethX is renamed to ethX to take the place of the pethX interface. As such, vethX devices exist for only a short time. You should never need to worry about or factor in these devices into your networking plans.

Replace 'X' in 'vethX' with a number reflecting a given virtual ethernet device. For example, 'veth0' is the first virtual ethernet device, 'veth1' the second and so on.

vifX.Y

The vifX.Y devices exist only on dom0.

Note; in this section, 'X' matches the domU number, 'Y' matches a given ethX device on dom0 and 'Z' matches a given ethX device inside a domU. I know this is a little odd, but you need to get this clear in order to see how things connect here.

These are Xen's virtual interfaces used to connect an ethX device on dom0 to a pethZ device on a domU. Please be aware that there need not be any correlation between eth0 on dom0 and eth0 on domU! This is why 'X' and 'Z' are used separately here.

Specifically, when Xen starts a domU, that virtual machine's ID# plus a given dom0 ethX device number are used to create one or more virtual interfaces inside dom0. For example, if you want to connect eth0 on dom0 to eth0 on dom4, you will do so via vif4.0 where '4' matches the domU ID# and '0' matches the ethernet device number in dom0. The actual routing goes through xenbr0. If this is still confusing, take a look at the network map below.

xenbrX

The xenbrX device(s) exist only on dom0.

Xen uses software bridges to route traffic within it's virtual network. These can be thought of as a "switch" your would find in any normal network. They serve to connect a given ethernet device on dom0 to a given ethernet device in a domU virtual machine. A different bridge is created for each ethX device on dom0.

Network Map - Putting it all together

This map is a sample of a four-VM setup in a single Xen host. You will notice that eth1 is not connected to any domU's because, in this case, it is used for the DRBD channel. All virtual machines have a connection to eth0 because that is the virtual machine network's "back channel". In practice, there is no internet connection on eth0, and thus on the xenbr0 bridge, except through the firewall, dom1. The dom6 shows a link to eth2, which would give it a direct connection to the Internet. 

  _________________________________________________________________
 | dom0                                           _______________  |
 |_______  ______   ________                     |______    dom1 | |
 = peth0 |= eth0 |-| xenbr0 |-----------[vif1.0]-= eth0 |        | |
 |-______| |_____| |        |---------\          |-_____|        | |
 |                 |        |-------\ |          |               | |
 |                 |        |-----\ | |          |______         | |
 |                 |        |   /-+-+-+-[viv1.2]-= eth1 |        | |
 |                 |        |   | | | |          |-_____|        | |
 |                 |________|   | | | |          |_______________| |
 |                              | | | |                            |
 |                              | | | |           _______________  |
 |_______  ______   ________    | | | |          |______    dom2 | |
 = peth1 |= eth1 |-| xenbr1 |   | | | \-[viv2.0]-= eth0 |        | |
 |-______| |_____| |        |   | | |            |-_____|        | |
 |                 |        |   | | |            |               | |
 |                 |        |   | | |            |               | |
 |                 |        |   | | |            |               | |
 |                 |        |   | | |            |               | |
 |                 |________|   | | |            |_______________| |
 |                              | | |             _______________  |
 |_______  ______   ________    | | |            |______    dom4 | |
 = peth2 |= eth2 |-| xenbr2 |---/ | \---[viv4.0]-= eth0 |        | |
 |-______| |_____| |        |---\ |              |-_____|        | |
 |                 |        |   | |              |               | |
 |                 |        |   | |              |               | |
 |                 |        |   | |              |               | |
 |                 |        |   | |              |               | |
 |                 |________|   | |              |_______________| |
 |                              | |               _______________  |
 |                              | |              |______    dom6 | |
 |                              | \-----[viv6.0]-= eth0 |        | |
 |                              |                |-_____|        | |
 |                              |                |               | |
 |                              |                |______         | |
 |                              \-------[viv6.2]-= eth1 |        | |
 |                                               |-_____|        | |
 |                                               |               | |
 |                                               |_______________| |
 |_________________________________________________________________|

This is one an example of networking inside a two node cluster.

 

Any questions, feedback, advice, complaints or meanderings are welcome.
Alteeve's Niche! Enterprise Support:
Alteeve Support 		Community Support
© Alteeve's Niche! Inc. 1997-2024   Anvil! "Intelligent Availability®" Platform
legal stuff: All info is provided "As-Is". Do not use anything here unless you are willing and able to take responsibility for your own actions.
Retrieved from ‘https://alteeve.com/w/index.php?title=Networking_in_Xen_3.x&oldid=1558’