Configuring Brocade Switches: Difference between revisions

From Alteeve Wiki
Jump to navigation Jump to search
 
(33 intermediate revisions by one other user not shown)
Line 13: Line 13:
= Serial Connection =
= Serial Connection =


You can use <span class="code">screen</span> to connect to the switch (default values are fine).
{{note|1=Depending on your terminal emulator, the <span class="code"><Backspace></span> key may not work. If it doesn't, try pressing <span class="code">ctrl + h</span> to delete the character to the left of your cursor.}}
 
You can use <span class="code">screen</span> to connect to the switch (default values are fine). The example below is what you would use on most servers with an on-board serial port. If you have two, then port two would be <span class="code">/dev/ttyS1</span>. If you are using a USB to serial adapter, then you will use <span class="code">/dev/ttyUSB0</span> (or if it is a multi-port adapter, <span class="code">/dev/ttyUSB1</span>, etc). If none of these work, you will need to consult your operating system documentation and/or serial port adapter to determine the proper <span class="code">/dev/ttyX</span> device to use.


<syntaxhighlight lang="text">
<syntaxhighlight lang="text">
Line 22: Line 24:


<syntaxhighlight lang="text">
<syntaxhighlight lang="text">
ICX6610-24 Switch>
ICX6610-48 Switch>
</syntaxhighlight>
</syntaxhighlight>
To log in;
<syntaxhighlight lang="text">
<syntaxhighlight lang="text">
ICX6610-24 Switch>enable  
ICX6610-48 Switch>enable  
No password has been assigned yet...
No password has been assigned yet...
</syntaxhighlight>
</syntaxhighlight>


There is no default user name and password. To use the web interface later, a user and password needs to be set.
There is no default user name and password.
 
== Exiting screen ==
 
To end your <span class="code">screen</span> session, press <span class="code">ctrl + a</span> together, release and then press <span class="code">k</span>.
 
You will be asked if you want to kill the session, press <span class="code">y</span> to confirm.
 
<syntaxhighlight lang="text">
Really kill this window [y/n]
</syntaxhighlight>
 
<syntaxhighlight lang="text">
[screen is terminating]
</syntaxhighlight>
 
== Configure a Password ==
 
To use the web interface later, a user and password needs to be set.


{{note|1=Super User level password can be an alphanumeric string, but cannot begin with a number.}}
{{note|1=Super User level password can be an alphanumeric string, but cannot begin with a number.}}


<syntaxhighlight lang="text">
<syntaxhighlight lang="text">
ICX6610-24 Switch#configure terminal  
ICX6610-48 Switch#configure terminal  
ICX6610-24 Switch(config)#enable super-user-password secret
</syntaxhighlight>
ICX6610-24 Switch(config)#enable user disable-on-login-failure 10
 
ICX6610-24 Switch(config)#user alteeve privilege 0 password secret  
<syntaxhighlight lang="text">
ICX6610-24 Switch(config)#show users
ICX6610-48 Switch(config)#enable super-user-password secret
</syntaxhighlight>
 
<syntaxhighlight lang="text">
ICX6610-48 Switch(config)#enable user disable-on-login-failure 10
</syntaxhighlight>
 
<syntaxhighlight lang="text">
ICX6610-48 Switch(config)#user alteeve privilege 0 password secret  
</syntaxhighlight>
 
<syntaxhighlight lang="text">
ICX6610-48 Switch(config)#show users
</syntaxhighlight>
 
<syntaxhighlight lang="text">
Username                                        Password                          Encrypt  Priv Status  Expire Time
Username                                        Password                          Encrypt  Priv Status  Expire Time
======================================================================================================================
======================================================================================================================
alteeve                                        $1$HF4..Zv.$uF.PUT.bqkMy5GicUIrhG/ enabled  0    enabled  Never
alteeve                                        $1$HF4..Zv.$uF.PUT.bqkMy5GicUIrhG/ enabled  0    enabled  Never
ICX6610-24 Switch(config)#write memory
</syntaxhighlight>
 
To enable the web interface;
 
<syntaxhighlight lang="text">
ICX6610-48 Switch(config)#aaa authentication web-server default local
</syntaxhighlight>
 
<syntaxhighlight lang="text">
ICX6610-48 Switch(config)#write memory
</syntaxhighlight>
 
<syntaxhighlight lang="text">
Write startup-config done.
Write startup-config done.


Automatic copy to member units:  2
Automatic copy to member units:  2
ICX6610-24 Switch(config)#aaa authentication web-server default local
Flash Memory Write (8192 bytes per dot) .
ICX6610-24 Switch(config)#Flash Memory Write (8192 bytes per dot) .
Flash to Flash Done.
Flash to Flash Done.
</syntaxhighlight>
</syntaxhighlight>
= Form the Stack =
= Form the Stack =


Line 55: Line 105:


<syntaxhighlight lang="text">
<syntaxhighlight lang="text">
> enable
ICX6610-48 Switch>enable
# config terminal
</syntaxhighlight>
## (config) stack enable
 
## (config) exit
<syntaxhighlight lang="text">
# show stack
ICX6610-48 Switch#configure terminal
# stack secure-setup
</syntaxhighlight>
Do you accept the topology (RING) (y/n)?: y
 
Do you accept the unit id's (y/n)?: y
<syntaxhighlight lang="text">
ICX6610-48 Switch(config)#stack enable
</syntaxhighlight>
 
<syntaxhighlight lang="text">
Enable stacking. This unit actively participates in stacking
stacking is enable. optical monitoring for stacking ports 1/2/1, 1/2/6 is not available.
</syntaxhighlight>
 
<syntaxhighlight lang="text">
ICX6610-48 Switch(config)#exit
</syntaxhighlight>
 
<syntaxhighlight lang="text">
ICX6610-48 Switch#stack secure-setup
</syntaxhighlight>
</syntaxhighlight>


(switches reboot)
<syntaxhighlight lang="text">
Discovering the stack topology...
 
Current Discovered Topology - RING
 
Available UPSTREAM units
Hop(s) Id      Type          Mac Address
1      new  ICX6610-48    xxxx.xxxx.xxxx
 
Available DOWNSTREAM units
Hop(s)  Id      Type          Mac Address
1      new  ICX6610-48    xxxx.xxxx.xxxx
</syntaxhighlight>


<syntaxhighlight lang="text">
<syntaxhighlight lang="text">
> enable
Do you accept the topology (RING) (y/n)?: y
# show stack
</syntaxhighlight>
</syntaxhighlight>
<syntaxhighlight lang="text">
<syntaxhighlight lang="text">
ICX6610-24 Switch#show stack
Selected Topology:
alone: standalone, D: dynamic config, S: static config
Active  Id      Type          Mac Address
ID  Type         Role   Mac Address    Pri State  Comment                 
        1    ICX6610-48    yyyy.yyyy.yyyy
1 S ICX6610-24  active 748e.f8ff.cf1c 128 local  Ready                             
 
Selected UPSTREAM units
Hop(s)  Id      Type       Mac Address
1      2   ICX6610-48    xxxx.xxxx.xxxx
 
Selected DOWNSTREAM units
Hop(s)  Id      Type        Mac Address
1      2   ICX6610-48    xxxx.xxxx.xxxx
</syntaxhighlight>
 
<syntaxhighlight lang="text">
Do you accept the unit id's (y/n)?: y
</syntaxhighlight>
 
<syntaxhighlight lang="text">
T=1d20h24m2.8: Election, was alone --> active, ID=1, pri=128, 2U(1-2), A=u1, nbr#=1 1,
T=1d20h24m3.5: Election, was active, no change, ID=1, pri=128, 2U(1-2), A=u1, nbr#=1 1,
reset unit 2: u2 diff bootup id=1
Unit 1 loses all neighbors.
Active unit 1 deletes u2 and its config because it is learned.
 
Config changed due to add/del units. Do write mem if you want to keep it
</syntaxhighlight>
 
(switch reboots)


    active                                                                   
<syntaxhighlight lang="text">
    +---+                                                                   
T=1d20h26m43.2: Election, was active, no change, ID=1, pri=128, 2U(1-2), A=u1, nbr#=1 1,  
=2/6| 1 |2/1=                                                               
    +---+                                                                   
                                                                             
Current stack management MAC is 748e.f8ff.cf1c
Note: no "stack mac" config. My MAC will change after failover.
ICX6610-24 Switch#T=1h25m22.3: Election, was active, no change, ID=1, pri=128, 2U(1-2), A=u1, nbr#=1 1,  
Done hot swap: active controller unit 1 sets unit 2 to Ready.
Done hot swap: active controller unit 1 sets unit 2 to Ready.


Config changed due to add/del units. Do write mem if you want to keep it
Config changed due to add/del units. Do write mem if you want to keep it
</syntaxhighlight>
<syntaxhighlight lang="text">
ICX6610-48 Switch#write memory
</syntaxhighlight>
<syntaxhighlight lang="text">
Write startup-config done.
Automatic copy to member units:  2
Flash Memory Write (8192 bytes per dot) .
Flash to Flash Done.
</syntaxhighlight>
A moment later, more boot messages.
<syntaxhighlight lang="text">
Stack unit 2 Power supply 1 is up
Stack unit 2 Power supply 1 is up
Stack unit 2 Power supply 2 is up
Stack unit 2 Power supply 2 is up
Line 93: Line 204:
start running config sync
start running config sync
Running config sync to standby is complete
Running config sync to standby is complete
</syntaxhighlight>


ICX6610-24 Switch#show stack
<syntaxhighlight lang="text">
ICX6610-48 Switch#show stack  
</syntaxhighlight>


<syntaxhighlight lang="text">
alone: standalone, D: dynamic config, S: static config
alone: standalone, D: dynamic config, S: static config
ID  Type        Role    Mac Address    Pri State  Comment                   
ID  Type        Role    Mac Address    Pri State  Comment                   
1  S ICX6610-24   active  748e.f8ff.cf1c 128 local  Ready                            
1  S ICX6610-48   active  xxxx.xxxx.xxxx 128 local  Ready
D ICX6610-24   standby 748e.f8ff.cf84   0 remote  Ready                            
S ICX6610-48   standby yyyy.yyyy.yyyy   0 remote  Ready


     active      standby                                                       
     active      standby                                                       
     +---+        +---+                                                         
     +---+        +---+                                                         
  =2/1| 1 |2/6==2/6| 2 |2/1=                                                     
  =2/1| 1 |2/6==2/1| 2 |2/6=                                                     
  |  +---+        +---+  |                                                     
  |  +---+        +---+  |                                                     
  |                        |                                                     
  |                        |                                                     
Line 109: Line 224:
                                                                                
                                                                                
Standby u2 - No hitless failover. Reason: hitless-failover not configured
Standby u2 - No hitless failover. Reason: hitless-failover not configured
Current stack management MAC is 748e.f8ff.cf1c
Current stack management MAC is xxxx.xxxx.xxxx
Note: no "stack mac" config. My MAC will change after failover.
Note: no "stack mac" config. My MAC will change after failover.
</syntaxhighlight>
</syntaxhighlight>
Configure the stack to handle switch "hitless" failure and recovery.


<syntaxhighlight lang="text">
<syntaxhighlight lang="text">
# write memory
ICX6610-48 Switch#config terminal
</syntaxhighlight>


<syntaxhighlight lang="text">
ICX6610-48 Switch(config)#hitless-failover enable
</syntaxhighlight>
Assign a static [[MAC]] to the stack. In this case, I always copy stack unit 1's real MAC address.
<syntaxhighlight lang="text">
ICX6610-48 Switch(config)#stack mac cc4e.24b9.5624
</syntaxhighlight>
Last, switch to unit 2 and change it's priority to match the priority of unit 1.
<syntaxhighlight lang="text">
ICX6610-48 Switch(config)#stack unit 2
</syntaxhighlight>
<syntaxhighlight lang="text">
ICX6610-48 Switch(config-unit-2)#priority 128
</syntaxhighlight>
<syntaxhighlight lang="text">
Priority change needs 120 seconds to take effect.
</syntaxhighlight>
<syntaxhighlight lang="text">
ICX6610-48 Switch(config-unit-2)#exit
</syntaxhighlight>
<syntaxhighlight lang="text">
ICX6610-48 Switch(config)#write memory
</syntaxhighlight>
<syntaxhighlight lang="text">
Write startup-config done.
Write startup-config done.


Automatic copy to member units:  2
Automatic copy to member units:  2
ICX6610-24 Switch#Flash Memory Write (8192 bytes per dot) .
Flash Memory Write (8192 bytes per dot) .
Flash to Flash Done.
Flash to Flash Done.
</syntaxhighlight>
</syntaxhighlight>


== ICX6430-C12 ==
Two minutes after you change the stack priority, the switch run an election.
 
<syntaxhighlight lang="text">
Will do election in 30 sec due to priority change
Will do election in 10 sec due to priority change
T=1d20h35m20.2: Election, was active, no change, ID=1, pri=128, 2U(1-2), A=u1, nbr#=1 1,
</syntaxhighlight>
 
Verify the stack is configured the way we want.
 
<syntaxhighlight lang="text">
ICX6610-48 Switch(config)#show stack
</syntaxhighlight>


<syntaxhighlight lang="text">
<syntaxhighlight lang="text">
alone: standalone, D: dynamic config, S: static config
ID  Type        Role    Mac Address    Pri State  Comment                 
1  S ICX6610-48  active  xxxx.xxxx.xxxx 128 local  Ready
2  S ICX6610-48  standby yyyy.yyyy.yyyy 128 remote  Ready
    active      standby                                                     
    +---+        +---+                                                       
=2/1| 1 |2/6==2/1| 2 |2/6=                                                   
|  +---+        +---+  |                                                   
|                        |                                                   
|------------------------|                                                   
                                                                             
Standby u2 - protocols ready, can failover or manually switch over
Current stack management MAC is xxxx.xxxx.xxxx
</syntaxhighlight>
</syntaxhighlight>
Excellent, done.
== ICX6430-C12 ==
The ICX6430-C12 doesn't support stacking. However, a stack can be somewhat simulated by tagging a dedicated port with all our VLANs. This will allow traffic from any VLAN to use the same "uplink" port. Also, without a proper stack, we can not create a trunk/LAG. So we enable rapid STP on each switch's customer-facing uplink port instead.
* [[#Pseudo_Stacking_on_ICX6430-C12|Pseudo Stacking on ICX6430-C12]]


= Enable Jumbo Frames =
= Enable Jumbo Frames =
Line 133: Line 318:


<syntaxhighlight lang="text">
<syntaxhighlight lang="text">
ICX6610-24 Switch>enable
ICX6610-48 Switch#configure terminal
Password:
</syntaxhighlight>
ICX6610-24 Switch#terminal config
 
ICX6610-24 Switch(config)#jumbo  
<syntaxhighlight lang="text">
ICX6610-48 Switch(config)#jumbo
</syntaxhighlight>
 
<syntaxhighlight lang="text">
Jumbo mode setting requires a reload to take effect!
Jumbo mode setting requires a reload to take effect!
ICX6610-24 Switch(config)#write memory
</syntaxhighlight>
 
<syntaxhighlight lang="text">
ICX6610-48 Switch(config)#write memory
Write startup-config done.
Write startup-config done.


Automatic copy to member units:  2
Automatic copy to member units:  2
ICX6610-24 Switch(config)#Flash Memory Write (8192 bytes per dot) .
Flash Memory Write (8192 bytes per dot) .
Flash to Flash Done.
Flash to Flash Done.
ICX6610-24 Switch(config)#exit
</syntaxhighlight>
ICX6610-24 Switch#reload
 
<syntaxhighlight lang="text">
ICX6610-48 Switch(config)#exit
</syntaxhighlight>
 
<syntaxhighlight lang="text">
ICX6610-48 Switch#reload
</syntaxhighlight>
 
<syntaxhighlight lang="text">
Are you sure? (enter 'y' or 'n'): y
Are you sure? (enter 'y' or 'n'): y
</syntaxhighlight>
<syntaxhighlight lang="text">
Reload request sent to attached stack member(s)...
Reload request sent to attached stack member(s)...
ICX6610-24 Switch#
ICX6610-24 Switch#HRebooting(0)...
lt and reboot
*
$
ICX Boot Code Version 7.3.02 (grz07302)
Enter 'a' to stop at memory test
Enter 'b' to stop at boot monitor
BOOT INFO: load monitor from boot flash, cksum = 1fef
BOOT INFO: verify flash files.......
BOOT INFO: load image from primary copy...


platform type = 8
Rebooting(0)...
PCIE-1 LTSSM status: 22
</syntaxhighlight>
PCIE Switch status: 0
..................
....
Starting Main Task ...CPSS DxCh Version: cpss3.4p1 release
Pre Parsing Config Data ...


Parsing Config Data ...
= Assign IP to Switch =
stacking is enable. optical monitoring for stacking ports 1/2/1, 1/2/6 is not available.


  Copyright (c) 1996-2012 Brocade Communications Systems, Inc. All rights reserved.
<syntaxhighlight lang="text">
    UNIT 1: compiled on Jul 08 2013 at 18:50:06 labeled as FCXS07400d
ICX6610-48 Switch#configure terminal
                (5702656 bytes) from Primary FCXS07400d.bin
</syntaxhighlight>
        SW: Version 07.4.00dT7f1
  Boot-Monitor Image size = 370555, Version:07.3.02T7f5 (grz07302)
  HW: Stackable ICX6610-24
==========================================================================
UNIT 1: SL 1: ICX6610-24 24-port Management Module
        Serial  #: BXP2547J053
        License: BASE_SOFT_PACKAGE  (LID: dzrHKJMlFKI)
        P-ENGINE  0: type E02B, rev 01
==========================================================================
UNIT 1: SL 2: ICX6610-QSFP 10-port 160G Module
==========================================================================
UNIT 1: SL 3: ICX6610-8-port Dual Mode(SFP/SFP+) Module
==========================================================================
  800 MHz Power PC processor 8544E (version 0021/0023) 400 MHz bus
65536 KB flash memory
  512 MB DRAM
STACKID 1  system uptime is 10 seconds
The system : started=warm start  reloaded=by "reload"
My stack unit ID = 1, bootup role = active


........................
<syntaxhighlight lang="text">
T=1m39.9: Election, was alone --> active, ID=1, pri=128, 2U(1-2), A=u1, nbr#=1 1,
ICX6610-48 Switch(config)#ip address 10.20.1.5 255.255.0.0
ICX6610-24 Switch>Done hot swap: active controller unit 1 sets unit 2 to Ready.
</syntaxhighlight>


Power supply 1  detected.
<syntaxhighlight lang="text">
ICX6610-48 Switch(config)#exit
</syntaxhighlight>


Power supply 2  detected.
The router is at <span class="code">10.20.255.254</span>, so ping that to verify the IP is configured.
Power supply 1  is up.
Power supply 2  is up.


ICX6610-24 Switch>
<syntaxhighlight lang="text">
ICX6610-24 Switch>
ICX6610-48 Switch#ping 10.20.255.254
ICX6610-24 Switch>Stack unit 2 Power supply 1 is up
</syntaxhighlight>
Stack unit 2 Power supply 2 is up


ICX6610-24 Switch>
<syntaxhighlight lang="text">
Sending 1, 16-byte ICMP Echo to 10.20.255.254, timeout 5000 msec, TTL 64
Type Control-c to abort
Reply from 10.20.255.254  : bytes=16 time<1ms TTL=64
Success rate is 100 percent (1/1), round-trip min/avg/max=0/0/0 ms.
</syntaxhighlight>
</syntaxhighlight>


Good, save the changes.


= Assign IP to Switch =
<syntaxhighlight lang="text">
ICX6610-48 Switch#write memory
</syntaxhighlight>


<syntaxhighlight lang="text">
<syntaxhighlight lang="text">
> enable
Write startup-config done.
# config terminal
## (config) ip address 10.255.8.1 255.255.0.0
## (config) exit
# ping 10.255.8.1


Ping self done.
Automatic copy to member units: 2
 
Flash Memory Write (8192 bytes per dot) .
# ping 10.255.255.254
Flash to Flash Done.
 
Sending 1, 16-byte ICMP Echo to 10.255.255.254, timeout 5000 msec, TTL 64
Type Control-c to abort
Reply from 10.255.255.254 : bytes=16 time=2ms TTL=64
Success rate is 100 percent (1/1), round-trip min/avg/max=2/2/2 ms.
</syntaxhighlight>
</syntaxhighlight>


= Update the firmware =
= Update the firmware =


{{note|1=These switches have two firmwares;
{{warning|1=Be sure to have your switches on UPSes before flashing them! As with all devices, a power loss during the flash process may cause the switch to become unusable.}}
* Layer 2 (S)
 
**  Compressed Pri Code size = 5325231, Version:07.3.00cT7f1 (/foundry/FGS/os/FCXS07300c.bin)
== Selecting the Firmware Version ==
* Layer 3 (R)
**  Compressed Sec Code size = 6811631, Version:07.3.00cT7f3 (/foundry/FGS/os/FCXR07300c.bin)}}


Create a my.brocade.com account
Before we start, we need to decide what version of the firmware we want to install. To do this, start by logging in to your switch(es) and check their current firmware version.


<syntaxhighlight lang="text">
<syntaxhighlight lang="text">
> enable
ICX6610-48 Switch#show flash
# config terminal
</syntaxhighlight>
## (config) show flash
<syntaxhighlight lang="text">
 
Stack unit 1:
Stack unit 1:
   Compressed Pri Code size = 5325231, Version:07.3.00cT7f1 (/foundry/FGS/os/FCXS07300c.bin)
   Compressed Pri Code size = 5370497, Version:07.3.00fT7f1 (/foundry/FGS/os/FCXS07300f.bin)
   Compressed Sec Code size = 6811631, Version:07.3.00cT7f3 (/foundry/FGS/os/FCXR07300c.bin)
   Compressed Sec Code size = 6905002, Version:07.3.00fT7f3 (/foundry/FGS/os/FCXR07300f.bin)
   Compressed Boot-Monitor Image size = 370555, Version:07.3.02T7f5
   Compressed Boot-Monitor Image size = 370555, Version:07.3.02T7f5
   Code Flash Free Space = 53215232
   Code Flash Free Space = 52822016
Stack unit 2:
Stack unit 2:
   Compressed Pri Code size = 5325231, Version 07.3.00cT7f1 (/foundry/FGS/os/FCXS07300c.bin)
   Compressed Pri Code size = 5370497, Version 07.3.00fT7f1 (/foundry/FGS/os/FCXS07300f.bin)
   Compressed Sec Code size = 6811631, Version 07.3.00cT7f3 (/foundry/FGS/os/FCXR07300c.bin)
   Compressed Sec Code size = 6905002, Version 07.3.00fT7f3 (/foundry/FGS/os/FCXR07300f.bin)
   Compressed Boot-Monitor Image size = 370555, Version 07.3.02T7f5
   Compressed Boot-Monitor Image size = 370555, Version 07.3.02T7f5
   Code Flash Free Space = 53084160
   Code Flash Free Space = 52822016
</syntaxhighlight>
 
Note that both switches are running the major version <span class="code">07.3.00f</span>. Being that we're interested in High Availability, we are going to be conservative and upgrade to the latest subversion available. Both switches are 7.3, so that is what we will use. If one had been 7.4, for example, then we would have upgraded both to the latest 7.4 version.
 
== Layer 2 vs Layer 3 ==
 
These switches have two firmwares;
* Layer 2 (<span class="code">S</span>); This is seen in the above output in the current version's flash file "<span class="code">/foundry/FGS/os/FCX'''S'''07300f.bin</span>".
* Layer 3 (<span class="code">R</span>); This is seen in the above output in the current version's flash file "<span class="code">/foundry/FGS/os/FCX'''R'''07300f.bin</span>".
 
We're only going to update the layer 2 "<span class="code">S</span>" firmware. We don't use the layer 3 features for HA so we won't update them. If you want to updated them, that is fine. The upgrade process below is easily ported to the layer 3 firmware update process.
 
== Downloading the Firmware ==
 
To get the firmware:
* Create or log into your [http://my.brocade.com My Brocade] account.
* Click on '<span class="code">Downloads</span>'.
* In the <span class="code">Download by</span> select box, select <span class="code">Ethernet Switches</span>.
* Scroll down and click to expand the arrow to the left of "<span class="code">ICX6610</span>" (or your switch model).
* Click to expand the version of firmware you want to use. For this tutorial, we're going to be conservative and stay in the same "7.3" release, so we will expand <span class="code">07300</span>.
* Click to expand the most recent subversion for your selected firmware major number. In my case, that is "<span class="code">07300j</span>".
* This takes you to a note about export restrictions. Ensure you meet the criteria and then click to acknowledge compliance, the click on "<span class="code">Submit</span>".
* You will see the EULA. Read it, understand it and then agree to it by clicking on "<span class="code">I Accept</span>".
* Save the file on your computer.
 
== Extracting the Image ==
 
Navigate to the downloaded file. It will be a <span class="code">.zip</span> file. Expand it. This will create several directories.
 
To upload the firmware, we need to copy the image to the root of our [[Setting_Up_a_PXE_Server_on_an_RPM-based_OS#tftp|TFTP]] server (<span class="code">/var/lib/tftpboot/</span> on [[EL6]]).
 
Browse into the extracted <span class="code">07300j/ICX/Images/</span> directory (or the matching directory for your firmware). You will see two file:
 
<syntaxhighlight lang="bash">
cd 07300j/ICX/Images/
ls -lah
</syntaxhighlight>
<syntaxhighlight lang="text">
total 12M
drwxr-xr-x 2 root root 4.0K May  8  2014 .
drwxr-xr-x 8 root root 4.0K May  9  2014 ..
-rwxr-xr-x 1 root root 6.7M May  8  2014 FCXR07300j.bin
-rwxr-xr-x 1 root root 5.2M May  8  2014 FCXS07300j.bin
</syntaxhighlight>


## (config) exit
As discussed above, we will be upgrading the layer 2 "<span class="code">S</span>" image. So this is the file we will copy to the TFTP root directory.
# copy tftp flash 10.255.255.254 FCXS07400d.bin primary


ICX6610-24 Switch#Flash Memory Write (8192 bytes per dot)  
<syntaxhighlight lang="bash">
cp FCXS07300j.bin /var/lib/tftpboot/
</syntaxhighlight>
<syntaxhighlight lang="text">
ls -lah /var/lib/tftpboot/FCXS07300j.bin
</syntaxhighlight>
 
== Performing the Flash ==
 
{{note|1=We will be flashing both switches at once as they are stacked.}}
 
We're ready to flash! So it's time to log into the switch.
 
<syntaxhighlight lang="text">
ICX6610-48 Switch#copy tftp flash 10.20.255.254 FCXS07300j.bin primary
</syntaxhighlight>
 
<syntaxhighlight lang="text">
Flash Memory Write (8192 bytes per dot)  
Automatic copy to member units:  2
Automatic copy to member units:  2
....<lots of dots>....
....<lots of dots>....
TFTP to Flash Done.
TFTP to Flash Done.
</syntaxhighlight>


# show flash
<syntaxhighlight lang="text">
ICX6610-48 Switch#show flash
</syntaxhighlight>


<syntaxhighlight lang="text">
Stack unit 1:
Stack unit 1:
   Compressed Pri Code size = 5702656, Version:07.4.00dT7f1 (FCXS07400d.bin)
   Compressed Pri Code size = 5430491, Version:07.3.00jT7f1 (FCXS07300j.bin)
   Compressed Sec Code size = 6811631, Version:07.3.00cT7f3 (/foundry/FGS/os/FCXR07300c.bin)
   Compressed Sec Code size = 6905002, Version:07.3.00fT7f3 (/foundry/FGS/os/FCXR07300f.bin)
   Compressed Boot-Monitor Image size = 370555, Version:07.3.02T7f5
   Compressed Boot-Monitor Image size = 370555, Version:07.3.02T7f5
   Code Flash Free Space = 52822016
   Code Flash Free Space = 52690944
Stack unit 2:
Stack unit 2:
   Compressed Pri Code size = 5702656, Version 07.4.00dT7f1 (FCXS07400d.bin)
   Compressed Pri Code size = 5430491, Version 07.3.00jT7f1 (FCXS07300j.bin)
   Compressed Sec Code size = 6811631, Version 07.3.00cT7f3 (/foundry/FGS/os/FCXR07300c.bin)
   Compressed Sec Code size = 6905002, Version 07.3.00fT7f3 (/foundry/FGS/os/FCXR07300f.bin)
   Compressed Boot-Monitor Image size = 370555, Version 07.3.02T7f5
   Compressed Boot-Monitor Image size = 370555, Version 07.3.02T7f5
   Code Flash Free Space = 52690944
   Code Flash Free Space = 52690944
</syntaxhighlight>


# write memory
Reboot to load the new firmware.


Write startup-config done.
<syntaxhighlight lang="text">
 
ICX6610-48 Switch#write memory
Automatic copy to member units:  2
ICX6610-48 Switch#reload
Flash Memory Write (8192 bytes per dot) .
</syntaxhighlight>
Flash to Flash Done.
 
# reload


<syntaxhighlight lang="text">
Are you sure? (enter 'y' or 'n'): y
Are you sure? (enter 'y' or 'n'): y
Reload request sent to attached stack member(s)...
Rebooting(0)...
</syntaxhighlight>
</syntaxhighlight>


Line 297: Line 521:


<syntaxhighlight lang="text">
<syntaxhighlight lang="text">
> enable
ICX6610-48 Switch#show flash
# show version
</syntaxhighlight>


  Copyright (c) 1996-2012 Brocade Communications Systems, Inc. All rights reserved.
<syntaxhighlight lang="text">
    UNIT 1: compiled on Jul 08 2013 at 18:50:06 labeled as FCXS07400d
Stack unit 1:
                (5702656 bytes) from Primary FCXS07400d.bin
  Compressed Pri Code size = 5430491, Version:07.3.00jT7f1 (FCXS07300j.bin)
        SW: Version 07.4.00dT7f1
  Compressed Sec Code size = 6905002, Version:07.3.00fT7f3 (/foundry/FGS/os/FCXR07300f.bin)
    UNIT 2: compiled on Jul 08 2013 at 18:50:06 labeled as FCXS07400d
   Compressed Boot-Monitor Image size = 370555, Version:07.3.02T7f5
                (5702656 bytes) from Primary FCXS07400d.bin
   Code Flash Free Space = 52690944
        SW: Version 07.4.00dT7f1
Stack unit 2:
   Boot-Monitor Image size = 370555, Version:07.3.02T7f5 (grz07302)
   Compressed Pri Code size = 5430491, Version 07.3.00jT7f1 (FCXS07300j.bin)
   HW: Stackable ICX6610-24
  Compressed Sec Code size = 6905002, Version 07.3.00fT7f3 (/foundry/FGS/os/FCXR07300f.bin)
==========================================================================
   Compressed Boot-Monitor Image size = 370555, Version 07.3.02T7f5
UNIT 1: SL 1: ICX6610-24 24-port Management Module
  Code Flash Free Space = 52690944
        Serial  #: xxxx
        License: BASE_SOFT_PACKAGE   (LID: dzrHKJMlFKI)
        P-ENGINE  0: type E02B, rev 01
==========================================================================
UNIT 1: SL 2: ICX6610-QSFP 10-port 160G Module
==========================================================================
UNIT 1: SL 3: ICX6610-8-port Dual Mode(SFP/SFP+) Module
==========================================================================
UNIT 2: SL 1: ICX6610-24 24-port Management Module
        Serial  #: xxxx
        License: BASE_SOFT_PACKAGE  (LID: dzrHKJMlFKd)
        P-ENGINE  0: type E02B, rev 01
==========================================================================
UNIT 2: SL 2: ICX6610-QSFP 10-port 160G Module
==========================================================================
UNIT 2: SL 3: ICX6610-8-port Dual Mode(SFP/SFP+) Module
==========================================================================
  800 MHz Power PC processor 8544E (version 0021/0023) 400 MHz bus
65536 KB flash memory
   512 MB DRAM
STACKID 1  system uptime is 4 minutes 2 seconds
STACKID 2  system uptime is 4 minutes 2 seconds
The system : started=warm start  reloaded=by "reload"
My stack unit ID = 1, bootup role = active
</syntaxhighlight>
</syntaxhighlight>
Done!


= Configure VLAN =
= Configure VLAN =
[[VLAN]]s provide a way of dividing up a switch (or stack of switches) into multiple, isolated switches (this is actually somewhat simplistic, but it gets the idea across). Traffic from one VLAN segment can not reach ports allocated to another VLAN, providing a useful layer of security.
== Planning ==


We're creating three separate [[VLAN]]s;
We're creating three separate [[VLAN]]s;
* VID 100 ([[BCN]])
* [[VLAN]] ID: <span class="code">100</span> for the [[BCN]].
* VID 200 ([[SN]])
* [[VLAN]] ID: <span class="code">200</span> for the [[SN]].
* VID 300 ([[IFN]])
* [[VLAN]] ID: <span class="code">300</span> for the ([[IFN]].
 
By default, all ports default to [[VID]] <span class="code">1</span>. If you want to improve security, you can leave all unused ports on VID <span class="code">1</span>. In this tutorial, we need to allocate all ports for use by other equipment.
 
Before you start, you need to decide how you want to assign physical ports on your switches to VLAN IDs. I like to do this by creating a table showing all the ports, label their external port numbers and logical numbers and then write in what will be plugged into each port. With this, it's fairly easy to mark what ports will be assigned to what VLANs.
 
=== Cabling a pair of ICX6450-24 ===
 
In this example, we're showing how to cable up a pair of [http://www.brocade.com/products/all/switches/product-details/icx-6430-and-6450-switches/specifications.page ICX6450-24] switches in a very typical configuration. It can drive two [[Anvil!]] systems along with the usual foundation pack.
 
[[Image:brocade_icx6450-24_iso_01.png|thumb|center|1080px|The table below represents a pair of these stacked together.]]
 
{|class="wikitable"
!an-switch01
!colspan="16"|&nbsp;
|-
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; background-color: #f7f7f7; text-align: center;" colspan="2"|Stack
|class="null"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; background-color: #FFFFFF; text-align: center;" colspan="5"|Back-Channel Network]]
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; background-color: #99FF66; text-align: center;" colspan="3"|Storage Network
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; background-color: #2f2f2f; color: #cfcfcf; text-align: center;" colspan="5"|Internet-Facing Network
|-
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|X1 (1/2/1)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|X3 (1/2/3)
|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|1 (1/1/1)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|3 (1/1/3)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|5 (1/1/5)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|7 (1/1/7)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|9 (1/1/9)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|11 (1/1/11)
|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|13 (1/1/13)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|15 (1/1/15)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|17 (1/1/17)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|19 (1/1/19)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|21 (1/1/21)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|23 (1/1/23)
|-
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #1f1f1f; text-align: center;"|To an-switch02<br />X3
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #1f1f1f; text-align: center;"|To an-switch02<br />X1
|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|an-a01n01<br />BCN - Link 1
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|an-a01n01<br />IPMI / Fencing
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|an-a02n01<br />BCN - Link 1
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|an-a02n01<br />IPMI / Fencing
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|an-striker01<br />BCN - Link 2
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #005C00; text-align: center;"|an-a01n01<br />SN - Link 1
|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #005C00; text-align: center;"|an-a02n01<br />SN - Link 1
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|an-a01n01<br />IFN - Link 1
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|an-a02n01<br />IFN - Link 1
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|an-striker01<br />IFN - Link 1
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|--
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|--
|-
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #1f1f1f; text-align: center;"|--
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #1f1f1f; text-align: center;"|--
|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|an-a01n02<br />BCN - Link 1
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|an-a01n02<br />IPMI / Fencing
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|an-a02n02<br />BCN - Link 1
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|an-a02n02<br />BCN - Link 1
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|--
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #005C00; text-align: center;"|an-a01n02<br />SN - Link 1
|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #005C00; text-align: center;"|an-a02n02<br />SN - Link 1
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|an-a01n02<br />IFN - Link 1
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|an-a02n02<br />IFN - Link 1
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|--
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|--
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|Uplink 1
|-
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|X2 (1/2/2)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|X4 (1/2/4)
|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|2 (1/1/2)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|4 (1/1/4)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|6 (1/1/6)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|8 (1/1/8)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|10 (1/1/10)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|12 (1/1/12)
|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|14 (1/1/14)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|16 (1/1/16)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|18 (1/1/18)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|20 (1/1/20)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|22 (1/1/22)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|24 (1/1/24)
|-
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; background-color: #f7f7f7; text-align: center;" colspan="2"|Unused
|class="null"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; background-color: #FFFFFF; text-align: center;" colspan="5"|VID 100
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; background-color: #99FF66; text-align: center;" colspan="3"|VID 200
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; background-color: #2f2f2f; color: #cfcfcf; text-align: center;" colspan="5"|VID 300
|-
|colspan="16"|&nbsp;
|-
!an-switch02
!colspan="16"|&nbsp;
|-
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; background-color: #f7f7f7; text-align: center;" colspan="2"|Stack
|class="null"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; background-color: #FFFFFF; text-align: center;" colspan="5"|Back-Channel Network]]
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; background-color: #99FF66; text-align: center;" colspan="3"|Storage Network
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; background-color: #2f2f2f; color: #cfcfcf; text-align: center;" colspan="5"|Internet-Facing Network
|-
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|X1 (2/2/1)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|X3 (2/2/3)
|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|1 (2/1/1)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|3 (2/1/3)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|5 (2/1/5)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|7 (2/1/7)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|9 (2/1/9)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|11 (2/1/11)
|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|13 (2/1/13)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|15 (2/1/15)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|17 (2/1/17)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|19 (2/1/19)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|21 (2/1/21)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|23 (2/1/23)
|-
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #1f1f1f; text-align: center;"|To an-switch01<br />X3
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #1f1f1f; text-align: center;"|To an-switch01<br />X1
|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|an-a01n01<br />BCN - Link 2
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|an-pdu01
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|an-a02n01<br />BCN - Link 2
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|an-ups01
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|an-striker02<br />BCN - Link 1
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #005C00; text-align: center;"|an-a01n01<br />SN - Link 2
|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #005C00; text-align: center;"|an-a02n01<br />SN - Link 2
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|an-a01n01<br />IFN - Link 2
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|an-a02n01<br />IFN - Link 2
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|an-striker01<br />IFN - Link 2
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|--
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|--
|-
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #1f1f1f; text-align: center;"|--
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #1f1f1f; text-align: center;"|--
|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|an-a01n02<br />BCN - Link 2
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|an-pdu02
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|an-a02n02<br />BCN - Link 2
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|an-ups02
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|--
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #005C00; text-align: center;"|an-a01n02<br />SN - Link 2
|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #005C00; text-align: center;"|an-a02n02<br />SN - Link 2
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|an-a01n02<br />IFN - Link 2
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|an-a02n02<br />IFN - Link 2
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|--
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|--
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|Uplink 2
|-
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|X2 (2/2/2)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|X4 (2/2/4)
|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|2 (2/1/2)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|4 (2/1/4)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|6 (2/1/6)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|8 (2/1/8)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|10 (2/1/10)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|12 (2/1/12)
|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|14 (2/1/14)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|16 (2/1/16)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|18 (2/1/18)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|20 (2/1/20)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|22 (2/1/22)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|24 (2/1/24)
|-
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; background-color: #f7f7f7; text-align: center;" colspan="2"|Unused
|class="null"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; background-color: #FFFFFF; text-align: center;" colspan="5"|VID 100
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; background-color: #99FF66; text-align: center;" colspan="3"|VID 200
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; background-color: #2f2f2f; color: #cfcfcf; text-align: center;" colspan="5"|VID 300
|}


All ports default to VID 1. All unassigned ports remain in VID 1.
=== Cabling a pair of ICX6610-48 ===


<syntaxhighlight lang="text">
In this example, we're showing how to cable up a pair of [http://www.brocade.com/products/all/switches/product-details/icx-6610-switch/specifications.page ICX6610-48] switches in a rather unusual configuration. It is driving just one [[Anvil!]], but that Anvil! is using 10 Gbps SFP+ links. The switches are going to be populated with non-Anvil! equipment.
# config terminal
 
## (config) vlan 100 name bcn
What you will configure will, of course, totally vary. The specific ports aren't the point, the planning and implementation are.
### (config-vlan-100) untag ethernet 1/1/1 to 1/1/10
 
[[Image:brocade_icx6610-48_iso_01.png|thumb|center|1080px|The table below represents a pair of these stacked together.]]
 
{|class="wikitable"
!an-switch01
!colspan="29"|&nbsp;
!rowspan="15"|&nbsp;
|-
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; background-color: #FFFFFF; text-align: center;"|BCN
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; background-color: #99FF66; text-align: center;"|SN
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; background-color: #2f2f2f; color: #cfcfcf; text-align: center;" colspan="2"|IFN
|class="null"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; background-color: #FFFFFF; text-align: center;" colspan="8"|Back-Channel Network
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; background-color: #2f2f2f; color: #cfcfcf; text-align: center;" colspan="16"|Internet-Facing Network
|-
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #7f7f7f; text-align: center;"|X1 (1/2/1)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #7f8f7f; text-align: center;"|X3 (1/2/3)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|X5 (1/2/5)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|X7 (1/2/7)
|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #7f7f7f; text-align: center;"|1 (1/1/1)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #7f7f7f; text-align: center;"|3 (1/1/3)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #7f7f7f; text-align: center;"|5 (1/1/5)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #7f7f7f; text-align: center;"|7 (1/1/7)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #7f7f7f; text-align: center;"|9 (1/1/9)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #7f7f7f; text-align: center;"|11 (1/1/11)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #7f7f7f; text-align: center;"|13 (1/1/13)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #7f7f7f; text-align: center;"|15 (1/1/15)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|17 (1/1/17)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|19 (1/1/19)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|21 (1/1/21)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|23 (1/1/23)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|25 (1/1/25)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|27 (1/1/27)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|29 (1/1/29)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|31 (1/1/31)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|33 (1/1/33)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|35 (1/1/35)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|37 (1/1/37)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|39 (1/1/39)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|41 (1/1/41)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|43 (1/1/43)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|45 (1/1/45)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|47 (1/1/47)
|-
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|an-a05n01<br />BCN - Link 1
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #005C00; text-align: center;"|an-a05n01<br />SN - Link 1
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|an-a05n01<br />IFN - Link 1
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|an-a05n01<br />BCN - Link 3
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|an-a05n01<br />IPMI / Fencing
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|an-striker01<br />BCN - Link 1
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|an-striker01<br />IPMI
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|an-a05n01<br />IFN - Link 3
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|an-striker01<br />IFN - Link 1
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|-
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #7f7f7f; text-align: center;"|X2 (1/2/2)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #7f8f7f; text-align: center;"|X4 (1/2/4)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|X6 (1/2/6)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|X8 (1/2/8)
|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #7f7f7f; text-align: center;"|2 (1/1/2)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #7f7f7f; text-align: center;"|4 (1/1/4)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #7f7f7f; text-align: center;"|6 (1/1/6)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #7f7f7f; text-align: center;"|8 (1/1/8)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #7f7f7f; text-align: center;"|10 (1/1/10)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #7f7f7f; text-align: center;"|12 (1/1/12)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #7f7f7f; text-align: center;"|14 (1/1/14)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #7f7f7f; text-align: center;"|16 (1/1/16)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|18 (1/1/18)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|20 (1/1/20)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|22 (1/1/22)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|24 (1/1/24)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|26 (1/1/26)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|28 (1/1/28)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|30 (1/1/30)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|32 (1/1/32)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|34 (1/1/34)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|36 (1/1/36)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|38 (1/1/38)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|40 (1/1/40)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|42 (1/1/42)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|44 (1/1/44)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|46 (1/1/46)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|48 (1/1/48)
|-
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|an-a05n02<br />BCN - Link 1
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #005C00; text-align: center;"|an-a05n02<br />SN - Link 1
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|an-a05n02<br />IFN - Link 1
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|an-a05n02<br />BCN - Link 3
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|an-a05n02<br />IPMI / Fencing
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|an-striker02<br />BCN - Link 1
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|an-a05n02<br />IFN - Link 3
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|an-striker02<br />BCN - Link 1
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|Uplink 1
|-
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; background-color: #FFFFFF; text-align: center;"|VID 100
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; background-color: #99FF66; text-align: center;"|VID 200
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; background-color: #2f2f2f; color: #cfcfcf; text-align: center;" colspan="2"|VID 300
|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; background-color: #FFFFFF; text-align: center;" colspan="8"|VID 100
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; background-color: #2f2f2f; color: #cfcfcf; text-align: center;" colspan="16"|VID 300
|-
|colspan="30"|&nbsp;
|-
!an-switch02
!colspan="29"|&nbsp;
|-
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; background-color: #FFFFFF; text-align: center;"|BCN
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; background-color: #99FF66; text-align: center;"|SN
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; background-color: #2f2f2f; color: #cfcfcf; text-align: center;" colspan="2"|IFN
|class="null"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; background-color: #FFFFFF; text-align: center;" colspan="8"|Back-Channel Network
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; background-color: #2f2f2f; color: #cfcfcf; text-align: center;" colspan="16"|Internet-Facing Network
|-
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #7f7f7f; text-align: center;"|X1 (2/2/1)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #7f8f7f; text-align: center;"|X3 (2/2/3)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|X5 (2/2/5)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|X7 (2/2/7)
|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #7f7f7f; text-align: center;"|1 (2/1/1)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #7f7f7f; text-align: center;"|3 (2/1/3)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #7f7f7f; text-align: center;"|5 (2/1/5)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #7f7f7f; text-align: center;"|7 (2/1/7)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #7f7f7f; text-align: center;"|9 (2/1/9)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #7f7f7f; text-align: center;"|11 (2/1/11)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #7f7f7f; text-align: center;"|13 (2/1/13)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #7f7f7f; text-align: center;"|15 (2/1/15)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|17 (2/1/17)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|19 (2/1/19)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|21 (2/1/21)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|23 (2/1/23)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|25 (2/1/25)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|27 (2/1/27)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|29 (2/1/29)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|31 (2/1/31)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|33 (2/1/33)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|35 (2/1/35)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|37 (2/1/37)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|39 (2/1/39)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|41 (2/1/41)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|43 (2/1/43)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|45 (2/1/45)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|47 (2/1/47)
|-
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|an-a05n01<br />BCN - Link 2
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #005C00; text-align: center;"|an-a05n01<br />SN - Link 2
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|an-a05n01<br />IFN - Link 2
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|an-pdu01
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|an-ups01
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|an-striker02<br />BCN - Link 2
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|an-striker02<br />IPMI
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|an-pdu03
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|an-striker02<br />IFN - Link 2
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|-
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #7f7f7f; text-align: center;"|X2 (2/2/2)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #7f8f7f; text-align: center;"|X4 (2/2/4)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|X6 (2/2/6)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|X8 (2/2/8)
|class="null_header"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #7f7f7f; text-align: center;"|2 (2/1/2)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #7f7f7f; text-align: center;"|4 (2/1/4)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #7f7f7f; text-align: center;"|6 (2/1/6)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #7f7f7f; text-align: center;"|8 (2/1/8)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #7f7f7f; text-align: center;"|10 (2/1/10)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #7f7f7f; text-align: center;"|12 (2/1/12)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #7f7f7f; text-align: center;"|14 (2/1/14)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #7f7f7f; text-align: center;"|16 (2/1/16)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|18 (2/1/18)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|20 (2/1/20)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|22 (2/1/22)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|24 (2/1/24)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|26 (2/1/26)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|28 (2/1/28)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|30 (2/1/30)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|32 (2/1/32)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|34 (2/1/34)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|36 (2/1/36)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|38 (2/1/38)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|40 (2/1/40)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|42 (2/1/42)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|44 (2/1/44)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|46 (2/1/46)
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #5f5f5f; text-align: center;"|48 (2/1/48)
|-
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|an-a05n02<br />BCN - Link 2
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #005C00; text-align: center;"|an-a05n02<br />SN - Link 2
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|an-a05n02<br />IFN - Link 2
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|an-pdu02
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|an-ups02
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|an-striker02<br />BCN - Link 2
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|an-pdu04
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #2f2f2f; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|an-striker02<br />BCN - Link 2
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; white-space: nowrap; color: #000000; text-align: center;"|Uplink 2
|-
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; background-color: #FFFFFF; text-align: center;"|VID 100
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; background-color: #99FF66; text-align: center;"|VID 200
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; background-color: #2f2f2f; color: #cfcfcf; text-align: center;" colspan="2"|VID 300
|&nbsp;
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; background-color: #FFFFFF; text-align: center;" colspan="8"|VID 100
|style="font-family: 'Dejavu Sans Mono', 'Courier 10 pitch', monospace, Courier; font-size: 12px; background-color: #2f2f2f; color: #cfcfcf; text-align: center;" colspan="16"|VID 300
|}
 
== Understanding Brocade Logical Port Numbers ==
 
Brocade uses the follow scheme for naming their ports:
* <span class="code">stack ID</span>/<span class="code">module</span>/<span class="code">port</span>


Added untagged port(s) ethe 1/1/1 to 1/1/10 to port-vlan 100.
The Stack ID is simply the integer representing the <span class="code">ID</span> number shown when you run '<span class="code">show stack</span>'.


### (config-vlan-100) untag ethernet 2/1/1 to 2/1/10
The module number is static across all switches. They are:
* <span class="code">1</span>: Standard RJ-45 1 Gbps copper ports.
* <span class="code">2</span>: Ports used for stacking. Typically these are only used for stacking.
* <span class="code">3</span>: The SFP+ fiber or copper ports that normally run at 1 Gbps but can be licensed to run at 10 Gbps.


Added untagged port(s) ethe 2/1/1 to 2/1/10 to port-vlan 100.
The port number is the simple integer representing the port. This is the number found above or below the physical port on the switch.


### (config-vlan-100) vlan 200 name sn
You see these <span class="code">x/y/z</span> numbers in the table above, and they will be the method used below to identify ports and port ranges.
### (config-vlan-200) untag ethernet 1/1/11 to 1/1/14 ethernet 2/1/11 to 2/1/24


Added untagged port(s) ethe 1/1/11 to 1/1/14 ethe 2/1/11 to 2/1/24 to port-vlan 200.
== Forming the VLANs ==
</syntaxhighlight>


Oops. Have to send some ports back to VID 1.
To start, lets take a look at the current VLAN configuration.


<syntaxhighlight lang="text">
<syntaxhighlight lang="text">
### (config-vlan-200) no untag ethernet 2/1/15 to 2/1/24
ICX6610-48 Switch#configure terminal
### (config-vlan-200) show vlan
ICX6610-48 Switch(config)#show vlan    
</syntaxhighlight>


Total PORT-VLAN entries: 3
<syntaxhighlight lang="text">
Total PORT-VLAN entries: 1
Maximum PORT-VLAN entries: 64
Maximum PORT-VLAN entries: 64


Legend: [Stk=Stack-Id, S=Slot]
Legend: [Stk=Stack-Id, S=Slot]
</syntaxhighlight>
Not much!
<syntaxhighlight lang="text">
ICX6610-48 Switch(config)#vlan 100 name bcn
</syntaxhighlight>


PORT-VLAN 1, Name DEFAULT-VLAN, Priority level0, Spanning tree On
The BCN spans SFP ports 1 and 2 (1/3/1 to 1/3/1, 2/3/1 to 2/3/1) and copper ports 1 through 16 (1/1/1 to 1/1/16, 2/1/1 to 2/1/16) will be assigned to this VLAN. We can do this in one command.
Untagged Ports: (U1/M1) 15  16 17  18  19  20  21  22  23  24
 
Untagged Ports: (U1/M3)  1   2   3  4  5  6  7  8
<syntaxhighlight lang="text">
Untagged Ports: (U2/M1) 15  16  17  18  19  20  21  22  23  24
ICX6610-48 Switch(config-vlan-100)#untag ethernet 1/3/1 to 1/3/2 ethernet 2/3/1 to 2/3/2 ethernet 1/1/1 to 1/1/16 ethernet 2/1/1 to 2/1/16
Untagged Ports: (U2/M3)  1   2   3   4  5  6  7  8
</syntaxhighlight>
  Tagged Ports: None
 
  Uplink Ports: None
<syntaxhighlight lang="text">
DualMode Ports: None
Added untagged port(s) ethe 1/1/1 to 1/1/16 ethe 1/3/1 to 1/3/2 ethe 2/1/1 to 2/1/16 ethe 2/3/1 to 2/3/2 to port-vlan 100.
Mac-Vlan Ports: None
</syntaxhighlight>
    Monitoring: Disabled
 
PORT-VLAN 100, Name bcn, Priority level0, Spanning tree On
The SN spans just four ports, all of which are on the SFP+ ports. They are 1/3/3 to 1/3/4 and 2/3/3 to 2/3/4.
Untagged Ports: (U1/M1)  1   2   3   4  5  6  7  8  9  10
 
Untagged Ports: (U2/M1)  1  2   3   4   5  6  7  8  9  10
<syntaxhighlight lang="text">
  Tagged Ports: None
ICX6610-48 Switch(config-vlan-100)#vlan 200 name sn
  Uplink Ports: None
ICX6610-48 Switch(config-vlan-200)#untag ethernet 1/3/3 to 1/3/4 ethernet 2/3/3 to 2/3/4
DualMode Ports: None
</syntaxhighlight>
Mac-Vlan Ports: None
 
    Monitoring: Disabled
<syntaxhighlight lang="text">
PORT-VLAN 200, Name sn, Priority level0, Spanning tree On
Added untagged port(s) ethe 1/3/3 to 1/3/4 ethe 2/3/3 to 2/3/4 to port-vlan 200.
Untagged Ports: (U1/M1)  11  12  13  14
Untagged Ports: (U2/M1)  11  12  13  14
  Tagged Ports: None
  Uplink Ports: None
DualMode Ports: None
Mac-Vlan Ports: None
    Monitoring: Disabled
</syntaxhighlight>
</syntaxhighlight>


{{note|1=A port in VID 1 will simply change to the newly assigned VID. Ports in other VIDs need to be reset with <span class="code">no untag ...</span> before reconfiguring for the new VLAN.}}
Finally, the biggest VLAN. We're going to allocate the last four SFP+ ports on both switch and copper ports 17 through 48 on both as well.


Back to configuring.
<syntaxhighlight lang="text">
ICX6610-48 Switch(config-vlan-200)#vlan 300 name ifn
ICX6610-48 Switch(config-vlan-300)#untag ethernet 1/3/5 to 1/3/8 ethernet 2/3/5 to 2/3/8 ethernet 1/1/17 to 1/1/48 ethernet 2/1/17 to 2/1/48
</syntaxhighlight>


<syntaxhighlight lang="text">
<syntaxhighlight lang="text">
### (config-vlan-200) vlan 300 name ifn
Added untagged port(s) ethe 1/1/17 to 1/1/48 ethe 1/3/5 to 1/3/8 ethe 2/1/17 to 2/1/48 ethe 2/3/5 to 2/3/8 to port-vlan 300.
### (config-vlan-300) untag ethernet 1/1/15 to 1/1/24 ethernet 2/1/15 to 2/1/24
</syntaxhighlight>


Added untagged port(s) ethe 1/1/15 to 1/1/24 ethe 2/1/15 to 2/1/24 to port-vlan 300.
Done! Let's see what it looks like now:


### show vlan
<syntaxhighlight lang="text">
ICX6610-48 Switch(config-vlan-300)#exit
ICX6610-48 Switch(config)#show vlan
</syntaxhighlight>


<syntaxhighlight lang="text">
Total PORT-VLAN entries: 4
Total PORT-VLAN entries: 4
Maximum PORT-VLAN entries: 64
Maximum PORT-VLAN entries: 64
Line 420: Line 1,101:


PORT-VLAN 1, Name DEFAULT-VLAN, Priority level0, Spanning tree On
PORT-VLAN 1, Name DEFAULT-VLAN, Priority level0, Spanning tree On
  Untagged Ports: (U1/M3)  1  2  3  4  5  6  7  8
  Untagged Ports: None
Untagged Ports: (U2/M3)  1  2  3  4  5  6  7  8
   Tagged Ports: None
   Tagged Ports: None
   Uplink Ports: None
   Uplink Ports: None
Line 428: Line 1,108:
     Monitoring: Disabled
     Monitoring: Disabled
PORT-VLAN 100, Name bcn, Priority level0, Spanning tree On
PORT-VLAN 100, Name bcn, Priority level0, Spanning tree On
  Untagged Ports: (U1/M1)  1  2  3  4  5  6  7  8  9  10  
  Untagged Ports: (U1/M1)  1  2  3  4  5  6  7  8  9  10 11  12
  Untagged Ports: (U2/M1)  1  2  3  4  5  6  7  8  9  10  
Untagged Ports: (U1/M1)  13  14  15  16
Untagged Ports: (U1/M3)  1  2
  Untagged Ports: (U2/M1)  1  2  3  4  5  6  7  8  9  10 11  12
Untagged Ports: (U2/M1)  13  14  15  16
Untagged Ports: (U2/M3)  1  2
   Tagged Ports: None
   Tagged Ports: None
   Uplink Ports: None
   Uplink Ports: None
  DualMode Ports: None
  DualMode Ports: None
  Mac-Vlan Ports: None
  Mac-Vlan Ports: None
     Monitoring: Disabled
     Monitoring: Disabled                                        
PORT-VLAN 200, Name sn, Priority level0, Spanning tree On
PORT-VLAN 200, Name sn, Priority level0, Spanning tree On
  Untagged Ports: (U1/M1) 11  12  13  14
  Untagged Ports: (U1/M3)   3  4
  Untagged Ports: (U2/M1) 11  12  13  14
  Untagged Ports: (U2/M3)   3  4
   Tagged Ports: None
   Tagged Ports: None
   Uplink Ports: None
   Uplink Ports: None
Line 444: Line 1,128:
     Monitoring: Disabled
     Monitoring: Disabled
PORT-VLAN 300, Name ifn, Priority level0, Spanning tree On
PORT-VLAN 300, Name ifn, Priority level0, Spanning tree On
  Untagged Ports: (U1/M1) 15  16 17  18  19  20  21  22  23  24  
  Untagged Ports: (U1/M1)  17  18  19  20  21  22  23  24 25  26  27  28
  Untagged Ports: (U2/M1)  15 16 17  18  19  20  21  22  23  24  
  Untagged Ports: (U1/M1)  29  30  31  32  33  34  35  36  37  38  39  40
Untagged Ports: (U1/M1)  41  42  43  44  45  46  47  48
Untagged Ports: (U1/M3)  5  6  7  8
  Untagged Ports: (U2/M1) 17  18  19  20  21  22  23  24 25  26  27  28
Untagged Ports: (U2/M1)  29  30  31  32  33  34  35  36  37  38  39  40
Untagged Ports: (U2/M1)  41  42  43  44  45  46  47  48
Untagged Ports: (U2/M3)  5  6  7  8
   Tagged Ports: None
   Tagged Ports: None
   Uplink Ports: None
   Uplink Ports: None
Line 451: Line 1,141:
  Mac-Vlan Ports: None
  Mac-Vlan Ports: None
     Monitoring: Disabled
     Monitoring: Disabled
</syntaxhighlight>
Perfectly perfect. Save the changes and then we're done.


### (config-vlan-300) exit
<syntaxhighlight lang="text">
### (config) write memory
ICX6610-48 Switch(config)#write memory
</syntaxhighlight>


<syntaxhighlight lang="text">
Write startup-config done.
Write startup-config done.


Line 460: Line 1,155:
Flash Memory Write (8192 bytes per dot) .
Flash Memory Write (8192 bytes per dot) .
Flash to Flash Done.
Flash to Flash Done.
</syntaxhighlight>
== Changing a Port's VLAN Assignment ==
In order to change a port from one VLAN to another, it must first to untagged as VLAN ID 1, then it can be re-assigned to the new VLAN ID.


### (config) exit
<syntaxhighlight lang="text">
### (config-vlan-200) no untag ethernet 2/1/15 to 2/1/24
### (config-vlan-200) show vlan
</syntaxhighlight>
</syntaxhighlight>


== Pseudo-Stacking on ICX-6430 ==
{{note|1=A port in VID 1 will simply change to the newly assigned VID. Ports in other VIDs need to be reset with <span class="code">no untag ...</span> before reconfiguring for the new VLAN.}}
 
== Pseudo Stacking on ICX6430-C12 ==


To share one uplink between multiple VLANs (to act like a stack);
To share one uplink between multiple VLANs (to act like a stack);
Line 526: Line 1,230:
</syntaxhighlight>
</syntaxhighlight>


= Installing Keys =
= Key Management =


Once you get the license file from Brocade, copy the key(s) to the root of your TFTP server (<span class="code">/var/lib/tftpboot/</span>).
To get a license, you must purchase the license from a Brocade reseller. With it, you will get a spreadsheet containing a <span class="code">TRANSACTION KEY</span>.


Note the LID number and which stack ID. In this case, I renamed the license files so that 'an-s01_01.key' was for the LID for switch stack 1 (the license key's suffix is the LID of the unit, see 'show config').
To convert this into a loadable license file, you will need to get the switch's '<span class="code">LID</span>', then create an account on "[http://my.brocade.com My Brocade]". You can then combine the LID and the TRANSACTION KEY and Brocade will email you the license key.
 
To get the LID;


<syntaxhighlight lang="text">
<syntaxhighlight lang="text">
ICX6610-24 Switch>enable
ICX6610-48 Switch#show version
Password:
</syntaxhighlight>
ICX6610-24 Switch#ping 10.255.255.254
<syntaxhighlight lang="text">
Sending 1, 16-byte ICMP Echo to 10.255.255.254, timeout 5000 msec, TTL 64
  Copyright (c) 1996-2013 Brocade Communications Systems, Inc.
Type Control-c to abort
    UNIT 1: compiled on May 10 2013 at 16:48:26 labeled as FCXS07300f
^C
                (5370497 bytes) from Primary /foundry/FGS/os/FCXS07300f.bin
Ping aborted!
        SW: Version 07.3.00fT7f1
ICX6610-24 Switch#terminal config
    UNIT 2: compiled on May 10 2013 at 16:48:26 labeled as FCXS07300f
Invalid input -> config
                (5370497 bytes) from Primary /foundry/FGS/os/FCXS07300f.bin
Type ? for a list
        SW: Version 07.3.00fT7f1
ICX6610-24 Switch#config terminal
  Boot-Monitor Image size = 370555, Version:07.3.02T7f5 (grz07302)
ICX6610-24 Switch(config)#ip address 10.255.8.1 255.255.0.0
  HW: Stackable ICX6610-48
ICX6610-24 Switch(config)#ping 10.255.255.254
==========================================================================
Invalid input -> ping 10.255.255.254
UNIT 1: SL 1: ICX6610-48 48-port Management Module
Type ? for a list
        Serial  #: xxxxxxxxxxx
ICX6610-24 Switch(config)#exit
        License: BASE_SOFT_PACKAGE  (LID: yyyyyyyyyyy)
ICX6610-24 Switch#ping 10.255.255.254
        P-ENGINE  0: type E02B, rev 01
Sending 1, 16-byte ICMP Echo to 10.255.255.254, timeout 5000 msec, TTL 64
        P-ENGINE  1: type E02B, rev 01
Type Control-c to abort
==========================================================================
Reply from 10.255.255.254  : bytes=16 time<1ms TTL=64
UNIT 1: SL 2: ICX6610-QSFP 10-port 160G Module
Success rate is 100 percent (1/1), round-trip min/avg/max=0/0/0 ms.
==========================================================================
ICX6610-24 Switch#copy tftp license 10.255.255.254 an-s01_01.key
UNIT 1: SL 3: ICX6610-8-port Dual Mode(SFP/SFP+) Module
Incomplete command.
==========================================================================
ICX6610-24 Switch#copy tftp license 10.255.255.254 an-s01_01.key
UNIT 2: SL 1: ICX6610-48 48-port Management Module
Incomplete command.
        Serial  #: aaaaaaaaaaa
ICX6610-24 Switch#copy tftp license 10.255.255.254 an-s01_01.key unit 1
        License: BASE_SOFT_PACKAGE  (LID: bbbbbbbbbbb)
ICX6610-24 Switch#Flash Memory Write (8192 bytes per dot) .
        P-ENGINE  0: type E02B, rev 01                         
        P-ENGINE  1: type E02B, rev 01
==========================================================================
UNIT 2: SL 2: ICX6610-QSFP 10-port 160G Module
==========================================================================
UNIT 2: SL 3: ICX6610-8-port Dual Mode(SFP/SFP+) Module
==========================================================================
  800 MHz Power PC processor 8544E (version 0021/0023) 400 MHz bus
65536 KB flash memory
  512 MB DRAM
STACKID 1 system uptime is 1 days 17 hours 8 minutes 17 seconds
STACKID 2  system uptime is 22 hours 36 minutes 24 seconds
The system : started=warm start  reloaded=by "reload"
My stack unit ID = 1, bootup role = active
</syntaxhighlight>
 
=== Navigating Brocade's Website ===
 
With the <span class="code">TRANSACTION KEY</span> and <span class="code">LID</span>, and having logged into [http://swportal.brocade.com/ Brocade Software Portal];
 
* Hover over the top-left 'License Management' button. A drop-down menu will appear. Hover over 'Brocade IP/ADP'. Another pop-out menu will appear, click on 'License Generation with Transaction Key'.
* A form will appear. Fill it our as instructed. Check to accept the EULA and then click 'Add'.
** '''IF YOU ARE ADDING TWO LICENSES TO ONE SWITCH''':
** Fill out the LID and Transaction key for the next license, again accept the EULA and click 'Add' again.
* Once all licenses for the specified switch are added, click on 'Generate'.
* If all was well, you will be shown a summary and an email with the license will be emailed to you.
 
Repeat as needed for each switch.
 
== Installing Keys ==
 
Once you get the license file from Brocade, copy the key to the root of your [[Setting_Up_a_PXE_Server_on_an_RPM-based_OS#tftp|TFTP]] server (<span class="code">/var/lib/tftpboot/</span> on [[EL6]]).
 
Note the LID number and which stack ID. In this case, I renamed the license files so that <span class="code">an-switch01_XXXXXXXXXXX.xml</span>, where <span class="code">XXXXXXXXXXX</span> is the switch's LID. In the example below, I will load two licenses, one for either switch in the stack.
 
<syntaxhighlight lang="text">
ICX6610-24 Switch#copy tftp license 10.255.255.254 an-switch01_XXXXXXXXXXX.xml unit 1
</syntaxhighlight>
 
<syntaxhighlight lang="text">
Flash Memory Write (8192 bytes per dot) .
Copy Software License from TFTP to Flash Done.
Copy Software License from TFTP to Flash Done.
T=1d20h6m27.7: Election, was active, no change, ID=1, pri=128, 2U(1-2), A=u1, nbr#=1 1,
</syntaxhighlight>


ICX6610-24 Switch#show license                          1_01.key unit 2
<syntaxhighlight lang="text">
Index      License Name              Lid          License Type    Status    License Period  License Capacity 
ICX6610-24 Switch#copy tftp license 10.255.255.254 an-switch02_XXXXXXXXXXX.xml unit 2
Stack unit 1:
</syntaxhighlight>
1          ICX6610-10G-LIC-POD      dzrHKJMlFKd  Normal          Invalid    Unlimited                  4
 
ICX6610-24 Switch#license delete unit 1 index 1
<syntaxhighlight lang="text">
ICX6610-24 Switch#show license
Flash Memory Write (8192 bytes per dot) .
License record empty
ICX6610-24 Switch#copy tftp license 10.255.255.254 an-s01_01.key unit 2
ICX6610-24 Switch#Flash Memory Write (8192 bytes per dot) .
Copy Software License from TFTP to Flash Done.
Copy Software License from TFTP to Flash Done.


Copy software license to stack unit 2 success
Copy software license to stack unit 2 success
Flash Memory Write (8192 bytes per dot) .
Flash Memory Write (8192 bytes per dot) .
T=20h33m21.2: Election, was active, no change, ID=1, pri=128, 2U(1-2), A=u1, nbr#=1 1,  
T=1d20h7m50.1: Election, was active, no change, ID=1, pri=128, 2U(1-2), A=u1, nbr#=1 1,  
</syntaxhighlight>


<syntaxhighlight lang="text">
ICX6610-24 Switch#show license
ICX6610-24 Switch#show license
</syntaxhighlight>
<syntaxhighlight lang="text">
Index      License Name              Lid          License Type    Status    License Period  License Capacity   
Index      License Name              Lid          License Type    Status    License Period  License Capacity   
Stack unit 1:
1          ICX6610-10G-LIC-POD      xxxxxxxxxxx  Normal          Active    Unlimited                  8
Stack unit 2:
Stack unit 2:
1          ICX6610-10G-LIC-POD      dzrHKJMlFKd Normal          Active    Unlimited                  4
1          ICX6610-10G-LIC-POD      aaaaaaaaaaa Normal          Active    Unlimited                  8
ICX6610-24 Switch#copy tftp license 10.255.255.254 an-s0201.key unit 1 
</syntaxhighlight>
ICX6610-24 Switch#TFTP: received error request -- code 1 message File not found
 
<syntaxhighlight lang="text">
write memory
</syntaxhighlight>
 
Repeat as needed.
 
== Enable 10 Gbps Ports ==
 
After installing license files to upgrade the eight 1 Gbps SFP+ ports to run as 10 Gbps, we need to actually enable the ports. This may seem silly, but consider that the upgrades are sold in lots of 4-ports at a time. This step would allow you to specify which four ports were upgraded. In our case, we upgraded them all.
 
<syntaxhighlight lang="text">
ICX6610-48 Switch#config terminal
</syntaxhighlight>


Copy Software License from TFTP to Flash copy stack Error: tftp file not found.
<syntaxhighlight lang="text">
ICX6610-48 Switch(config)#interface ethernet 1/3/1 to 1/3/8
</syntaxhighlight>


ICX6610-24 Switch#copy tftp license 10.255.255.254 an-s02_01.key unit 1
<syntaxhighlight lang="text">
ICX6610-24 Switch#Flash Memory Write (8192 bytes per dot) .
ICX6610-48 Switch(config-mif-1/3/1-1/3/8)#speed-duplex 10g-full
Copy Software License from TFTP to Flash Done.
</syntaxhighlight>
T=20h34m13.1: Election, was active, no change, ID=1, pri=128, 2U(1-2), A=u1, nbr#=1 1,
 
This will take a minute, please wait.
 
<syntaxhighlight lang="text">
ICX6610-48 Switch(config-mif-1/3/1-1/3/8)#interface ethernet 2/3/1 to 2/3/8
</syntaxhighlight>
 
<syntaxhighlight lang="text">
ICX6610-48 Switch(config-mif-2/3/1-2/3/8)#speed-duplex 10g-full           
</syntaxhighlight>
 
<syntaxhighlight lang="text">
ICX6610-48 Switch(config-mif-2/3/1-2/3/8)#exit
</syntaxhighlight>
 
 
If you (accidentally) use a 1 Gbps SFP+ cable, the upgrade will likely throw an error like the one below:
 
<syntaxhighlight lang="text">
U2-MSG: INFO: 2/3/4: optics <-> speed mismatch. Replace with SFP+ to enable link.
U2-MSG: INFO: 2/3/4: optics <-> speed mismatch. Replace with SFP+ to enable link.
</syntaxhighlight>
 
You can reduce the port speed to make it work (albeit at 1 Gbps):
 
<syntaxhighlight lang="text">
ICX6610-48 Switch(config-mif-1/3/1-1/3/8)#interface ethernet 2/3/4
</syntaxhighlight>
 
<syntaxhighlight lang="text">
ICX6610-48 Switch(config-if-e10000-2/3/4)#speed-duplex 1000-full-master
</syntaxhighlight>
 
When done, save your changes.
 
<syntaxhighlight lang="text">
ICX6610-48 Switch(config)#write memory
</syntaxhighlight>


ICX6610-24 Switch#show license
<syntaxhighlight lang="text">
Index      License Name              Lid          License Type    Status    License Period  License Capacity 
Stack unit 1:
1          ICX6610-10G-LIC-POD      dzrHKJMlFKI  Normal          Active    Unlimited                  4
Stack unit 2:
1          ICX6610-10G-LIC-POD      dzrHKJMlFKd  Normal          Active    Unlimited                  4
ICX6610-24 Switch#write memory
Write startup-config done.
Write startup-config done.


Automatic copy to member units:  2
Automatic copy to member units:  2
ICX6610-24 Switch#Flash Memory Write (8192 bytes per dot) .
ICX6610-48 Switch(config)#Flash Memory Write (8192 bytes per dot) .
Flash to Flash Done.
Flash to Flash Done.
</syntaxhighlight>
</syntaxhighlight>


Enable ports
== Deleting Keys ==


If you need to remove a key (ie: replacing a demo license), you can do so thusly:
<syntaxhighlight lang="text">
ICX6610-24 Switch#license delete unit 1 index 1
ICX6610-24 Switch#show license
License record empty
</syntaxhighlight>
= Replacing a Switch =
{{warning|1=The switch that is the stack master will overwrite the configuration of the stack slave. So be sure to flash the firmware to match the existing switch, license the ports if needed and enable stacking on the replacement switch while it is stand-alone. Power it off, plug in the stack cables and '''then''' apply power. When it boots, it's config should be replaced with the good switch's configuration. To be safe, in an [[Anvil!]], withdraw a node before reforming the stack.}}
{{warning|1=You must contact Brocade to move the licenses!}}
If a switch needs to be replaced, we may need to "de-license" the old switch.
Call Brocade support (1-800-752-8061) and use the case number the switch was replaced under. Confirm with them the [[LID]] of the switch being replaced (the one that the license will be removed from) and the LID of the new/replacement switch.
With the switch being returned, there is no need to prove the old license is removed as you will be returning the switch. '''If''' you will NOT be returning the switch, then you will need to provide a <span class="code">show license</span> before and after deleting the old license in order for the license to be reissued. This is a rare case and the Brocade technician will walk you through the process should it be required.
They will email you the replacement license, at which point you will license the replacement switch as if it were a new switch.
{{note|1=Depending your email client, you may need to request the [[XML]] license file be attached separately.}}
== Checking the Firmware ==
Before you proceed, make sure the Firmware is the same version as the stack master. If not, [[Update the firmware|upgrade (or downgrade)]] the RMAed switch to match.
== Licensing the Switch ==
Once you have the XML file, [[Configuring_Brocade_Switches#Installing_Keys|proceed from here]].
== Adding the Switch to the Stack ==
For this step, we want to enable stacking on the new switch and nothing more.
<syntaxhighlight lang="text">
stack enable
</syntaxhighlight>
<syntaxhighlight lang="text">
Enable stacking. This unit actively participates in stacking
stacking is enable. optical monitoring for stacking ports 1/2/1, 1/2/6 is not available.
</syntaxhighlight>
<syntaxhighlight lang="text">
write memory
</syntaxhighlight>
<syntaxhighlight lang="text">
<syntaxhighlight lang="text">
ICX6610-24 Switch#config terminal
Write startup-config done.
ICX6610-24 Switch(config)#interface ethernet 1/3/1 to 1/3/4
 
ICX6610-24 Switch(config-mif-1/3/1-1/3/4)#speed-du
ICX6610-48 Switch(config)#Flash Memory Write (8192 bytes per dot) .
  speed-duplex              Set to 100 or 10 (or 10G on ICX6450), half or full
Flash to Flash Done.
ICX6610-24 Switch(config-mif-1/3/1-1/3/4)#speed-duplex
  10-full           
  10-half           
  100-full         
  100-half         
  1000-full-master 
  1000-full-slave   
  10g-full         
  auto             
ICX6610-24 Switch(config-mif-1/3/1-1/3/4)#speed-duplex 10g-full
ICX6610-24 Switch(config-mif-1/3/1-1/3/4)#inter
ICX6610-24 Switch(config-mif-1/3/1-1/3/4)#interfa
ICX6610-24 Switch(config-mif-1/3/1-1/3/4)#interface ethernet 2/3/1 to 2/3/4
ICX6610-24 Switch(config-mif-2/3/1-2/3/4)#speed-duplex 10g-full
ICX6610-24 Switch(config-mif-2/3/1-2/3/4)#U2-MSG: INFO: 2/3/4: optics <-> speed mismatch. Replace with SFP+ to enable link.
U2-MSG: INFO: 2/3/4: optics <-> speed mismatch. Replace with SFP+ to enable link.
interface ethernet 2/3/4
ICX6610-24 Switch(config-if-e10000-2/3/4)#speed-duplex 1000-full-master
</syntaxhighlight>
</syntaxhighlight>


= Configure Failover and Recovery =
{{note|1=In the example below, switch 1 is being replaced.}}


Configure the stack to handle switch "hitless" failure and recovery.
At this point, switch to the production switch (which ever isn't being replaced). Connect the newly flashed and licensed replacement switch and you should see the configuration automatically upload to the newly replaced switch.
 
<syntaxhighlight lang="text">
ICX6610-48 Switch#T=4d4h15m31.2: Election, was active, no change, ID=2, pri=128, 2U(1-2), A=u2, nbr#=1 0,
T=4d4h15m32.2: Election, was active, no change, ID=2, pri=128, 2U(1-2), A=u2, nbr#=1 0,
Detect stack unit 1 has different startup config flash, will synchronize it
T=4d4h15m34.0: Election, was active, no change, ID=2, pri=128, 2U(1-2), A=u2, nbr#=1 1,
Detect stack unit 1 has different startup config flash, will synchronize it
reset unit 1: jumbo mismatch. Act u2: 1, Mem u1: 0
reset unit 1: jumbo mismatch. Act u2: 1, Mem u1: 0
Unit 2 loses all neighbors.
Active unit 2 deletes u1 but keeps its static config.
</syntaxhighlight>
 
Some time passes:


<syntaxhighlight lang="text">
<syntaxhighlight lang="text">
ICX6610-24 Switch#config terminal
T=4d4h18m52.3: Election, was active, no change, ID=2, pri=128, 2U(1-2), A=u2, nbr#=1 1,
ICX6610-24 Switch(config)#hitless-failover enable
Detect stack unit 1 has different startup config flash, will synchronize it
Done hot swap: active controller unit 2 sets unit 1 to Ready.
Synchronize startup config to stack unit 1
Flash Memory Write (8192 bytes per dot) .
Assigned unit 1 to be standby
start running config sync
Running config sync to standby is complete
</syntaxhighlight>
 
At this point, recovery is complete.
 
== Unconfigure Stack ==


# Didn't help on it's own.
If you boot a stacked switch on it's own, you will not have normal access to the switch configuration.
ICX6610-24 Switch(config)#stack mac 0000.0000.0123


# Set matching priorities
<syntaxhighlight lang="text">
config terminal
MEMBER]local-1@ICX6610-48 Switch>enable
stack unit 2
No password has been assigned yet...
priority 128
[MEMBER]local-1@ICX6610-48 Switch#
<wait 2 minutes>
</syntaxhighlight>
</syntaxhighlight>


Done
To clear the stack config and restore the switch to standard mode, run:
 
<syntaxhighlight lang="text">
stack unconfigure clean
</syntaxhighlight>
<syntaxhighlight lang="text">
This unit will delete all config files and boot up as a clean unit. Are you sure? (enter 'y' or 'n'): y
</syntaxhighlight>
<syntaxhighlight lang="text">
Remove startup config and stacking files. Will reload as a clean unit
[MEMBER]local-1@ICX6610-48 Switch#HRebooting(1)...
</syntaxhighlight>


= Multicast Config =
= Multicast Config =
Line 739: Line 1,600:
* 2; Fibre stack
* 2; Fibre stack
* 3; 10 Gbps fibre
* 3; 10 Gbps fibre
View the port status:
<syntaxhighlight lang="text">
ICX6610-48 Switch(config)#show interfaces brief
Port    Link    State  Dupl Speed Trunk Tag Pvid Pri MAC            Name     
1/1/1  Down    None    None None  None  No  1    0  cc4e.24b9.4b74         
1/1/2  Down    None    None None  None  No  1    0  cc4e.24b9.4b75         
1/1/3  Down    None    None None  None  No  1    0  cc4e.24b9.4b76         
1/1/4  Down    None    None None  None  No  1    0  cc4e.24b9.4b77         
1/1/5  Down    None    None None  None  No  1    0  cc4e.24b9.4b78         
1/1/6  Down    None    None None  None  No  1    0  cc4e.24b9.4b79         
1/1/7  Down    None    None None  None  No  1    0  cc4e.24b9.4b7a         
1/1/8  Down    None    None None  None  No  1    0  cc4e.24b9.4b7b         
1/1/9  Down    None    None None  None  No  1    0  cc4e.24b9.4b7c         
1/1/10  Down    None    None None  None  No  1    0  cc4e.24b9.4b7d         
1/1/11  Down    None    None None  None  No  1    0  cc4e.24b9.4b7e         
1/1/12  Down    None    None None  None  No  1    0  cc4e.24b9.4b7f         
1/1/13  Down    None    None None  None  No  1    0  cc4e.24b9.4b80         
1/1/14  Down    None    None None  None  No  1    0  cc4e.24b9.4b81         
1/1/15  Down    None    None None  None  No  1    0  cc4e.24b9.4b82         
1/1/16  Down    None    None None  None  No  1    0  cc4e.24b9.4b83         
1/1/17  Down    None    None None  None  No  1    0  cc4e.24b9.4b84         
1/1/18  Down    None    None None  None  No  1    0  cc4e.24b9.4b85         
1/1/19  Down    None    None None  None  No  1    0  cc4e.24b9.4b86         
1/1/20  Down    None    None None  None  No  1    0  cc4e.24b9.4b87         
1/1/21  Down    None    None None  None  No  1    0  cc4e.24b9.4b88         
1/1/22  Down    None    None None  None  No  1    0  cc4e.24b9.4b89         
1/1/23  Down    None    None None  None  No  1    0  cc4e.24b9.4b8a         
1/1/24  Down    None    None None  None  No  1    0  cc4e.24b9.4b8b         
1/1/25  Down    None    None None  None  No  1    0  cc4e.24b9.4b8c         
1/1/26  Down    None    None None  None  No  1    0  cc4e.24b9.4b8d         
1/1/27  Down    None    None None  None  No  1    0  cc4e.24b9.4b8e         
1/1/28  Down    None    None None  None  No  1    0  cc4e.24b9.4b8f         
1/1/29  Down    None    None None  None  No  1    0  cc4e.24b9.4b90         
1/1/30  Up      Forward Full 1G    None  No  1    0  cc4e.24b9.4b91         
1/1/31  Up      Forward Full 1G    None  No  1    0  cc4e.24b9.4b92         
1/1/32  Down    None    None None  None  No  1    0  cc4e.24b9.4b93         
1/1/33  Up      Forward Full 1G    None  No  1    0  cc4e.24b9.4b94         
1/1/34  Up      Forward Full 1G    None  No  1    0  cc4e.24b9.4b95         
1/1/35  Up      Forward Full 1G    None  No  1    0  cc4e.24b9.4b96         
1/1/36  Up      Forward Full 1G    None  No  1    0  cc4e.24b9.4b97         
1/1/37  Up      Forward Full 1G    None  No  1    0  cc4e.24b9.4b98         
1/1/38  Up      Forward Full 1G    None  No  1    0  cc4e.24b9.4b99         
1/1/39  Down    None    None None  None  No  1    0  cc4e.24b9.4b9a         
1/1/40  Down    None    None None  None  No  1    0  cc4e.24b9.4b9b         
1/1/41  Up      Forward Full 1G    None  No  1    0  cc4e.24b9.4b9c         
1/1/42  Up      Forward Full 1G    None  No  1    0  cc4e.24b9.4b9d         
1/1/43  Up      Forward Full 1G    None  No  1    0  cc4e.24b9.4b9e         
1/1/44  Down    None    None None  None  No  1    0  cc4e.24b9.4b9f         
1/1/45  Down    None    None None  None  No  1    0  cc4e.24b9.4ba0         
1/1/46  Up      Forward Full 1G    None  No  1    0  cc4e.24b9.4ba1         
1/1/47  Down    None    None None  None  No  1    0  cc4e.24b9.4ba2         
1/1/48  Up      Forward Full 1G    None  No  1    0  cc4e.24b9.4ba3         
1/2/1  Up      Forward Full 40G  None  No  N/A  0  cc4e.24b9.4ba5         
1/2/2  Up      Forward Full 10G  None  No  N/A  0  cc4e.24b9.4ba6         
1/2/6  Up      Forward Full 40G  None  No  N/A  0  cc4e.24b9.4ba7         
1/2/7  Up      Forward Full 10G  None  No  N/A  0  cc4e.24b9.4ba8         
1/3/1  Down    None    None None  None  No  1    0  cc4e.24b9.4ba9         
1/3/2  Up      Forward Full 10G  None  No  1    0  cc4e.24b9.4baa         
1/3/3  Down    None    None None  None  No  1    0  cc4e.24b9.4bab         
1/3/4  Down    None    None None  None  No  1    0  cc4e.24b9.4bac         
1/3/5  Down    None    None None  None  No  1    0  cc4e.24b9.4bad         
1/3/6  Up      Forward Full 10G  None  No  1    0  cc4e.24b9.4bae         
1/3/7  Down    None    None None  None  No  1    0  cc4e.24b9.4baf         
1/3/8  Down    None    None None  None  No  1    0  cc4e.24b9.4bb0         
mgmt1  Down    None    None None  None  No  None 0  cc4e.24b9.4b74         
</syntaxhighlight>


{{footer}}
{{footer}}

Latest revision as of 23:59, 14 November 2015

 AN!Wiki :: How To :: Configuring Brocade Switches

Validated Switches

The following switches (and firmwares) have been validates to work with the Anvil!:

  • ICX6610
    • 7.40d
  • ICX6450
    • 7.40a
    • 7.40e
    • 8.00a

Serial Connection

Note: Depending on your terminal emulator, the <Backspace> key may not work. If it doesn't, try pressing ctrl + h to delete the character to the left of your cursor.

You can use screen to connect to the switch (default values are fine). The example below is what you would use on most servers with an on-board serial port. If you have two, then port two would be /dev/ttyS1. If you are using a USB to serial adapter, then you will use /dev/ttyUSB0 (or if it is a multi-port adapter, /dev/ttyUSB1, etc). If none of these work, you will need to consult your operating system documentation and/or serial port adapter to determine the proper /dev/ttyX device to use.

screen /dev/ttySO

The screen will be blank until you press <enter>.

ICX6610-48 Switch>

To log in;

ICX6610-48 Switch>enable 
No password has been assigned yet...

There is no default user name and password.

Exiting screen

To end your screen session, press ctrl + a together, release and then press k.

You will be asked if you want to kill the session, press y to confirm.

Really kill this window [y/n]
[screen is terminating]

Configure a Password

To use the web interface later, a user and password needs to be set.

Note: Super User level password can be an alphanumeric string, but cannot begin with a number.
ICX6610-48 Switch#configure terminal
ICX6610-48 Switch(config)#enable super-user-password secret
ICX6610-48 Switch(config)#enable user disable-on-login-failure 10
ICX6610-48 Switch(config)#user alteeve privilege 0 password secret
ICX6610-48 Switch(config)#show users
Username                                        Password                           Encrypt   Priv Status   Expire Time
======================================================================================================================
alteeve                                         $1$HF4..Zv.$uF.PUT.bqkMy5GicUIrhG/ enabled   0    enabled  Never

To enable the web interface;

ICX6610-48 Switch(config)#aaa authentication web-server default local
ICX6610-48 Switch(config)#write memory
Write startup-config done.

Automatic copy to member units:  2
Flash Memory Write (8192 bytes per dot) .
Flash to Flash Done.

Form the Stack

ICX6450 and ICX6610

ICX6610-48 Switch>enable
ICX6610-48 Switch#configure terminal
ICX6610-48 Switch(config)#stack enable
Enable stacking. This unit actively participates in stacking
stacking is enable. optical monitoring for stacking ports 1/2/1, 1/2/6 is not available.
ICX6610-48 Switch(config)#exit
ICX6610-48 Switch#stack secure-setup
Discovering the stack topology...

Current Discovered Topology - RING

Available UPSTREAM units
Hop(s)  Id      Type          Mac Address
1       new  ICX6610-48     xxxx.xxxx.xxxx

Available DOWNSTREAM units
Hop(s)  Id      Type          Mac Address
1       new  ICX6610-48     xxxx.xxxx.xxxx
Do you accept the topology (RING) (y/n)?: y
Selected Topology:
Active  Id      Type          Mac Address
        1    ICX6610-48     yyyy.yyyy.yyyy

Selected UPSTREAM units
Hop(s)  Id      Type        Mac Address
1       2    ICX6610-48     xxxx.xxxx.xxxx

Selected DOWNSTREAM units
Hop(s)  Id      Type        Mac Address
1       2    ICX6610-48     xxxx.xxxx.xxxx
Do you accept the unit id's (y/n)?: y
T=1d20h24m2.8: Election, was alone --> active, ID=1, pri=128, 2U(1-2), A=u1, nbr#=1 1, 
T=1d20h24m3.5: Election, was active, no change, ID=1, pri=128, 2U(1-2), A=u1, nbr#=1 1, 
reset unit 2: u2 diff bootup id=1
Unit 1 loses all neighbors.
Active unit 1 deletes u2 and its config because it is learned.

Config changed due to add/del units. Do write mem if you want to keep it

(switch reboots)

T=1d20h26m43.2: Election, was active, no change, ID=1, pri=128, 2U(1-2), A=u1, nbr#=1 1, 
Done hot swap: active controller unit 1 sets unit 2 to Ready.

Config changed due to add/del units. Do write mem if you want to keep it
ICX6610-48 Switch#write memory
Write startup-config done.

Automatic copy to member units:  2
Flash Memory Write (8192 bytes per dot) .
Flash to Flash Done.

A moment later, more boot messages.

Stack unit 2 Power supply 1 is up
Stack unit 2 Power supply 2 is up
Assigned unit 2 to be standby
start running config sync
Running config sync to standby is complete
ICX6610-48 Switch#show stack
alone: standalone, D: dynamic config, S: static config
ID   Type         Role    Mac Address    Pri State   Comment                   
1  S ICX6610-48   active  xxxx.xxxx.xxxx 128 local   Ready
2  S ICX6610-48   standby yyyy.yyyy.yyyy   0 remote  Ready

    active       standby                                                       
     +---+        +---+                                                        
 =2/1| 1 |2/6==2/1| 2 |2/6=                                                    
 |   +---+        +---+   |                                                    
 |                        |                                                    
 |------------------------|                                                    
                                                                              
Standby u2 - No hitless failover. Reason: hitless-failover not configured
Current stack management MAC is xxxx.xxxx.xxxx
Note: no "stack mac" config. My MAC will change after failover.

Configure the stack to handle switch "hitless" failure and recovery.

ICX6610-48 Switch#config terminal
ICX6610-48 Switch(config)#hitless-failover enable

Assign a static MAC to the stack. In this case, I always copy stack unit 1's real MAC address.

ICX6610-48 Switch(config)#stack mac cc4e.24b9.5624

Last, switch to unit 2 and change it's priority to match the priority of unit 1.

ICX6610-48 Switch(config)#stack unit 2
ICX6610-48 Switch(config-unit-2)#priority 128
Priority change needs 120 seconds to take effect.
ICX6610-48 Switch(config-unit-2)#exit
ICX6610-48 Switch(config)#write memory
Write startup-config done.

Automatic copy to member units:  2
Flash Memory Write (8192 bytes per dot) .
Flash to Flash Done.

Two minutes after you change the stack priority, the switch run an election.

Will do election in 30 sec due to priority change
Will do election in 10 sec due to priority change
T=1d20h35m20.2: Election, was active, no change, ID=1, pri=128, 2U(1-2), A=u1, nbr#=1 1,

Verify the stack is configured the way we want.

ICX6610-48 Switch(config)#show stack
alone: standalone, D: dynamic config, S: static config
ID   Type         Role    Mac Address    Pri State   Comment                   
1  S ICX6610-48   active  xxxx.xxxx.xxxx 128 local   Ready
2  S ICX6610-48   standby yyyy.yyyy.yyyy 128 remote  Ready

    active       standby                                                       
     +---+        +---+                                                        
 =2/1| 1 |2/6==2/1| 2 |2/6=                                                    
 |   +---+        +---+   |                                                    
 |                        |                                                    
 |------------------------|                                                    
                                                                              
Standby u2 - protocols ready, can failover or manually switch over
Current stack management MAC is xxxx.xxxx.xxxx

Excellent, done.

ICX6430-C12

The ICX6430-C12 doesn't support stacking. However, a stack can be somewhat simulated by tagging a dedicated port with all our VLANs. This will allow traffic from any VLAN to use the same "uplink" port. Also, without a proper stack, we can not create a trunk/LAG. So we enable rapid STP on each switch's customer-facing uplink port instead.

Enable Jumbo Frames

The ICX6610 support packets up to 10240 bytes. That is to say, if jumbo frames are enabled, the MTU of the switch is 10 KiB.

ICX6610-48 Switch#configure terminal
ICX6610-48 Switch(config)#jumbo
Jumbo mode setting requires a reload to take effect!
ICX6610-48 Switch(config)#write memory
Write startup-config done.

Automatic copy to member units:  2
Flash Memory Write (8192 bytes per dot) .
Flash to Flash Done.
ICX6610-48 Switch(config)#exit
ICX6610-48 Switch#reload
Are you sure? (enter 'y' or 'n'): y
Reload request sent to attached stack member(s)...

Rebooting(0)...

Assign IP to Switch

ICX6610-48 Switch#configure terminal
ICX6610-48 Switch(config)#ip address 10.20.1.5 255.255.0.0
ICX6610-48 Switch(config)#exit

The router is at 10.20.255.254, so ping that to verify the IP is configured.

ICX6610-48 Switch#ping 10.20.255.254
Sending 1, 16-byte ICMP Echo to 10.20.255.254, timeout 5000 msec, TTL 64
Type Control-c to abort
Reply from 10.20.255.254   : bytes=16 time<1ms TTL=64
Success rate is 100 percent (1/1), round-trip min/avg/max=0/0/0 ms.

Good, save the changes.

ICX6610-48 Switch#write memory
Write startup-config done.

Automatic copy to member units:  2
Flash Memory Write (8192 bytes per dot) .
Flash to Flash Done.

Update the firmware

Warning: Be sure to have your switches on UPSes before flashing them! As with all devices, a power loss during the flash process may cause the switch to become unusable.

Selecting the Firmware Version

Before we start, we need to decide what version of the firmware we want to install. To do this, start by logging in to your switch(es) and check their current firmware version.

ICX6610-48 Switch#show flash
Stack unit 1:
  Compressed Pri Code size = 5370497, Version:07.3.00fT7f1 (/foundry/FGS/os/FCXS07300f.bin)
  Compressed Sec Code size = 6905002, Version:07.3.00fT7f3 (/foundry/FGS/os/FCXR07300f.bin)
  Compressed Boot-Monitor Image size = 370555, Version:07.3.02T7f5
  Code Flash Free Space = 52822016
Stack unit 2:
  Compressed Pri Code size = 5370497, Version 07.3.00fT7f1 (/foundry/FGS/os/FCXS07300f.bin)
  Compressed Sec Code size = 6905002, Version 07.3.00fT7f3 (/foundry/FGS/os/FCXR07300f.bin)
  Compressed Boot-Monitor Image size = 370555, Version 07.3.02T7f5
  Code Flash Free Space = 52822016

Note that both switches are running the major version 07.3.00f. Being that we're interested in High Availability, we are going to be conservative and upgrade to the latest subversion available. Both switches are 7.3, so that is what we will use. If one had been 7.4, for example, then we would have upgraded both to the latest 7.4 version.

Layer 2 vs Layer 3

These switches have two firmwares;

  • Layer 2 (S); This is seen in the above output in the current version's flash file "/foundry/FGS/os/FCXS07300f.bin".
  • Layer 3 (R); This is seen in the above output in the current version's flash file "/foundry/FGS/os/FCXR07300f.bin".

We're only going to update the layer 2 "S" firmware. We don't use the layer 3 features for HA so we won't update them. If you want to updated them, that is fine. The upgrade process below is easily ported to the layer 3 firmware update process.

Downloading the Firmware

To get the firmware:

  • Create or log into your My Brocade account.
  • Click on 'Downloads'.
  • In the Download by select box, select Ethernet Switches.
  • Scroll down and click to expand the arrow to the left of "ICX6610" (or your switch model).
  • Click to expand the version of firmware you want to use. For this tutorial, we're going to be conservative and stay in the same "7.3" release, so we will expand 07300.
  • Click to expand the most recent subversion for your selected firmware major number. In my case, that is "07300j".
  • This takes you to a note about export restrictions. Ensure you meet the criteria and then click to acknowledge compliance, the click on "Submit".
  • You will see the EULA. Read it, understand it and then agree to it by clicking on "I Accept".
  • Save the file on your computer.

Extracting the Image

Navigate to the downloaded file. It will be a .zip file. Expand it. This will create several directories.

To upload the firmware, we need to copy the image to the root of our TFTP server (/var/lib/tftpboot/ on EL6).

Browse into the extracted 07300j/ICX/Images/ directory (or the matching directory for your firmware). You will see two file:

cd 07300j/ICX/Images/
ls -lah
total 12M
drwxr-xr-x 2 root root 4.0K May  8  2014 .
drwxr-xr-x 8 root root 4.0K May  9  2014 ..
-rwxr-xr-x 1 root root 6.7M May  8  2014 FCXR07300j.bin
-rwxr-xr-x 1 root root 5.2M May  8  2014 FCXS07300j.bin

As discussed above, we will be upgrading the layer 2 "S" image. So this is the file we will copy to the TFTP root directory.

cp FCXS07300j.bin /var/lib/tftpboot/
ls -lah /var/lib/tftpboot/FCXS07300j.bin

Performing the Flash

Note: We will be flashing both switches at once as they are stacked.

We're ready to flash! So it's time to log into the switch.

ICX6610-48 Switch#copy tftp flash 10.20.255.254 FCXS07300j.bin primary
Flash Memory Write (8192 bytes per dot) 
Automatic copy to member units:  2
....<lots of dots>....
TFTP to Flash Done.
ICX6610-48 Switch#show flash
Stack unit 1:
  Compressed Pri Code size = 5430491, Version:07.3.00jT7f1 (FCXS07300j.bin)
  Compressed Sec Code size = 6905002, Version:07.3.00fT7f3 (/foundry/FGS/os/FCXR07300f.bin)
  Compressed Boot-Monitor Image size = 370555, Version:07.3.02T7f5
  Code Flash Free Space = 52690944
Stack unit 2:
  Compressed Pri Code size = 5430491, Version 07.3.00jT7f1 (FCXS07300j.bin)
  Compressed Sec Code size = 6905002, Version 07.3.00fT7f3 (/foundry/FGS/os/FCXR07300f.bin)
  Compressed Boot-Monitor Image size = 370555, Version 07.3.02T7f5
  Code Flash Free Space = 52690944

Reboot to load the new firmware.

ICX6610-48 Switch#write memory
ICX6610-48 Switch#reload
Are you sure? (enter 'y' or 'n'): y
Note: It takes a while to reboot because it will update the firmware of the second node. Please be patient.
ICX6610-48 Switch#show flash
Stack unit 1:
  Compressed Pri Code size = 5430491, Version:07.3.00jT7f1 (FCXS07300j.bin)
  Compressed Sec Code size = 6905002, Version:07.3.00fT7f3 (/foundry/FGS/os/FCXR07300f.bin)
  Compressed Boot-Monitor Image size = 370555, Version:07.3.02T7f5
  Code Flash Free Space = 52690944
Stack unit 2:
  Compressed Pri Code size = 5430491, Version 07.3.00jT7f1 (FCXS07300j.bin)
  Compressed Sec Code size = 6905002, Version 07.3.00fT7f3 (/foundry/FGS/os/FCXR07300f.bin)
  Compressed Boot-Monitor Image size = 370555, Version 07.3.02T7f5
  Code Flash Free Space = 52690944

Done!

Configure VLAN

VLANs provide a way of dividing up a switch (or stack of switches) into multiple, isolated switches (this is actually somewhat simplistic, but it gets the idea across). Traffic from one VLAN segment can not reach ports allocated to another VLAN, providing a useful layer of security.

Planning

We're creating three separate VLANs;

By default, all ports default to VID 1. If you want to improve security, you can leave all unused ports on VID 1. In this tutorial, we need to allocate all ports for use by other equipment.

Before you start, you need to decide how you want to assign physical ports on your switches to VLAN IDs. I like to do this by creating a table showing all the ports, label their external port numbers and logical numbers and then write in what will be plugged into each port. With this, it's fairly easy to mark what ports will be assigned to what VLANs.

Cabling a pair of ICX6450-24

In this example, we're showing how to cable up a pair of ICX6450-24 switches in a very typical configuration. It can drive two Anvil! systems along with the usual foundation pack.

The table below represents a pair of these stacked together.
an-switch01  
Stack   Back-Channel Network]] Storage Network Internet-Facing Network
X1 (1/2/1) X3 (1/2/3)   1 (1/1/1) 3 (1/1/3) 5 (1/1/5) 7 (1/1/7) 9 (1/1/9) 11 (1/1/11)   13 (1/1/13) 15 (1/1/15) 17 (1/1/17) 19 (1/1/19) 21 (1/1/21) 23 (1/1/23)
To an-switch02
X3
To an-switch02
X1
  an-a01n01
BCN - Link 1
an-a01n01
IPMI / Fencing
an-a02n01
BCN - Link 1
an-a02n01
IPMI / Fencing
an-striker01
BCN - Link 2
an-a01n01
SN - Link 1
  an-a02n01
SN - Link 1
an-a01n01
IFN - Link 1
an-a02n01
IFN - Link 1
an-striker01
IFN - Link 1
-- --
-- --   an-a01n02
BCN - Link 1
an-a01n02
IPMI / Fencing
an-a02n02
BCN - Link 1
an-a02n02
BCN - Link 1
-- an-a01n02
SN - Link 1
  an-a02n02
SN - Link 1
an-a01n02
IFN - Link 1
an-a02n02
IFN - Link 1
-- -- Uplink 1
X2 (1/2/2) X4 (1/2/4)   2 (1/1/2) 4 (1/1/4) 6 (1/1/6) 8 (1/1/8) 10 (1/1/10) 12 (1/1/12)   14 (1/1/14) 16 (1/1/16) 18 (1/1/18) 20 (1/1/20) 22 (1/1/22) 24 (1/1/24)
Unused   VID 100 VID 200 VID 300
 
an-switch02  
Stack   Back-Channel Network]] Storage Network Internet-Facing Network
X1 (2/2/1) X3 (2/2/3)   1 (2/1/1) 3 (2/1/3) 5 (2/1/5) 7 (2/1/7) 9 (2/1/9) 11 (2/1/11)   13 (2/1/13) 15 (2/1/15) 17 (2/1/17) 19 (2/1/19) 21 (2/1/21) 23 (2/1/23)
To an-switch01
X3
To an-switch01
X1
  an-a01n01
BCN - Link 2
an-pdu01 an-a02n01
BCN - Link 2
an-ups01 an-striker02
BCN - Link 1
an-a01n01
SN - Link 2
  an-a02n01
SN - Link 2
an-a01n01
IFN - Link 2
an-a02n01
IFN - Link 2
an-striker01
IFN - Link 2
-- --
-- --   an-a01n02
BCN - Link 2
an-pdu02 an-a02n02
BCN - Link 2
an-ups02 -- an-a01n02
SN - Link 2
  an-a02n02
SN - Link 2
an-a01n02
IFN - Link 2
an-a02n02
IFN - Link 2
-- -- Uplink 2
X2 (2/2/2) X4 (2/2/4)   2 (2/1/2) 4 (2/1/4) 6 (2/1/6) 8 (2/1/8) 10 (2/1/10) 12 (2/1/12)   14 (2/1/14) 16 (2/1/16) 18 (2/1/18) 20 (2/1/20) 22 (2/1/22) 24 (2/1/24)
Unused   VID 100 VID 200 VID 300

Cabling a pair of ICX6610-48

In this example, we're showing how to cable up a pair of ICX6610-48 switches in a rather unusual configuration. It is driving just one Anvil!, but that Anvil! is using 10 Gbps SFP+ links. The switches are going to be populated with non-Anvil! equipment.

What you will configure will, of course, totally vary. The specific ports aren't the point, the planning and implementation are.

The table below represents a pair of these stacked together.
an-switch01    
BCN SN IFN   Back-Channel Network Internet-Facing Network
X1 (1/2/1) X3 (1/2/3) X5 (1/2/5) X7 (1/2/7)   1 (1/1/1) 3 (1/1/3) 5 (1/1/5) 7 (1/1/7) 9 (1/1/9) 11 (1/1/11) 13 (1/1/13) 15 (1/1/15) 17 (1/1/17) 19 (1/1/19) 21 (1/1/21) 23 (1/1/23) 25 (1/1/25) 27 (1/1/27) 29 (1/1/29) 31 (1/1/31) 33 (1/1/33) 35 (1/1/35) 37 (1/1/37) 39 (1/1/39) 41 (1/1/41) 43 (1/1/43) 45 (1/1/45) 47 (1/1/47)
an-a05n01
BCN - Link 1
an-a05n01
SN - Link 1
an-a05n01
IFN - Link 1
    an-a05n01
BCN - Link 3
an-a05n01
IPMI / Fencing
an-striker01
BCN - Link 1
an-striker01
IPMI
        an-a05n01
IFN - Link 3
an-striker01
IFN - Link 1
                           
X2 (1/2/2) X4 (1/2/4) X6 (1/2/6) X8 (1/2/8)   2 (1/1/2) 4 (1/1/4) 6 (1/1/6) 8 (1/1/8) 10 (1/1/10) 12 (1/1/12) 14 (1/1/14) 16 (1/1/16) 18 (1/1/18) 20 (1/1/20) 22 (1/1/22) 24 (1/1/24) 26 (1/1/26) 28 (1/1/28) 30 (1/1/30) 32 (1/1/32) 34 (1/1/34) 36 (1/1/36) 38 (1/1/38) 40 (1/1/40) 42 (1/1/42) 44 (1/1/44) 46 (1/1/46) 48 (1/1/48)
an-a05n02
BCN - Link 1
an-a05n02
SN - Link 1
an-a05n02
IFN - Link 1
    an-a05n02
BCN - Link 3
an-a05n02
IPMI / Fencing
an-striker02
BCN - Link 1
          an-a05n02
IFN - Link 3
an-striker02
BCN - Link 1
                          Uplink 1
VID 100 VID 200 VID 300   VID 100 VID 300
 
an-switch02  
BCN SN IFN   Back-Channel Network Internet-Facing Network
X1 (2/2/1) X3 (2/2/3) X5 (2/2/5) X7 (2/2/7)   1 (2/1/1) 3 (2/1/3) 5 (2/1/5) 7 (2/1/7) 9 (2/1/9) 11 (2/1/11) 13 (2/1/13) 15 (2/1/15) 17 (2/1/17) 19 (2/1/19) 21 (2/1/21) 23 (2/1/23) 25 (2/1/25) 27 (2/1/27) 29 (2/1/29) 31 (2/1/31) 33 (2/1/33) 35 (2/1/35) 37 (2/1/37) 39 (2/1/39) 41 (2/1/41) 43 (2/1/43) 45 (2/1/45) 47 (2/1/47)
an-a05n01
BCN - Link 2
an-a05n01
SN - Link 2
an-a05n01
IFN - Link 2
    an-pdu01 an-ups01 an-striker02
BCN - Link 2
an-striker02
IPMI
an-pdu03         an-striker02
IFN - Link 2
                           
X2 (2/2/2) X4 (2/2/4) X6 (2/2/6) X8 (2/2/8)   2 (2/1/2) 4 (2/1/4) 6 (2/1/6) 8 (2/1/8) 10 (2/1/10) 12 (2/1/12) 14 (2/1/14) 16 (2/1/16) 18 (2/1/18) 20 (2/1/20) 22 (2/1/22) 24 (2/1/24) 26 (2/1/26) 28 (2/1/28) 30 (2/1/30) 32 (2/1/32) 34 (2/1/34) 36 (2/1/36) 38 (2/1/38) 40 (2/1/40) 42 (2/1/42) 44 (2/1/44) 46 (2/1/46) 48 (2/1/48)
an-a05n02
BCN - Link 2
an-a05n02
SN - Link 2
an-a05n02
IFN - Link 2
    an-pdu02 an-ups02 an-striker02
BCN - Link 2
  an-pdu04         an-striker02
BCN - Link 2
                          Uplink 2
VID 100 VID 200 VID 300   VID 100 VID 300

Understanding Brocade Logical Port Numbers

Brocade uses the follow scheme for naming their ports:

  • stack ID/module/port

The Stack ID is simply the integer representing the ID number shown when you run 'show stack'.

The module number is static across all switches. They are:

  • 1: Standard RJ-45 1 Gbps copper ports.
  • 2: Ports used for stacking. Typically these are only used for stacking.
  • 3: The SFP+ fiber or copper ports that normally run at 1 Gbps but can be licensed to run at 10 Gbps.

The port number is the simple integer representing the port. This is the number found above or below the physical port on the switch.

You see these x/y/z numbers in the table above, and they will be the method used below to identify ports and port ranges.

Forming the VLANs

To start, lets take a look at the current VLAN configuration.

ICX6610-48 Switch#configure terminal
ICX6610-48 Switch(config)#show vlan
Total PORT-VLAN entries: 1
Maximum PORT-VLAN entries: 64

Legend: [Stk=Stack-Id, S=Slot]

Not much!

ICX6610-48 Switch(config)#vlan 100 name bcn

The BCN spans SFP ports 1 and 2 (1/3/1 to 1/3/1, 2/3/1 to 2/3/1) and copper ports 1 through 16 (1/1/1 to 1/1/16, 2/1/1 to 2/1/16) will be assigned to this VLAN. We can do this in one command.

ICX6610-48 Switch(config-vlan-100)#untag ethernet 1/3/1 to 1/3/2 ethernet 2/3/1 to 2/3/2 ethernet 1/1/1 to 1/1/16 ethernet 2/1/1 to 2/1/16
Added untagged port(s) ethe 1/1/1 to 1/1/16 ethe 1/3/1 to 1/3/2 ethe 2/1/1 to 2/1/16 ethe 2/3/1 to 2/3/2 to port-vlan 100.

The SN spans just four ports, all of which are on the SFP+ ports. They are 1/3/3 to 1/3/4 and 2/3/3 to 2/3/4.

ICX6610-48 Switch(config-vlan-100)#vlan 200 name sn
ICX6610-48 Switch(config-vlan-200)#untag ethernet 1/3/3 to 1/3/4 ethernet 2/3/3 to 2/3/4
Added untagged port(s) ethe 1/3/3 to 1/3/4 ethe 2/3/3 to 2/3/4 to port-vlan 200.

Finally, the biggest VLAN. We're going to allocate the last four SFP+ ports on both switch and copper ports 17 through 48 on both as well.

ICX6610-48 Switch(config-vlan-200)#vlan 300 name ifn
ICX6610-48 Switch(config-vlan-300)#untag ethernet 1/3/5 to 1/3/8 ethernet 2/3/5 to 2/3/8 ethernet 1/1/17 to 1/1/48 ethernet 2/1/17 to 2/1/48
Added untagged port(s) ethe 1/1/17 to 1/1/48 ethe 1/3/5 to 1/3/8 ethe 2/1/17 to 2/1/48 ethe 2/3/5 to 2/3/8 to port-vlan 300.

Done! Let's see what it looks like now:

ICX6610-48 Switch(config-vlan-300)#exit
ICX6610-48 Switch(config)#show vlan
Total PORT-VLAN entries: 4
Maximum PORT-VLAN entries: 64

Legend: [Stk=Stack-Id, S=Slot]

PORT-VLAN 1, Name DEFAULT-VLAN, Priority level0, Spanning tree On
 Untagged Ports: None
   Tagged Ports: None
   Uplink Ports: None
 DualMode Ports: None
 Mac-Vlan Ports: None
     Monitoring: Disabled
PORT-VLAN 100, Name bcn, Priority level0, Spanning tree On
 Untagged Ports: (U1/M1)   1   2   3   4   5   6   7   8   9  10  11  12 
 Untagged Ports: (U1/M1)  13  14  15  16 
 Untagged Ports: (U1/M3)   1   2 
 Untagged Ports: (U2/M1)   1   2   3   4   5   6   7   8   9  10  11  12 
 Untagged Ports: (U2/M1)  13  14  15  16 
 Untagged Ports: (U2/M3)   1   2 
   Tagged Ports: None
   Uplink Ports: None
 DualMode Ports: None
 Mac-Vlan Ports: None
     Monitoring: Disabled                                         
PORT-VLAN 200, Name sn, Priority level0, Spanning tree On
 Untagged Ports: (U1/M3)   3   4 
 Untagged Ports: (U2/M3)   3   4 
   Tagged Ports: None
   Uplink Ports: None
 DualMode Ports: None
 Mac-Vlan Ports: None
     Monitoring: Disabled
PORT-VLAN 300, Name ifn, Priority level0, Spanning tree On
 Untagged Ports: (U1/M1)  17  18  19  20  21  22  23  24  25  26  27  28 
 Untagged Ports: (U1/M1)  29  30  31  32  33  34  35  36  37  38  39  40 
 Untagged Ports: (U1/M1)  41  42  43  44  45  46  47  48 
 Untagged Ports: (U1/M3)   5   6   7   8 
 Untagged Ports: (U2/M1)  17  18  19  20  21  22  23  24  25  26  27  28 
 Untagged Ports: (U2/M1)  29  30  31  32  33  34  35  36  37  38  39  40 
 Untagged Ports: (U2/M1)  41  42  43  44  45  46  47  48 
 Untagged Ports: (U2/M3)   5   6   7   8 
   Tagged Ports: None
   Uplink Ports: None
 DualMode Ports: None
 Mac-Vlan Ports: None
     Monitoring: Disabled

Perfectly perfect. Save the changes and then we're done.

ICX6610-48 Switch(config)#write memory
Write startup-config done.

Automatic copy to member units:  2
Flash Memory Write (8192 bytes per dot) .
Flash to Flash Done.

Changing a Port's VLAN Assignment

In order to change a port from one VLAN to another, it must first to untagged as VLAN ID 1, then it can be re-assigned to the new VLAN ID.

### (config-vlan-200) no untag ethernet 2/1/15 to 2/1/24
### (config-vlan-200) show vlan
Note: A port in VID 1 will simply change to the newly assigned VID. Ports in other VIDs need to be reset with no untag ... before reconfiguring for the new VLAN.

Pseudo Stacking on ICX6430-C12

To share one uplink between multiple VLANs (to act like a stack);

config terminal
an-s05(config)#vlan 100
an-s05(config-vlan-100)#tagged ethernet 1/2/1
Added tagged port(s) ethe 1/2/1 to port-vlan 100.
an-s05(config-vlan-100)#vlan 200
an-s05(config-vlan-200)#tagged ethernet 1/2/1
Added tagged port(s) ethe 1/2/1 to port-vlan 200.
an-s05(config-vlan-200)#vlan 300
an-s05(config-vlan-300)#tagged ethernet 1/2/1
Added tagged port(s) ethe 1/2/1 to port-vlan 300.

To prevent switch loops when plugging in both C2.

an-s05(config-vlan-300)#spanning-tree rstp

Trunking

Passive Trunk

This is used when the connected switch does not support LACP.

Note: Configure with only one cable connected into the trunk ports.
# config terminal
## (config) trunk ethernet 1/1/24 ethernet 2/1/24

Trunk will be created in next trunk deploy.

## (config) trunk deploy
## (config) write memory

Active Trunk

This is used if you can create an LACP on the client's switch.

Note: Configure with only one cable connected into the trunk ports.
# config terminal
## (config) interface ethernet 1/1/23 ethernet 2/1/23
## (config-mif-1/1/23,2/1/23) link-aggregate active
## exit
## show link-aggregate

System ID: 748e.f8ff.cf1c
Long  timeout: 90, default: 90
Short timeout: 3, default: 3
Port  [Sys P] [Port P] [  Key ] [Act][Tio][Agg][Syn][Col][Dis][Def][Exp][Ope]
1/1/23      1        1      482   Yes   S   Agg  Syn  Col  Dis  Def  No   Dwn
2/1/23      1        1      482   Yes   S   Agg  Syn  Col  Dis  Def  No   Dwn

Key Management

To get a license, you must purchase the license from a Brocade reseller. With it, you will get a spreadsheet containing a TRANSACTION KEY.

To convert this into a loadable license file, you will need to get the switch's 'LID', then create an account on "My Brocade". You can then combine the LID and the TRANSACTION KEY and Brocade will email you the license key.

To get the LID;

ICX6610-48 Switch#show version
  Copyright (c) 1996-2013 Brocade Communications Systems, Inc.
    UNIT 1: compiled on May 10 2013 at 16:48:26 labeled as FCXS07300f
                (5370497 bytes) from Primary /foundry/FGS/os/FCXS07300f.bin
        SW: Version 07.3.00fT7f1 
    UNIT 2: compiled on May 10 2013 at 16:48:26 labeled as FCXS07300f
                (5370497 bytes) from Primary /foundry/FGS/os/FCXS07300f.bin
        SW: Version 07.3.00fT7f1 
  Boot-Monitor Image size = 370555, Version:07.3.02T7f5 (grz07302)
  HW: Stackable ICX6610-48
==========================================================================
UNIT 1: SL 1: ICX6610-48 48-port Management Module
         Serial  #: xxxxxxxxxxx
         License: BASE_SOFT_PACKAGE   (LID: yyyyyyyyyyy)
         P-ENGINE  0: type E02B, rev 01
         P-ENGINE  1: type E02B, rev 01
==========================================================================
UNIT 1: SL 2: ICX6610-QSFP 10-port 160G Module
==========================================================================
UNIT 1: SL 3: ICX6610-8-port Dual Mode(SFP/SFP+) Module
==========================================================================
UNIT 2: SL 1: ICX6610-48 48-port Management Module
         Serial  #: aaaaaaaaaaa
         License: BASE_SOFT_PACKAGE   (LID: bbbbbbbbbbb)
         P-ENGINE  0: type E02B, rev 01                           
         P-ENGINE  1: type E02B, rev 01
==========================================================================
UNIT 2: SL 2: ICX6610-QSFP 10-port 160G Module
==========================================================================
UNIT 2: SL 3: ICX6610-8-port Dual Mode(SFP/SFP+) Module
==========================================================================
  800 MHz Power PC processor 8544E (version 0021/0023) 400 MHz bus
65536 KB flash memory
  512 MB DRAM
STACKID 1  system uptime is 1 days 17 hours 8 minutes 17 seconds 
STACKID 2  system uptime is 22 hours 36 minutes 24 seconds 
The system : started=warm start  reloaded=by "reload"
My stack unit ID = 1, bootup role = active

Navigating Brocade's Website

With the TRANSACTION KEY and LID, and having logged into Brocade Software Portal;

  • Hover over the top-left 'License Management' button. A drop-down menu will appear. Hover over 'Brocade IP/ADP'. Another pop-out menu will appear, click on 'License Generation with Transaction Key'.
  • A form will appear. Fill it our as instructed. Check to accept the EULA and then click 'Add'.
    • IF YOU ARE ADDING TWO LICENSES TO ONE SWITCH:
    • Fill out the LID and Transaction key for the next license, again accept the EULA and click 'Add' again.
  • Once all licenses for the specified switch are added, click on 'Generate'.
  • If all was well, you will be shown a summary and an email with the license will be emailed to you.

Repeat as needed for each switch.

Installing Keys

Once you get the license file from Brocade, copy the key to the root of your TFTP server (/var/lib/tftpboot/ on EL6).

Note the LID number and which stack ID. In this case, I renamed the license files so that an-switch01_XXXXXXXXXXX.xml, where XXXXXXXXXXX is the switch's LID. In the example below, I will load two licenses, one for either switch in the stack.

ICX6610-24 Switch#copy tftp license 10.255.255.254 an-switch01_XXXXXXXXXXX.xml unit 1
Flash Memory Write (8192 bytes per dot) .
Copy Software License from TFTP to Flash Done.
T=1d20h6m27.7: Election, was active, no change, ID=1, pri=128, 2U(1-2), A=u1, nbr#=1 1,
ICX6610-24 Switch#copy tftp license 10.255.255.254 an-switch02_XXXXXXXXXXX.xml unit 2
Flash Memory Write (8192 bytes per dot) .
Copy Software License from TFTP to Flash Done.

Copy software license to stack unit 2 success
Flash Memory Write (8192 bytes per dot) .
T=1d20h7m50.1: Election, was active, no change, ID=1, pri=128, 2U(1-2), A=u1, nbr#=1 1,
ICX6610-24 Switch#show license
Index      License Name              Lid          License Type    Status     License Period  License Capacity  
Stack unit 1:
1          ICX6610-10G-LIC-POD       xxxxxxxxxxx  Normal          Active     Unlimited                  8 
Stack unit 2:
1          ICX6610-10G-LIC-POD       aaaaaaaaaaa  Normal          Active     Unlimited                  8
write memory

Repeat as needed.

Enable 10 Gbps Ports

After installing license files to upgrade the eight 1 Gbps SFP+ ports to run as 10 Gbps, we need to actually enable the ports. This may seem silly, but consider that the upgrades are sold in lots of 4-ports at a time. This step would allow you to specify which four ports were upgraded. In our case, we upgraded them all.

ICX6610-48 Switch#config terminal
ICX6610-48 Switch(config)#interface ethernet 1/3/1 to 1/3/8
ICX6610-48 Switch(config-mif-1/3/1-1/3/8)#speed-duplex 10g-full

This will take a minute, please wait.

ICX6610-48 Switch(config-mif-1/3/1-1/3/8)#interface ethernet 2/3/1 to 2/3/8
ICX6610-48 Switch(config-mif-2/3/1-2/3/8)#speed-duplex 10g-full
ICX6610-48 Switch(config-mif-2/3/1-2/3/8)#exit


If you (accidentally) use a 1 Gbps SFP+ cable, the upgrade will likely throw an error like the one below:

U2-MSG: INFO: 2/3/4: optics <-> speed mismatch. Replace with SFP+ to enable link.
U2-MSG: INFO: 2/3/4: optics <-> speed mismatch. Replace with SFP+ to enable link.

You can reduce the port speed to make it work (albeit at 1 Gbps):

ICX6610-48 Switch(config-mif-1/3/1-1/3/8)#interface ethernet 2/3/4
ICX6610-48 Switch(config-if-e10000-2/3/4)#speed-duplex 1000-full-master

When done, save your changes.

ICX6610-48 Switch(config)#write memory
Write startup-config done.

Automatic copy to member units:  2
ICX6610-48 Switch(config)#Flash Memory Write (8192 bytes per dot) .
Flash to Flash Done.

Deleting Keys

If you need to remove a key (ie: replacing a demo license), you can do so thusly:

ICX6610-24 Switch#license delete unit 1 index 1

ICX6610-24 Switch#show license
License record empty

Replacing a Switch

Warning: The switch that is the stack master will overwrite the configuration of the stack slave. So be sure to flash the firmware to match the existing switch, license the ports if needed and enable stacking on the replacement switch while it is stand-alone. Power it off, plug in the stack cables and then apply power. When it boots, it's config should be replaced with the good switch's configuration. To be safe, in an Anvil!, withdraw a node before reforming the stack.
Warning: You must contact Brocade to move the licenses!

If a switch needs to be replaced, we may need to "de-license" the old switch.

Call Brocade support (1-800-752-8061) and use the case number the switch was replaced under. Confirm with them the LID of the switch being replaced (the one that the license will be removed from) and the LID of the new/replacement switch.

With the switch being returned, there is no need to prove the old license is removed as you will be returning the switch. If you will NOT be returning the switch, then you will need to provide a show license before and after deleting the old license in order for the license to be reissued. This is a rare case and the Brocade technician will walk you through the process should it be required.

They will email you the replacement license, at which point you will license the replacement switch as if it were a new switch.

Note: Depending your email client, you may need to request the XML license file be attached separately.

Checking the Firmware

Before you proceed, make sure the Firmware is the same version as the stack master. If not, upgrade (or downgrade) the RMAed switch to match.

Licensing the Switch

Once you have the XML file, proceed from here.

Adding the Switch to the Stack

For this step, we want to enable stacking on the new switch and nothing more.

stack enable
Enable stacking. This unit actively participates in stacking
stacking is enable. optical monitoring for stacking ports 1/2/1, 1/2/6 is not available.
write memory
Write startup-config done.

ICX6610-48 Switch(config)#Flash Memory Write (8192 bytes per dot) .
Flash to Flash Done.
Note: In the example below, switch 1 is being replaced.

At this point, switch to the production switch (which ever isn't being replaced). Connect the newly flashed and licensed replacement switch and you should see the configuration automatically upload to the newly replaced switch.

ICX6610-48 Switch#T=4d4h15m31.2: Election, was active, no change, ID=2, pri=128, 2U(1-2), A=u2, nbr#=1 0, 
T=4d4h15m32.2: Election, was active, no change, ID=2, pri=128, 2U(1-2), A=u2, nbr#=1 0, 
Detect stack unit 1 has different startup config flash, will synchronize it
T=4d4h15m34.0: Election, was active, no change, ID=2, pri=128, 2U(1-2), A=u2, nbr#=1 1, 
Detect stack unit 1 has different startup config flash, will synchronize it
reset unit 1: jumbo mismatch. Act u2: 1, Mem u1: 0
reset unit 1: jumbo mismatch. Act u2: 1, Mem u1: 0
Unit 2 loses all neighbors.
Active unit 2 deletes u1 but keeps its static config.

Some time passes:

T=4d4h18m52.3: Election, was active, no change, ID=2, pri=128, 2U(1-2), A=u2, nbr#=1 1, 
Detect stack unit 1 has different startup config flash, will synchronize it
Done hot swap: active controller unit 2 sets unit 1 to Ready.
Synchronize startup config to stack unit 1
Flash Memory Write (8192 bytes per dot) .
Assigned unit 1 to be standby
start running config sync
Running config sync to standby is complete

At this point, recovery is complete.

Unconfigure Stack

If you boot a stacked switch on it's own, you will not have normal access to the switch configuration.

MEMBER]local-1@ICX6610-48 Switch>enable
No password has been assigned yet...
[MEMBER]local-1@ICX6610-48 Switch#

To clear the stack config and restore the switch to standard mode, run:

stack unconfigure clean
This unit will delete all config files and boot up as a clean unit. Are you sure? (enter 'y' or 'n'): y
Remove startup config and stacking files. Will reload as a clean unit
[MEMBER]local-1@ICX6610-48 Switch#HRebooting(1)...

Multicast Config

ICX6610-24 Switch#config terminal 
ICX6610-24 Switch(config)#vlan 300
ICX6610-24 Switch(config-vlan-300)#multicast passive 
ICX6610-24 Switch(config-vlan-300)#multicast version 2

Done

Resetting Lost Password

If you've lost the system password, you will need physical access to the switch.

Note: If you can't sit at your computer and press the reset button directly, ask someone to help with this next step as the timing is short.

On the front of the switch will be a small "Reset" switch. To press this, you will need a very thin object in order to press the button, like a small paper clip, needle or so on.

During the boot process, press b when you see the Enter 'b' to stop at boot monitor: prompt.

Enter 'b' to stop at boot monitor:  0 
ICX64XX-boot>> no password
OK! Skip password check when the system is up.
ICX64XX-boot>> boot system flash primary
Usage:
boot    - boot default, i.e., run 'bootcmd'

ICX64XX-boot>> boot_primary
Booting image from Primary
## Booting image at 00007fc0 ...
   Created:      2013-04-27   3:19:57 UTC
   Data Size:    10552120 Bytes = 10.1 MB
   Load Address: 00008000
   Entry Point:  00008000
   Verifying Checksum ... OK
OK

On next boot, you will be able to log into the switch without a password.

Reset to Factory Defaults

Note: This will erase any loaded licenses. If you have loaded licenses, be sure you have them backed up before proceeding.

At the command prompt, do:

SHESW11#erase startup-config 
Erase startup-config Done.
dhcp server lease database is also removed
Warning: Do not write memory! This will reload the running configuration back onto the system.
SHESW11#reload
Are you sure? (enter 'y' or 'n'): 
Sent SIGKILL to all processes
Requesting system reboot
Restarting syste

On the next boot, you should have an as-new system.

SNMP

ICX-6430-C12

To enable read-write access of MIB data;

an-s06(config)#snmp-server community public rw

Notes

Port numbering syntax is:

<stack>/<module>/<port>

Module numbering;

  • 1; Copper 1 Gbit
  • 2; Fibre stack
  • 3; 10 Gbps fibre

View the port status:

ICX6610-48 Switch(config)#show interfaces brief

Port    Link    State   Dupl Speed Trunk Tag Pvid Pri MAC            Name      
1/1/1   Down    None    None None  None  No  1    0   cc4e.24b9.4b74          
1/1/2   Down    None    None None  None  No  1    0   cc4e.24b9.4b75          
1/1/3   Down    None    None None  None  No  1    0   cc4e.24b9.4b76          
1/1/4   Down    None    None None  None  No  1    0   cc4e.24b9.4b77          
1/1/5   Down    None    None None  None  No  1    0   cc4e.24b9.4b78          
1/1/6   Down    None    None None  None  No  1    0   cc4e.24b9.4b79          
1/1/7   Down    None    None None  None  No  1    0   cc4e.24b9.4b7a          
1/1/8   Down    None    None None  None  No  1    0   cc4e.24b9.4b7b          
1/1/9   Down    None    None None  None  No  1    0   cc4e.24b9.4b7c          
1/1/10  Down    None    None None  None  No  1    0   cc4e.24b9.4b7d          
1/1/11  Down    None    None None  None  No  1    0   cc4e.24b9.4b7e          
1/1/12  Down    None    None None  None  No  1    0   cc4e.24b9.4b7f          
1/1/13  Down    None    None None  None  No  1    0   cc4e.24b9.4b80          
1/1/14  Down    None    None None  None  No  1    0   cc4e.24b9.4b81          
1/1/15  Down    None    None None  None  No  1    0   cc4e.24b9.4b82          
1/1/16  Down    None    None None  None  No  1    0   cc4e.24b9.4b83          
1/1/17  Down    None    None None  None  No  1    0   cc4e.24b9.4b84          
1/1/18  Down    None    None None  None  No  1    0   cc4e.24b9.4b85          
1/1/19  Down    None    None None  None  No  1    0   cc4e.24b9.4b86          
1/1/20  Down    None    None None  None  No  1    0   cc4e.24b9.4b87          
1/1/21  Down    None    None None  None  No  1    0   cc4e.24b9.4b88          
1/1/22  Down    None    None None  None  No  1    0   cc4e.24b9.4b89          
1/1/23  Down    None    None None  None  No  1    0   cc4e.24b9.4b8a          
1/1/24  Down    None    None None  None  No  1    0   cc4e.24b9.4b8b          
1/1/25  Down    None    None None  None  No  1    0   cc4e.24b9.4b8c          
1/1/26  Down    None    None None  None  No  1    0   cc4e.24b9.4b8d          
1/1/27  Down    None    None None  None  No  1    0   cc4e.24b9.4b8e          
1/1/28  Down    None    None None  None  No  1    0   cc4e.24b9.4b8f          
1/1/29  Down    None    None None  None  No  1    0   cc4e.24b9.4b90          
1/1/30  Up      Forward Full 1G    None  No  1    0   cc4e.24b9.4b91          
1/1/31  Up      Forward Full 1G    None  No  1    0   cc4e.24b9.4b92          
1/1/32  Down    None    None None  None  No  1    0   cc4e.24b9.4b93          
1/1/33  Up      Forward Full 1G    None  No  1    0   cc4e.24b9.4b94          
1/1/34  Up      Forward Full 1G    None  No  1    0   cc4e.24b9.4b95          
1/1/35  Up      Forward Full 1G    None  No  1    0   cc4e.24b9.4b96          
1/1/36  Up      Forward Full 1G    None  No  1    0   cc4e.24b9.4b97          
1/1/37  Up      Forward Full 1G    None  No  1    0   cc4e.24b9.4b98          
1/1/38  Up      Forward Full 1G    None  No  1    0   cc4e.24b9.4b99          
1/1/39  Down    None    None None  None  No  1    0   cc4e.24b9.4b9a          
1/1/40  Down    None    None None  None  No  1    0   cc4e.24b9.4b9b          
1/1/41  Up      Forward Full 1G    None  No  1    0   cc4e.24b9.4b9c          
1/1/42  Up      Forward Full 1G    None  No  1    0   cc4e.24b9.4b9d          
1/1/43  Up      Forward Full 1G    None  No  1    0   cc4e.24b9.4b9e          
1/1/44  Down    None    None None  None  No  1    0   cc4e.24b9.4b9f          
1/1/45  Down    None    None None  None  No  1    0   cc4e.24b9.4ba0          
1/1/46  Up      Forward Full 1G    None  No  1    0   cc4e.24b9.4ba1          
1/1/47  Down    None    None None  None  No  1    0   cc4e.24b9.4ba2          
1/1/48  Up      Forward Full 1G    None  No  1    0   cc4e.24b9.4ba3          
1/2/1   Up      Forward Full 40G   None  No  N/A  0   cc4e.24b9.4ba5          
1/2/2   Up      Forward Full 10G   None  No  N/A  0   cc4e.24b9.4ba6          
1/2/6   Up      Forward Full 40G   None  No  N/A  0   cc4e.24b9.4ba7          
1/2/7   Up      Forward Full 10G   None  No  N/A  0   cc4e.24b9.4ba8          
1/3/1   Down    None    None None  None  No  1    0   cc4e.24b9.4ba9          
1/3/2   Up      Forward Full 10G   None  No  1    0   cc4e.24b9.4baa          
1/3/3   Down    None    None None  None  No  1    0   cc4e.24b9.4bab          
1/3/4   Down    None    None None  None  No  1    0   cc4e.24b9.4bac          
1/3/5   Down    None    None None  None  No  1    0   cc4e.24b9.4bad          
1/3/6   Up      Forward Full 10G   None  No  1    0   cc4e.24b9.4bae          
1/3/7   Down    None    None None  None  No  1    0   cc4e.24b9.4baf          
1/3/8   Down    None    None None  None  No  1    0   cc4e.24b9.4bb0          
mgmt1   Down    None    None None  None  No  None 0   cc4e.24b9.4b74

 

Any questions, feedback, advice, complaints or meanderings are welcome.
Alteeve's Niche! Enterprise Support:
Alteeve Support
Community Support
© Alteeve's Niche! Inc. 1997-2024   Anvil! "Intelligent Availability®" Platform
legal stuff: All info is provided "As-Is". Do not use anything here unless you are willing and able to take responsibility for your own actions.