Managing_Ethernet_Switches

• Overview
• Discovering Switches
  • Installing and configuring lldpd
• Running Remote Commands in Parallel
  • How to Add New Switch Types

Overview

When managing Ethernet switches, the admin often logs into the switches one by one using SSH or Telnet and runs the switch commands. However, it becomes time consuming when there are a lot of switches in a cluster. In a very large cluster, the switches are often identical and the configurations are identical. It helps to configure and monitor them in parallel from a single command.

For managing Mellanox IB switches, see the following: [Managing_the_Mellanox_Infiniband_Network].

For managing Qlogic IB switches , see the following: [Managing_the_Infiniband_Network]

Discovering Switches

From xCAT 2.10, you can use switchdiscover command to discover the switches that are attaches to the neighboring subnets on a xCAT management node. The syntax of the command is:

    switchdiscover [noderange|--range ip_ranges][-s scan_methods][-r|-x|-z][-w]

where the scan_methods can be nmap or lldp. The default is nmap. (nmap comes from most os distribution. lldp needs lldpd package. Please refer to "Running Remote Commands in Parallel" section for details.)

For example:

    # switchdiscover --range 10.4.25.0/24,192.168.0.0/24
    Discovering switches using nmap...
    ip              name                    vendor                  mac
    ------------    ------------            ------------            ------------
    192.168.0.131   switch_192_168_0_131    Mellanox Technologie    00:02:C9:AA:00:53
    10.4.25.1       switch_10_4_25_1        Juniper networks        2C:6B:F5:00:11:22

If -w flag is specified, the command will write the discovered switches into xCAT databases. It looks like this.

# lsdef switch_10_4_25_1
Object name: switch_10_4_25_1
    groups=switch
    ip=10.4.25.1
    mgt=switch
    nodetype=switch
    switchtype=Juniper

The ip address is stored in the hosts table. You can run the following command to add the ip addresses in the /etc/hosts

    makehosts

The discovery process works with the following 4 kind of switches:

• Mellanox (IB and Ethernet switches)
• Cisco
• BNT
• Juniper

The switchdiscover command can display the output in xml format, stanza forma and normal list format. Please see the man pages for this command for details.

Installing and configuring lldpd

If you want to use lldp scan method for the switchdiscover command, please make sure that lldpd is installed. lldpd comes with xcat-dep package. You can simply installed it from there using yum or zipper command. Or you can follow the instructions from http://vincentbernat.github.io/lldpd/installation.html to install it.

After the installation, please add the following line in the /etc/sysconfig/lldpd file.

LLDPD_OPTIONS="-c -s -e -f"

Then you can start or stop the lldpd daemon using the service command.

    service lldpd start

Running Remote Commands in Parallel

From xCAT 2.8, you can use xdsh to run parallel commands on Ethernet switches. The following shows how to configure xCAT to run xdsh on the switches:

1. Configure the switch to allow ssh or telnet. This varies for switch to switch. Please refer to the switch command references to find out how to do it.

2. Add the switch in xCAT DB. Please refer to the "Discovering Switches" section if you want xCAT to discover and define the switches for you.

       mkdef bntc125 groups=switch mgt=switch ip=10.4.25.1 nodetype=switch switchtype=BNT

1. Set the ssh or telnet username an d password.

       chdef bntc125 username=admin \
                     password=password \
                     protocol=ssh
     or 
       chdef bntc125 username=admin \
                     password=password \
                     protocol=telnet

If there are a lot of switches and they have the same user name and password for ssh or telnet connection, you can put them in the passwd table keyed by 'switch'. You can use the comments attribute to describe it is for ssh to telnet. The blank means ssh.

    #key,username,password,cryptmethod,authdomain,comments,disable
    "system","root","cluster",,,,
    "switch","admin","password",,,,

1. Run xdsh command

    xdsh bntc125 --devicetype EthSwitch::BNT "enable;configure terminal;vlan 3;end;show vlan"

Please note that you can run multiple switch commands, they are separated by comma.
Please also note that --devicetype is used here. xCAT ships the following switch types:

             * BNT 
             * Cisco 
             * Juniper
             * Mellanox (for IB and Ethernet switches)

If you have different type of switches, you can either use the general flag
"--devicetype EthSwitch" or add your own switch types. (See the following section).
Here is what result will look like:

       bntc125: start SSH session...
       bntc125:  RS G8000>enable
       bntc125:  Enable privilege granted.
       bntc125: configure terminal
       bntc125:  Enter configuration commands, one per line.  End with Ctrl/Z.
       bntc125: vlan 3
       bntc125: end
       bntc125: show vlan
       bntc125: VLAN                Name                Status            Ports
       bntc125:  ----  --------------------------------  ------  ------------------------ 
       bntc125:  1     Default VLAN                      ena     45-XGE4
       bntc125:  3     VLAN 3                            dis     empty
       bntc125:  101   xcatpriv101                       ena     24-44
       bntc125:  2047  9.114.34.0-pub                    ena     1-23 44

You can run xdsh against more than one switches at a time,just like running xdsh against nodes.
Use xcoll to summarize the result. For example:

      xdsh bntc1,bntc2 --devicetype EthSwitch::BNT  "show access-control" |xcoll

The output looks like this:

      ====================================
       bntc1,bntc2
      ====================================
      start Telnet session...
      terminal-length 0
      show access-control
      Current access control configuration:
         No ACLs configured.
         No IPv6 ACL configured.
         No ACL group configured.
         No VMAP configured.

How to Add New Switch Types

For any new switch types that's not supported by xCAT yet, you can use the general "--device EthSwitch" flag with xdsh command.

       xdsh <switch_names> --devicetype EthSwitch "cmd1;cmd2..."

The only problem is that the page break is not handled well when the command output is long. To remove the page break, you can add a switch command that sets the terminal length to 0 before all other commands.

     xdsh <switch_names> --devicetype EthSwitch "command-to-set-term-length-to-0;cmd1;cmd2..."

 where command-to-set-term-length-to-0 is the command 
 to set the terminal length to 0 so that the output does not have page breaks.

You can add this command to the configuration file to avoid specifying it for each xdsh by creating a new switch type. Here is what you do:

       cp /opt/xcat/share/xcat/devicetype/EthSwitch/Cisco/config \
           /var/opt/xcat/EthSwitch/XXX/config

where XXX is the name of the new switch type. You can give it any name.
Then add the command for set terminal length to 0 to the "pre-command" line.
The new configuration file will look like this:

      # cat /var/opt/xcat/EthSwitch/XXX/config
      [main]
      ssh-setup-command=
      [xdsh]
      pre-command=command-to-set-term-length-to-0;
      post-command=NULL

For BNT switches, the command-to-set-term-length-to-0 is "terminal-length 0".

Please make sure to add a semi-colon at the end of the "pre-command" line.

Then you can run the xdsh like this:

       xdsh <switch_names> --devicetype EthSwitch::XXX "cmd1;cmd2..."