XCAT_system_x_support_for_IBM_Flex

There is a newer version of this page. You can find it here.

THIS PAGE UNDER CONSTRUCTION

Introduction

IBM Flex combines networking, storage, and compute nodes in a single offering. It's consist of an IBM Flex Chassis, one or two Chassis Management Modules(CMM) and compute nodes. The compute nodes include the IBM Flex System™ p260, p460, and 24L Power 7 servers as well as the IBM Flex System™ x240 x86 server. In this document only the management of x240 blade server will be covered.

Terminology

The following terms will be used in this document:

Chassis Management Module(CMM) - this term is used to reflect the pair of management modules installed in the rear of the chassis and connected by ethernet to the MN. The CMM is used to discover the compute nodes within the chassis and for some data collection regarding the nodes and chassis.

Compute node: This term is used to refer to the servers in an IBM Flex system. Compute nodes can be either Power 7 servers or x86 Intel based servers.

blade node: blade node refers to a node with the hwtype set to blade and represents the whole compute node server.

Compute blade: refers to a compute server with a blade hwtype.

Define the CMM and blades

This section will describe the commands needed to define both the CMM node definitions as well as the blade definitions.

Define the CMM with IP setting

This command will define 2 cmm nodes, cmm1 and cmm2, with the required attributes for management and configuration. 

mkdef cmm[1-2] groups=all,amm mgt=blade mpa=amm$i nodetype=mm username=USERID password=passw0rd ip=70.0.0.($1+6)

Define the blades for each CMM

This command will define all of the blades for cmm1 and cmm2. Note that it will create 14 blades per chassis. If this is not your configuration you can use this method and then delete unwanted blade definitions or use a more manual method. 

mkdef cmm[01-02]node[01-14] cons=blade groups=blade,all mgt=ipmi nodetype=osi,blade conserver=9.114.34.219 arch=x86_64 
installnic=mac monserver=9.114.34.219 profile=compute netboot=xnba nfsserver=9.114.34.219 tftpserver=9.114.34.219
xcatmaster=9.114.34.219 bmcpassword=passw0rd bmcusername=USERID getmac=blade

Define the mpa and slot id for each blade in the mp table

This section provides an example of how to modify the mp table to define the blade and cmm configuration. 

# tabdump mp
#node,mpa,id,nodetype,comments,disable
"blade","|cmm(\d+)node(\d+)|cmm($1)|","|cmm(\d+)node(\d+)|($2+0)|",,,
"cmm01","cmm01",,,,

For example the cmm mpa will be defined as follows: 

# lsdef cmm01
Object name: cmm01
    groups=all,cmm
    ip=70.0.0.7
    mac=5c:f3:fc:25:da:99
    mgt=blade 
    mpa=cmm01
    postbootscripts=otherpkgs
    postscripts=syslog,remoteshell,syncfiles

Prepare the CMM discovery

xCAT supports two methods to do the CMM matching, one is matching CMM by the CMM switch port information; the other way is matching CMM by the known CMM MAC address (This requires that the user has already collected the MAC of CMM).

Note: xCAT NGP discovery now does not support the CMM with both primary and standby port.

Matching by the switch port information

If you use the CMM switch port information to do the xCAT CMM matching, you need add the switch/port information to the table switch and switches: 

# tabdump switch
#node,switch,port,vlan,interface,comments,disable
"cmm01","xcat3750","Fa1/0/1",,,,


# tabdump switches
#switch,snmpversion,username,password,privacy,auth,linkports,sshusername,sshpassword,switchtype,comments,disable
"xcat3750","3","xcat","passw0rd",,"SHA",,,,,,

Matching by the CMM MAC address

If you use MAC infomration to do the matching then you can get the mac manually using the arp command.

You can also collect the CMM mac addresses automatically by discovering the CMM via lsslp 

# lsslp -m -z -s CMM
cmm01:
        objtype=node
        mpa=cmm01
        nodetype=mp
        mtm=789392X
        serial=100037A
        side=2
        groups=cmm,all
        mgt=blade
        mac=5c:f3:fc:25:da:99
        hidden=0
        otherinterfaces=70.0.0.7
        hwtype=cmm

run slpdiscover

Note: please delete the ipmi.bmcid, if you have run slpdiscover before.

Matching CMM by switch port information

For matching the CMM with switch port information, slpdiscover includes the following steps:

  1. locate the CMM
  2. configure the CMM
  3. locate the blades by slot id
  4. configure the IMM IP/USER ID/PW for each nodes that has been defined in MN.

Now, We only support the CMM connecting to the vlan 1 of the switch setting. 

run slpdiscover

Matching CMM by MAC

For the way that matching CMM by MAC address, slpdiscover includes the following steps: Locate the CMM, locate the blades by slot id, configure the IMM IP/USER ID/PW for each nodes that have been defined in MN. 

# slpdiscover
cmm01: Found service:management-hardware.IBM:chassis-management-module at address 70.0.0.7
cmm01: Ignoring target in bay 7, no node found with mp.mpa/mp.id matching
cmm01: Ignoring target in bay 8, no node found with mp.mpa/mp.id matching
Configuration of cmm01node05[70.0.0.16] commencing, configuration may take a few minutes to take effect

Note: 

  1. "cmm01: Ignoring target in bay 7, no node found with mp.mpa/mp.id matching" This message means the bay7 is not defined on MN. 
  2. the IMM password will be same with the CMM password setting.

Checking the result of the slpdiscover command

After the slpdiscover the interfaces should be defined and configured properly. User can ping the bmc ip and try the hardware control commands to verify that the bmc ip and userid/password have been defined and configured correctly.

After the slpdiscover, user also can check the ipmi table if the bmcid has been updated with blades mac. 

# tabdump ipmi
#node,bmc,bmcport,taggedvlan,bmcid,username,password,comments,disable
"blade","|cmm(\d+)node(\d+)|10.0.($1+0).($2+0)|",,,,"USERID","passw0rd",,
"cmm01node05","70.0.0.16",,,"5c:f3:fc:6e:00:41",,,,
"cmm01node06","70.0.0.15",,,"5c:f3:fc:6e:03:94",,,,
MongoDB Logo MongoDB