IBM_HPC_Software_Kits

There is a newer version of this page. You can find it here.

New in xCAT 2.8. Supported for Linux OS images only.

xCAT Kit support replaces the Linux IBM HPC Integration Support (xCAT-IBMhpc) previously shipped with xCAT 2.7.x and older releases. If you are running on an AIX cluster or are using IBM HPC products in an xCAT 2.7.x or older Linux release, follow the instructions in [IBM_HPC_Stack_in_an_xCAT_Cluster].

Introduction

This document contains specific information for using IBM HPC Product software kits in an xCAT cluster. For general use of software kits with xCAT Linux OS images, see: [Using_Software_Kits_in_OS_Images]

It is important to first understand the concepts, commands, and procedures presented in that document in order to apply the specific details listed here. For most IBM HPC products, you will simply follow the general procedures in working with kits.

IBM HPC product software kits combine the product packages with configuration files, installation scripts, environment variables, exclude lists, and other data that is unique to deploying and running that product in an HPC xCAT cluster. Kit components for different cluster roles such as compute, storage, login, utility, servicenode, and mgtnode are available as supported by the HPC product.

IBM HPC Product Software Kits

Obtaining the Kits

Kits for the IBM HPC Products are distributed with the product software. Currently, only incomplete kits for some of the product PTFs are available from the FixCentral download site. Select the Product Group "Cluster Software" and the appropriate HPC product.

See the specific product sections below for detailed information on each supported IBM HPC product.

This document will be updated as more IBM HPC products develop kits for their products and make them available for use in xCAT clusters.

General Instructions for all HPC Kits

Again, be sure you understand all of the information, concepts, and processes for general use of software kits with xCAT Linux OS images documented in: [Using_Software_Kits_in_OS_Images]

This is a quick overview of the commands you will use to add kits to your xCAT cluster and use them in your Linux OS images. Please read the specific product sections below for any exceptions or additions to this process for a particular product.

  • Obtain your IBM HPC product kits

  • If your kits are partial kits, obtain our IBM HPC product packages and build complete kits by combining the partial kit with the product packages: [#_Completing_the_kit_build_for_a_partial_kit]

    Note: Before completing the GPFS kit, you will need to first build the GPFS portability layer package on one of your servers. See the special instructions for [#]General_Parallel_File_System(GPFS) kits below.

  • Make the completed kits locally available on your xCAT management node

  • Unpack each kit and add it to the xCAT database by running:

    addkit <product_kit_tarfile>

  • List the product kitcomponents that are available for your OS image:

    lsdef -t kitcomponent | grep <product>

  • Kit components are typically named based on server role, product version, OS version, and optionally if it is for a minimal image (minimal images exclude documentation, includes, and other optional files to reduce the diskless image size). To list the details of a particular kit component:

    lsdef -t kitcomponent -o <kitcomponent name> -l

  • Your OS image must be defined with a supported serverrole in order to add a kit component to the image. To query the role assigned to an image:

    lsdef -t osimage -o <image> -i serverrole

    And to change the serverrole of an image:

    chdef -t osimage -o <image> serverrole=<role>

  • To add or update a kitcomponent in an osimage, first check if the kitcomponent is compatible with your image:

    chkkitcomp -i <image> <kitcomponent name>

    If compatible, add the component to that image:

    addkitcomp -i <image> <kitcomponent name>

    This will add various files for otherpkgs, postinstall, and postbootscripts to the OS image definition. To view some of these:

    lsdef -t osimage -l <image>

  • If this is a diskless stateless or statelite OS image, rebuild, pack, and deploy the image:

    genimage <image>
    #
    packimage <image>
    # OR
    liteimg <image>
    #
    nodeset <noderange> osimage=<image>
    #
    rpower <noderange>

  • If this is a statefull OS image, the kitcomponent will be installed with the next full node deployment. To apply the image update to an active node, run:

    updatenode <noderange>

    and the kitcomponent will be installed as part of the otherpkgs processing.

Completing the kit build for a partial kit

Typically, a software kit will contain all of the product package files. However, in some instances, software kits may be delivered as partial or incomplete kits, and not be bundled with all of the product rpms. You will then need to obtain your product packages through your reqular distribution channels, download them to a server along with the incomplete kit, and run the buildkit addpkgs command to build a complete kit that can be used by xCAT. The file name of the kit tarfile will indicate if it is a partial kit. For example, if the name of your product kit tarfile is something like: 

_product-version_-Linux.NEED_PRODUCT_PKGS.tar.bz2

the string "NEED_PRODUCT_PACKAGES" in the file name indicates that you have a partial kit.

If you received your IBM HPC product software kit as a partial kit, follow these steps to complete the kit build process:

  1. Install the optional xCAT-buildkit rpm on your server. This rpm will not automatically install with your other xCAT packages and does not necessarily need to be installed on an xCAT management node. The xCAT-buildkit rpm does not have any dependencies on any other xCAT packages. However, it does require the rpmbuild and createrepo commands to be available. For RHELS, these commands are provided by the rpm-build and createrepo packages respectively. For SLES, they are provided by the rpm and createrepo packages(note that createrepo is shipped on the SLES SDK iso).
  2. Download the kit tarfile and the product package files and make them locally available on your server.
  3. cd to a work directory

  4. Build the complete kit tarfile:

    buildkit addpkgs <kit.NEED_PRODUCT_PKGS.tar.bz2> --pkgdir <product package directory>

You may receive warning messages about the current OS/arch not matching the kit repository being built. These can be ignored.

Parallel Environment Runtime Edition (PE RTE)

PE RTE software kits are available for PE RTE 1.3 and newer releases.

For PE RTE 1.2 and older releases, use the xCAT HPC Integration Support: [IBM_HPC_Stack_in_an_xCAT_Cluster]

No special procedures are required for using the PE RTE kit. If you received an incomplete kit, simply follow the previously documented process for adding the product packages and building the complete kit: [#Completing_the_kit_build_for_a_partial_kit]

Parallel Environment Developer Edition (PE DE)

PE DE software kits are available for PE DE 1.2.0.1 and newer releases.

For older releases, use the xCAT HPC Integration Support: [IBM_HPC_Stack_in_an_xCAT_Cluster]

No special procedures are required for using the PE DE kit. If you received an incomplete kit, simply follow the previously documented process for adding the product packages and building the complete kit: [#Completing_the_kit_build_for_a_partial_kit]

Engineering and Scientific Subroutine Library (ESSL)

ESSL is not yet available as a Kit

For ESSL, use the xCAT HPC Integration Support: [IBM_HPC_Stack_in_an_xCAT_Cluster]

Parallel Engineering and Scientific Subroutine Library (PESSL)

PESSL is not yet available as a Kit

For PESSL, use the xCAT HPC Integration Support:

[IBM_HPC_Stack_in_an_xCAT_Cluster]

General Parallel File System (GPFS)

GPFS software kits are available for GPFS 3.5.0.7 and newer releases.

For GPFS 3.5.0.6 and older releases, use the xCAT HPC Integration Support: [IBM_HPC_Stack_in_an_xCAT_Cluster]

The GPFS kit requires the addition of the GPFS portability layer package to be added to it. This rpm must be built at your site on a server that matches the architecture and kernel version of all OS images that will be using this kit.

Follow this procedure before using the GPFS kit that you received:

  • On a server that has the correct architecture and kernel version, manually install the GPFS rpms and build the portability layer according to the instructions documented by GPFS: General Parallel File System
  • Copy the gpfs.gplbin rpm that you have successfully created to the server that you are using to complete the build of your GPFS kit, placing it in the same directory as your other GPFS rpms.

  • Complete the kit build:

    buildkit addpkgs <gpfs-kit-NEED_PRODUCT_PKGS-tarfile> -p <gpfs-rpm-directory>

At this point you can now follow the general instructions for working with kits to add the kit to your xCAT database and add the GPFS kitcomponents to your OS images.

IBM Compilers

IBM Compilers are not yet available as Kits

For IBM Compilers, use the xCAT HPC Integration Support: [IBM_HPC_Stack_in_an_xCAT_Cluster]

Toolkit for Event Analysis and Logging (TEAL)

TEAL is not yet available as a Kit

For TEAL, use the xCAT HPC Integration Support: [IBM_HPC_Stack_in_an_xCAT_Cluster]

Switching from xCAT IBM HPC Integration Support to Using Software Kits

If you currently have OS images defined and built using the xCAT IBM HPC Integration Support ([IBM_HPC_Stack_in_an_xCAT_Cluster]), you will need to create completely new OS images to use with IBM HPC Software Kits. xCAT does not provide any upgrade or migration support for replacing the HPC products in your current OS images with the equivalent kits.

Existing xCAT OS images built with xCAT 2.7.x and older releases can continue to be built and deployed on xCAT 2.8+. However, the xCAT IBM HPC Integration Support for Linux will not be updated as new versions of the HPC products become available. To use new versions of the products in your Linux OS images, you will either need to manually update your image customization files and scripts to accommodate any changes to the products, or you will need to switch over to using the new software kits built by the products for use in xCAT HPC clusters.

You can mix HPC Integration Support with HPC kits for different HPC products in the same OS image definition if necessary.

IBM HPC software kits are available with the following and newer versions of these software products. For previous versions of the products, you will need to continue to use the HPC Integration Support.

For x86_64:

  • PE RTE 1.3
  • PE DE 1.2.0.1
  • GPFS 3.5.0.7

For ppc64: 

 _no IBM HPC Software Kits are available at this time_
MongoDB Logo MongoDB