Building_Software_Kits
There is a newer version of this page. You can find it here.
New in xCAT 2.8. Supported for Linux OS images only.
TO BE WRITTEN!! This Document is a WORK IN PROGRESS
Introduction
This document describes how to build a product Kit to package software for installation in an xCAT cluster.
Input to buildkit
buildkit.conf Configuration File
Product Packages
Scripts
xCAT Plugins
Documentation
The buildkit Process
To build a Kit, the following steps are required:
Install the xCAT-buildkit RPM
Install the xCAT-buildkit RPM on your build server.
This rpm does not have any other dependencies on xCAT, and does not necessarily have to be installed on an xCAT management node. The xCAT-buildkit RPM does require rpmbuild and createrepo which are available from the Linux distribution..
Starting with xCAT version 2.8.2 the xCAT-buildkit RPM will be installed automatically as part of installing base xCAT.
If you are using an older version of xCAT or your build server is not an xCAT management node, then you will have to either:
1) Download the the xCAT tar file and install the xCAT-buildkit RPM from a local repository.
OR
2) Install the RPM directly from the internet-hosted repository.
This general process is decribed in: [Install_xCAT_on_the_Management_Node].
Once the repositories are set up you would either:
[RH]: Use yum to install xCAT-buildkit and all its dependencies:
yum clean metadata
yum install xCAT-buildkit
[SLES]: Use zypper to install xCAT-buildkit and all its dependencies:
zypper install xCAT-buildkit
Create a Kit Template directory
Create a template work directory structure for your kit.
buildkit create <kit_basename>
This will create a sub-directory called "<kit_basename>" in your current directory. If you wish to have your work directory in a different location you can specify it on the buildkit command line by using the "-l" option. See the buildkit manpage for details.
This Kit directory location will automatically be populated with additional subdirectories and sample files. See [#Kit_directory_structure] for a description.
Edit the kit build configuration file.
To build a Kit, you need to modify the Kit build configuration file which is used to build the Kit tarfile, Kit Package Repositories, Kit Component Meta-Packages, and individual Kit Packages.
The name of the file is "buildkit.conf" and it is located in the Kit template work directory that you just created.
<kit directory location>/buildkit.conf
The sample buildkit.conf file contains a description of all the supported attributes and an indication of wheteher or not they are required or optional. See [#Sample_buildkit.conf_file].
Partial vs. complete Kits
A partial kit is one that does NOT include the product packages.
In this case, the user will need to download both the kit tarfiles and the product packages, and complete the kit build process before using the kit in an xCAT cluster. See [Using_Software_Kits_in_OS_Images]
If you wish to build a partial kit be sure to set "isexternalpkg=yes" in the "kitpackage" stanzas of your buildkit.conf file.
For example:
kitpackage:
filename=foobar_runtime-*.x86_64.rpm
mkitrepoid=rhels6_x86_64,sles11_x86_64
# Method 1: Use pre-built RPM package
isexternalpkg=yes
Copy Files into the Kit directory structure
After editing the buildkit configuration file (buildkit.conf), you need to copy files into the Kit directory structure.
The types of files you need to copy are:
Source Packages - IBM HPC Product RPM packages. - These files should be copied to:
<Kit directory location>/source_packages
Deployment Scripts - These files should be copied to:
<Kit directory location>/scripts
Plugins - For IBM HPC Products in this first release, no plugins will be developed. However, future support may be considered by the product packaging developers. For example, GPFS may choose to implement a nodemgmt.pm plugin that would add a new node to the GPFS cluster. - These files should be copied to:
<Kit directory location>/plugins
Doc Files - These files should be copied to:
<Kit directory location>/docs
Other Files - These include: kit deployment parameter file, exclude lists - These files should be copied to:
<Kit directory location>/other_files
When you copy these files, make sure their location matches what is in the Kit Build File. For example, if the location of an RPM spec file is “pkg2/pkg2.spec” in the Kit Build File, then the actual spec file should be found in this location:
<Kit directory location>/source_packages/pkg2/pkg2.spec
Validate the Kit Build File
After you copy the files to your kit build directory, you can manually run a command to validate if the buildkit configuration file is correct:
buildkit chkconfig
This will verify that all required fields are specified, that all internally referenced attributes are defined, and that all referenced files exist. Fix any errors that are reported.
Build the Kit Package Repositories
After the buildkit configuration file is validated, you build the Kit Package Repositories.
You may need to build these on a build host which has the same OS name/version/distro as the Kit Package Repository. What this means is if you have one RHEL 6 repo, and one SLES 11 repo, then you must log into a RHEL 6 host to build the first repo, then log into a SLES 11 host to build the second repo.
For IBM HPC Products, since you are using pre-built rpms, you should be able to build all repositories on the same server since there should not be anything OS/arch specific in the kitcomponent meta-package rpm build.
buildkit buildrepo <Kit Pkg Repo>
or
buildkit buildrepo all
If the Kit Package Repository is already fully built, then this command performs no operation.
- If the Kit Package Repository is not fully built, the command builds it as follows:
- Create the Kit Package Repository directory “<Kit directory location>/build/kit_repodir/<Kit Pkg Repo> “
- Build the Component Meta-Packages associated with this Kit Package Repository. Create the packages under the Kit Package Repository directory
- Build the Kit Packages associated with this Kit Package Repository. Create the packages under the Kit Package Repository directory
- Build the repository meta-data for the Kit Package Repository. The repository meta-data is based on the OS native package format. For example, for RHEL, we build YUM repository meta-data with the createrepo command.
Build the Kit Tarfile
After you build the Kit Package Repositories, you can build the final Kit tarfile.
buildkit buildtar
Note:
A complete kit name will be named something like:
testprod-1.0.0-x86_64.tar.bz2
A partial kit name will be indicated by including the "NEED_PRODUCT_PKGS" string in its name.
testprod-1.0.0-x86_64.NEED_PRODUCT_PKGS.tar.bz2
Kit directory structure
**<kit directory location>** \- The full path name of _either_ the location specified on the command line, or the current working directory where you ran the **buildkit** command plus the <kit_basename> you provided.
**<kit directory location>/source_packages** \- This directory stores the source packages for Kit Packages and Non-Native Packages. The **buildkit** CLI will search these directories for source packages when building packages. This directory stores:
- RPM spec and tarballs. (A sample spec file is provided.)
- Source RPMs.
- Pre-built RPMs
-
Non-Native Packages
<kit directory location>/scripts - This directory stores the Kit Deployment Scripts. Samples are provided for each type of script.
<kit directory location>plugins - This directory stores the Kit Plugins. Samples are provided for each type of plugin.
<kit directory location>/docs - This directory stores the Kit documentation files.
<kit directory location>/tmp - tbd
<kit directory location>/rpmbuild - tbd
<kit directory location>/other_files
-
kitdeployparams.lst: Kit Deployment parameters file
-
exclude.lst: File containing files/dirs to exclude in stateless image.
<kit directory location>/build - This directory stores files when the Kit is built.
<kit directory location>/build/kit_repodir - This directory stores the fully built Kit Package Repositories
<kit directory location>/build/<kitname> - This directory stores the contents of the Kit tarfile before it is tar'red up.
Sample buildkit.conf file
Note: The sample buildkit.conf file will change over time. To get the latest version of the file look for it in /opt/xcat/share/xcat/kits/kit_template after you have installed the xCAT_buildkit RPM.
# Kit Build File
#
# This file was initially auto-generated by the buildkit CLI command.
# To create a new kit, the user should modify this file accordingly.
# Refer to the buildkit manpage for further details.
#
# kit: This section defines general info for the Kit.
# There must be exactly one kit section in a kit build file.
#
# kit attributes:
# basename (mandatory) Kit base name. e.g., kit-lsf
# description (optional) Kit description.
# version (mandatory) Kit version. e.g., 1.0
# release (optional) Kit release. e.g., 1
# ostype (mandatory) Kit OS type. Must be Linux.
# AIX is currently not supported.
# vendor (optional) The vendor tag is used to define the name of
# the entity that is responsible for packaging
# the software.
# packager (optional) The packager tag is used to hold the name and
# contact information for the person or persons
# who built the package.
# url (optional) The url tag is used to provide a location to
# obtain additional information about the
# packaged software.
# osbasename (optional) OS distro base name. e.g., rhels.
# osmajorversion (optional) OS major version. e.g., 6
# osminorversion (optional) OS minor version.
# osarch (optional) OS architecture. e.g., x86_64
# isinternal (optional) PCM use only.
# Indicate if Kit is for internal use.
# Use 1 for yes, 0 for no. Default: 0
# kitdeployparams (optional) Filename containing a list of kit deployment
# parameters, relative to
# <Kit Build Directory>/other_files
# kitlicense (mandatory) Kit license string to be built into all
# kitcomponent packages
# kittarfilename (optional) Filename.tar.bz2 to be used for the generated
# kit. Default is
# <basename>-<version>-<opt-release>-<opt-osbasename>-<opt-osmajorversion>-<opt-osminorversion>-<opt-osarch>.tar.bz2
kit:
basename=<INSERT_kitbasename_HERE>>>
description=description for <INSERT_kitbasename_HERE>>>
version=1.0
release=1
ostype=Linux
kitdeployparams=sample/kitdeployparams.lst
kitlicense=EPL
#
#
# kitrepo: This section defines a Kit Package Repository.
# There must be at least one kitrepo section in a kit build file.
# If you want to support multiple OSes, you should create a separate
# repo for each OS. Also, no two repos can be defined with the same
# OS name, major/minor version and arch. For example, you cannot have
# two repos for RHEL 6.2 x86_64 in the same kit.
#
# kitrepo attributes:
# kitrepoid (mandatory) Kit package repository ID.
# Must be unique within this file.
# osbasename (mandatory) OS distro base name. e.g., rhels.
# osmajorversion (mandatory) OS major version. e.g., 6
# osminorversion (optional) OS minor version.
# osarch (mandatory) OS architecture. e.g., x86_64
# compat_osbasenames (optional) Comma-separated list of compatible
# OS distro base names. e.g., centos
kitrepo:
kitrepoid=<INSERT_kitrepoid_HERE>>>
osbasename=<INSERT_osbasename_HERE>>>
osmajorversion=<INSERT_osmajorversion_HERE>>>
osminorversion=<INSERT_osminorversion_HERE>>>
osarch=<INSERT_osarch_HERE>>>
#compat_osbasenames=
#
# kitcomponent: This section defines one Kit Component.
# There can be zero or more kitcomponent sections.
# If you want to build a component which supports multiple OSes,
# you should create one kitcomponent section for each OS.
# You can define multiple kit components with the same base name
# only if each kit component using this base name meets these
# requirements:
# - Each kit component must be defined with the same version
# and release number
# - Each kit component must be defined with a unique kitrepoid
#
#
# kitcomponent attributes:
# basename (mandatory) Kit component base name
# description (optional) Kit component description
# version (optional) Kit component version. If not set, defaults
# to kit version
# release (optional) Kit component release. If not set, defaults
# to kit release. Error if both are not set.
# serverroles (mandatory) Comma-separated list of servers that this
# component can install on. Valid values:
# mgtnode,servicenode,compute,login,storage,utility
# kitrepoid (mandatory) The ID of the kit package repository this
# component belongs to
# kitcompdeps (optional) Comma-separated list of kit component
# dependencies. These kit components can be included in
# this kit or in other kits.
# ospkgdeps (optional) Comma-separated list of OS package dependencies
# These packages must be shipped with the OS distro.
# kitpkgdeps (optional) Comma-separated list of kit package names that
# will be listed as "REQUIRES" when building this kit
# component. Each package must be defined in a separate
# kitpackage section. Each package must be in the same
# kitrepo as this kit component.
# non_native_pkgs (optional)
# Comma-separated list of non-native package
# paths that will be included as files in this kit
# component. All filenames are relative to
# <Kit Build Directory>/source_packages and may contain
# wildcards. If a filename is prefixed by 'EXTERNALPKGS:'
# the file will not be built into the kit tarfile, but
# will need to be added later with a 'buildkit addpkgs'
# command.
# Files will be placed in
# /opt/xcat/kits/<kitbasename>/<kitcomponent_name>
# when the kitcomponent package is deployed to an
# OS image.
# Kit component deployment scripts must be specified
# to manage these files.
# driverpacks (optional) Comma-separated list of driver package filenames
# Each driverpack must be defined in a separate kitpackage
# section.
# exlist (optional) Exclude list file for stateless image, relative
# to <Kit Build Directory>/other_files
# Kit component deployment scripts (optional) Each attribute specifies
# script path relative to <Kit Build Directory>/scripts
# Script attributes:
# preinstall, postinstall, preuninstall, postuninstall,
# preupgrade, postupgrade, postbootscripts,
# genimage_postinstall
kitcomponent:
basename=<INSERT_kitcomponent_basename_HERE>>>
description=description for component <INSERT_kitcomponent_basename_HERE>>>
serverroles=compute
kitrepoid=<INSERT_kitrepoid_HERE>>>
#kitcompdeps=
#ospkgdeps=
kitpkgdeps=pkg1
#kitpkgdeps=pkg1,pkg2,pkg2,pkg4
#non_native_pkgs=a_kitcomponent.file
#non_native_pkgs=EXTERNALPKGS:a_kitcomponent.file
#driverpacks=
exlist=sample/exclude.lst
preinstall=sample/pre.sh
postinstall=sample/post.sh
preuninstall=sample/preun.sh
postuninstall=sample/postun.sh
preupgrade=sample/preup.sh
postupgrade=sample/postup.sh
postbootscripts=sample/postboot.sh
genimage_postinstall=sample/genimage_post.sh
# kitpackage: This section defines one Kit Package, and how to build it.
# There can be zero or more kitpackage sections.
# All filenames should be relative paths to
# <Kit Build Directory>/source_packages
# If you want to build a package which can run on multiple OSes,
# you have two options:
# 1. Build a separate package for each OS you want to support.
# For this option, you need to define one kitpackage section
# per supported OS.
# 2. Build one package that can run on multiple OSes.
# If you are building an RPM package, you are responsible for
# creating an RPM spec file that can run on multiple OSes.
# For this option, you need to define one kitpackage section
# which contains multiple kitrepoid lines.
#
#
# kitpackage attributes:
# filename (mandatory) Package filename.
# kitrepoid (mandatory) A comma-separated list of kit reponames this package
# belongs to. If multiple repos are defined, the package will
# be built for the first repo only. For the other repos,
# a symlink is created to the package built for the first repo.
# Package build methods (optional)
# Define how to build the packages.
# If you don't specify a build method, the default behavior is
# to assume the package is pre-built under
# <Kit Build Directory>/source_packages
# All files in this section are relative to
# <Kit Build Directory>/source_packages
# There are four methods to build packages.
# 1. Use pre-built RPM package
# The filename may contain wildcards to avoid needing to
# specify an explicit package version-release filename.
# isexternalpkg: 'no'|'0', 'yes'|'1' (default: 'no')
# Indicates whether the RPM package will be added to the
# the kit tar file now as part of the kit build process,
# or whether the customer will need to separately
# obtain the RPM pacakage and add it to the kit tar file
# using 'buildkit addpkgs'
# rpm_prebuiltdir: If isexternalpkg=no, path to directory
# containing pre-built RPM package
# 2. Build RPM from spec + src dir
# rpm_spec: Path to spec file.
# rpm_srcdir: Path to source directory.
# 3. Build RPM from spec + src tarball
# rpm_spec: Path to spec file.
# rpm_srctarball: Path to source tarball.
# 4. Build RPM from source RPM
# rpm_srpm: Path to source RPM package.
kitpackage:
filename=pkg1-1-1.noarch.rpm
kitrepoid=<INSERT_kitrepoid_HERE>>>
# Method 1: Use pre-built RPM package
isexternalpkg=no
rpm_prebuiltdir=sample/pkg1
#
# kitpackage:
# filename=pkg2-1-1.noarch.rpm
# kitrepoid=<INSERT_kitrepoid_HERE>>>
# # Method 2: Build RPM from spec + src dir
# rpm_spec=sample/pkg2/pkg2.spec
# rpm_srcdir=sample/pkg2/pkg2
#
# kitpackage:
# filename=pkg3-1-1.noarch.rpm
# kitrepoid=<INSERT_kitrepoid_HERE>>>
# # Method 3: Build RPM from spec + src tarball
# rpm_spec=sample/pkg3/pkg3.spec
# rpm_srctarball=sample/pkg3/pkg3.tar.gz
#
# kitpackage:
# filename=pkg4-1-1.noarch.rpm
# kitrepoid=<INSERT_kitrepoid_HERE>>>
# # Method 4: Build RPM from source RPM
# rpm_srpm=sample/pkg4/pkg4-1-1.src.rpm
#
