XCAT_P8LE_Hardware_Management

There is a newer version of this page. You can find it here.

Hardware discoverying for P8 LE machines

Firmware updating for P8 LE machine

The firmware updating process can be done during discovery or at a later time. The steps are:

Download firmware file from Support Portal in IBM webpage(www.ibm.com). The firmware name is like this: 01TVXXX_XXX_XXX.img

Put it into a tarball:

The firmware file downloaded
a runme.sh script that you create that runs the executable with appropriate flags

For example:
#cd /install/firmware/
#ls -lh

total 197M
-rw-r--r-- 1 root root 197M Oct 10 08:10 01TV810_061_054.img
-rwxr-xr-x 1 root root  149 Oct 13 07:36 runme.sh

*#cat runme.sh

echo "================Start update"
/bin/update_flash -f ./01TV810_061_054.img

*# chmod +x runme.sh 
*# tar -zcvf firmware-update.tgz .

./
./runme.sh
./01TV810_061_054.img
tar: .: file changed as we read it

. start firmware update:

Option 1 - update during discovery:

If you want to update the firmware during the node discovery process, ensure you have already added a dynamic range to the networks table and run "makedhcp -n". Then update the chain table to do both bmcsetup and the firmware update:
chdef node001 chain="runcmd=bmcsetup,runimage=http://mgmtnode/install/firmware/firmware-update.tgz,shell"

Option2 - update after node deployment:

If you are updating the firmware at a later time (i.e. not during the node discovery process), tell nodeset that you want to do the firmware update, and then set currchain to drop the nodes into a shell when they are done:

nodeset node001 runimage=http://mgmtnode/install/firmware/firmware-update.tgz,boot

Provisioning OS for powerKVM and VMs

Provisioning PowerKVM for p8 LE machine

This is the process for setting up PowerKVM with xCAT

create the osimage object

Since the powerkvm has not been released yet, you need to following steps before copycds:

 mkdir /tmp/pkvm_iso 

 mount –o loop /iso/ibm-powerkvm-201403250837.iso /tmp/pkvm_iso

 cat /tmp/pkvm_iso/.discinfo
 1395751343.334203
 IBM_PowerKVM 2.1
 ppc64
 1

Edit the file: /opt/xcat/lib/perl/xCAT/data/discinfo.pm

replace the follow line

 "1394549807.920540" => "pkvm2.1", # ppc64
 with the num get from /tmp/pkvm_iso/.discinfo.

The line will be similar to:

  "1395751343.334203" => "pkvm2.1", # ppc64

Then run copycds

 copycds /iso/ibm-powerkvm-201403250837.iso

To check the osimage object created by copycds, run the following:

   lsdef -t osimage
   pkvm2.1-ppc64-install-compute (osimage)

Define the node object

   mkdef n1 groups=all,kvm cons=ipmi mgt=ipmi
   chdef n1 bmc=<bmc ip> bmcpassword=<bmc password>
   chdef n1 mac=<xx:xx:xx:xx:xx:xx> installnic=mac primarynic=mac
   chdef n1 tftpserver=<ip_of_xCAT_MN> conserver=<ip_of_xCAT_MN> nfsserver=<ip_of_xCAT_MN>

Note: The discovery is not supported. So the mac address must be obtained by user.

Setup dns

define the name domain for this cluster

  chdef -t site domain=vmcluster.com

Define IP address for the node

  chdef n1 ip=x.x.x.x
  makedns –n

Config DNS server, the resolv.conf file will be similar to:

cat /etc/resolv.conf
domain vmcluster.com
search vmcluster.com
nameserver <ip_of_xCAT_MN>
Check the DNS setup
nslookup n1
Server: <ip_of_xCAT_MN>
Address: <ip_of_xCAT_MN>

Name: n1.vmcluster.com
Address: <ip_of_node_n1>

Prepare the petitboot and dhcpd configurations

  chdef n1 netboot=petitboot
  makedhcp -n
  makedhcp -a
  nodeset n1 osimage=pkvm2.1-ppc64-install-compute
  n1: install pkvm2.1-ppc64-compute
  rsetboot n1 net

Reboot the node to start the provisioning process

rpower n1 on/reset

Steps to install ubuntu14.04 LE for powerkvm VM through network

Define node "vm1" as a normal vm node:

  mkdef vm1 groups=vm,all
  chdef vm1 vmhost=pkvm01
  chdef vm1 tftpserver=<ip_of_xCAT_MN> conserver=<ip_of_xCAT_MN> nfsserver=<ip_of_xCAT_MN>
  chdef vm1 ip=x.x.x.x
  makehosts vm1
  makedns -n
  makedns -a

create VM

   chdef vm1 mgt=kvm cons=kvm

   chdef vm1 vmcpus=2 vmmemory=4096 vmnics=br0 vmstorage=phy://dev/sdd

   mkvm vm1

For more information about modifying VM attributes, pls refer to Define Virtual Machines attributes

Create ubuntu LE osimage object

After you download the latest ubuntu LE ISO, pls run the following command to create osimage objects.

  copycds trusty-server-ppc64el.iso

You can check the /install/<ubuntu-version>/ppc64el directory have been created. And you can find the osimage objects with:</ubuntu-version>

   lsdef -t osimage
   ubuntu14.04-ppc64el-install-compute  (osimage)
   ubuntu14.04-ppc64el-install-kvm  (osimage)
   ubuntu14.04-ppc64el-netboot-compute  (osimage)
   ubuntu14.04-ppc64el-statelite-compute  (osimage)

And in order to boot from network, you need to download the mini.iso from "http://ports.ubuntu.com/ubuntu-ports/dists/trusty/main/installer-ppc64el/current/images/netboot/",(pls use http://ports.ubuntu.com/ubuntu-ports/dists/utopic/main/installer-ppc64el/current/images/netboot/mini.iso for ubuntu 14.10) then mount the mini.iso to a tmp directory:

  mkdir /tmp/iso
  mount -o loop mini.iso /tmp/iso
  ls /tmp/iso/install
  initrd.gz  vmlinux

Then, copy the file /tmp/iso/install/initrd.gz to /install/<ubuntu-version>/ppc64el/install/netboot.</ubuntu-version>

  mkdir -p  /install/<ubuntu-version>/ppc64el/install/netboot
  cp  /tmp/iso/install/initrd.gz  /install/<ubuntu-version>/ppc64el/install/netboot

Prepare grub2 and dhcpd configurations

Make sure the grub2 had been installed on your Management Node:

  rpm -aq | grep grub2
  grub2-xcat-1.0-1.noarch

Note: At present, the modules for xCAT shipped grub2 can not support ubuntu LE smoothly. So the following steps needed to complete the grub2 setting.

  rm /tftpboot/boot/grub2/grub2.ppc
  cp /tftpboot/boot/grub2/powerpc-ieee1275/core.elf /tftpboot/boot/grub2/grub2.ppc
  /bin/cp -rf /tmp/iso/boot/grub/powerpc-ieee1275/elf.mod /tftpboot/boot/grub2/powerpc-ieee1275/

Set 'netboot' attribute to 'grub2'

  chdef vm1 netboot=grub2

Config password for root:

chtab key=system passwd.username=root passwd.password=xxxxxx

Create grub2 boot configuration fileby running nodeset:

nodeset vm1 osimage=ubuntu14.04-ppc64el-install-compute

Power on the VM to start OS installing

rpower vm1 on

Use console to monitor the installing process

On the pkvm host, pls make sure firewalld service had been stopped.

chkconfig firewalld off

Note: Forwarding request to systemctl will disable firewalld.service.

  rm /etc/systemd/system/basic.target.wants/firewalld.service 
  rm /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service

Then, run wvid vm1 on MN

wvid vm1

Provisioning OS for powerNV

The steps below are used to provisioning ubuntu LE for powerNV.

Create ubuntu LE osimage object

copycds trusty-server-ppc64el.iso

lsdef -t osimage
ubuntu14.04-ppc64el-install-compute (osimage)
ubuntu14.04-ppc64el-install-kvm (osimage)
ubuntu14.04-ppc64el-netboot-compute (osimage)
ubuntu14.04-ppc64el-statelite-compute (osimage)

And in order to boot from network, you need to download the mini.iso from "http://ports.ubuntu.com/ubuntu-ports/dists/trusty/main/installer-ppc64el/current/images/netboot/",(pls use http://ports.ubuntu.com/ubuntu-ports/dists/utopic/main/installer-ppc64el/current/images/netboot/mini.iso for ubuntu 14.10) then mount the mini.iso to a tmp directory:

  mkdir /tmp/iso
  mount -o loop mini.iso /tmp/iso
  ls /tmp/iso/install
  initrd.gz  vmlinux

Then, copy the file /tmp/iso/install/initrd.gz to /install/<ubuntu-version>/ppc64el/install/netboot.</ubuntu-version>

  mkdir -p  /install/<ubuntu-version>/ppc64el/install/netboot
  cp  /tmp/iso/install/initrd.gz  /install/<ubuntu-version>/ppc64/installel/netboot

Define the powerNV object

mkdef n1 groups=all,kvm cons=ipmi mgt=ipmi
chdef n1 bmc=<bmc ip> bmcpassword=<bmc password>
chdef n1 mac=<xx:xx:xx:xx:xx:xx> installnic=mac promarynic=mac
chdef n1 tftpserver=<ip_of_xCAT_MN> conserver=<ip_of_xCAT_MN> nfsserver=<ip_of_xCAT_MN>

Setup dns, conserver and DHCP

  chdef n1 ip=x.x.x.x
  makedns –n

  makeconservercf

  makedhcp -n
  makedhcp -a

Prepare the petitboot and dhcpd configurations

nodeset n1 osimage=ubuntu14.04-ppc64el-install-compute
n1: install ubuntu14.04-ppc64el-compute

Configure node boot from network

rsetboot n1 net

Reboot the node to start the provisioning process

rpower n1 on/reset
MongoDB Logo MongoDB