extreme CSM
tools
Author: Bruce Potter, Document Version 0.8, 2/18/05
Introduction
The xCSM package is a set of additional tools for CSM to give
you even more capabilities to manage your cluster with.
There is not official support for these tools, but you can
certainly post your problems, questions, and suggestions to the CSM
mailing list and we will do our best to address them.
(There is also a version of xCSM that comes with xCAT that makes use of some of the
xCAT utilities to provide additional features.)
What's New
(Note: the links above only work as links if you are browsing
this file from the /opt/xcsm directory.)
Commands - /opt/xcsm/bin
Using the -h option on at least some of the commands will give
you extended usage information.
- chnodeprop - change node properties. Properties are attributes that users have
added to the node definition. (For now, the properties are stored in the
UserComment attribute.) This is not needed from CSM 1.4.1 and above.
See the chnode -P option.
- csm2hosts - write the hostname and IP info in the
CSM DB to a file in the /etc/hosts format. This
includes network info in the UserComment attribute.
- csmmpcli - run some hardware control operations
against the hardware control point using the MPCLI
- dping - ping the specified
nodes in parallel and display the results. Also supports
pinging multiple interfaces and devices, and pinging
recursively (ping the nodes from each of the nodes).
- genhosts - create an /etc/hosts file for the
cluster
- gennodedef - create a CSM nodedef file based on
some general pieces of information
- getmacs - get MACs after the nodes are displaying
them to the console
- hawkname -
- healdhcp - add information to the DHCP config file
to make the install servers DHCP and TFTP servers for
scalability
- hosts2csm - put the hostname and IP info from the
/etc/hosts file into the CSM DB. This script
assumes the convention of
putting the adapter name after the node name for all the secondary adapter hostnames
(for example "n001-eth1.site.com" would be
the hostname for the eth1 adapter of node 1.) The
secondary adapter info is stored in the UserComment
attribute for now.
- lsboot - list the installation attributes of the
specified nodes
- lscons - list the console attributes of the
specified nodes
- lsip - list the IP address of the specified nodes
- lsisvr - list the install server for the specified
node
- lsmac - list the MAC address of the specified
nodes
- lsmode - list the CSM node mode of the specified
nodes
- lsnodeprop - list node properties. This is not needed from CSM 1.4.1
and above. See the lsnode -P option.
- lsos - list the installation attributes of the
specified nodes
- lspos - calculate the position of nodes in racks
based on IP address
- lspower - list the hardware control
attributes of the specified nodes
- mkleafms - turns a node into a management server
- mkpxeimage - create an initrd boot image
- nodeset - configure a node to boot in a specific
way
- pdate - display the date on each node
- rbeacon - light up the front panel of the
specified nodes
- reventlog-ipmi - manage BMC logs
- rinv-ipmi - return BMC inventory info
- rmnodeprop - remove node properties
- rpower.dummy - rpower method that does nothing to
disable CSM from doing any power operations
- rpower-ipmi - power operations via the BMC
- rvitals-ipmi - return BMC environmental info
- sethwctrlnodeidattr - set the HWControlNodeId
attribute of the specified nodes to be the short version
of the node's hostname
Blade Utilities - /opt/xcsm/blade
- csm-blade-mini-howto -
- CSMBLADE.README -
- dhcpjs20 -
- mkzimage -
- mpname -
- pfwchk -
- rbootseq -
- rcons -
- rconscmd -
- rconsole -
- rvid -
These are commands that can be used to create and manage
database tables. The tables are stored in the RSCT
registry, using the RSCT DBI driver that is supplied with
CSM/RSCT. Note that these commands are just for convenience
for interactive use. To manipulate the tables
programmatically, use the DBI interface directly, since that is
more standard and will be faster. You can use these
commands below as examples for writing your own DBI scripts.
The DBI interface can also be used to access the RMC
classes in a standard way. See lscsmtab as an example.
Run any of these commands with the -h option to get
usage help.
- mkcsmtab - create a new table.
- mkcsmrow - add a row to an existing table
- lscsmtab - list available tables, or list rows
from an existing table. Note: You must apply the RSCT.pm.patch
below for the -l option to work.
- chcsmrow - change rows in a table.
- rmcsmrow - delete rows in a table.
- rmcsmtab - delete a table.
- generic_power - an example user-defined power method that uses WOL to wake up machines.
To use set node attributes: PowerMethod=generic HWControlPoint=<mgmtsvr>
HWControlNodeId=<nodeMAC> . This power method also does some
other rpower and lshwstat actions as examples.
Console Methods - /opt/xcsm/consolemethods
- e325sol_console - CSM console method for
serial-over-lan on e325
MAC Methods - /opt/xcsm/macmethods
- mmsnmp_mac - MAC collection method for the BladeCenter management module that uses SNMP to get MACs for adapters on the mother board and on the 1st daughter card. To use it, make sure the HW ctrl node attributes are set correctly and then run: getadapters -w -n <nodelist> -m mmsnmp
HPC Set Up Utilities - /opt/xcsm/hpc
- Scripts to help build mpich, pbs, and torque - these haven't been
updated in a while
Node Installation Customization Scripts - /opt/xcsm/install
In the bullets below "###" represents a 3 digit
number that is put at the beginning of the script to control what
order they are run in.
- Pre-reboot scripts:
- ###CSM_syslog - configure syslog on the
nodes to forward messages to the management
server
- ###CSM_setupnis - configure NIS on the
node
- ###Mount - mount /tftpboot from the
management server
- Post-reboot scripts:
- ###CSM_IBMCompiliers - install the IBM
compilers on the node
- ###_configOtherNICs - set up other network
adapters based on the info in the UserComment
attributes. Use this in conjunction with hosts2csm
and chnodeprop, and the ServerUtils.pm
patch.
- ###_makeMS - automatically make this node
into a first-line management server. See HierarchicalMS.pdf
for more info.
OS Cloning - /opt/xcsm/clone
- Install a System Imager clone using CSM - this hasn't been updated in a
while
- MultipleCondition-Response - combine multiple
conditions together to form a complex condition
(including conditions that have to occur for a certain
length of time). See the comments in the beginning
of this script for directions on how to use it.
- ODCsamples - examples of conditions and responses from IBM's On Demand Center.
- HPSsamples - more examples of conditions and responses done for a pSeries customer.
You can use the CSM TEC adapter to forward cluster events to
the Tivoli Enterprise Console. See the README in the tivoli
sub-directory for instructions on how to set it up.
CSM Patches - /opt/xcsm/csm-patches
Most of the code in xCSM is structured so that it can be run
on top of the standard CSM release, without changing any CSM
files. But in some limited cases, this is not possible.
The files in this sub-directory are fixes or enhancements
to CSM files. Using this directory as the root, the files are
given the full path name of the corresponding CSM file. In
some cases the file is a full replacement, in other cases it is a
patch file. In both cases, proceed with caution:
always keep a backup of the file being replaced/patched,
and restore the original files before upgrading CSM.
- /opt/csm/install/pkgdefs/Linux-SLES9-ppc64.pm -
defines all the SLES 9 CDs so copycds will copy all RPMs.
This is necessary to do a full installation of the
node.
- /opt/csm/pm/ServerUtils.pm.patch - copies the
UserComment and InstallAdapterName attributes to the
configinfo file during node installation so they are
available to the ###_configOtherNICs script. This
patch applies to 1.4.0.x versions of CSM. Apply this
patch using: cd /opt/csm/pm; patch -b
</opt/xcsm/csm-patches/opt/csm/pm/ServerUtils.pm.patch
- /usr/sbin/rsct/pm/DBD/RSCT.pm.patch - fixes a bug
with lscsmtab. This patch applies to the 1.3.3 version of
CSM/RSCT and above. Apply this patch using: cd /usr/sbin/rsct/pm/DBD;
patch -b </opt/xcsm/csm-patches/usr/sbin/rsct/pm/DBD/RSCT.pm.patch
- /opt/csm/csmbin/nodestatus.client.patch
- disables install status messages from the node back to
the management server to increase install performance in
large clusters. This patch applies to 1.4.0.x versions of
CSM. Apply this patch using: cd /opt/csm/csmbin;
patch -b
</opt/xcsm/csm-patches/opt/csm/csmbin/nodestatus.client.patch;
cp nodestatus.client /csminstall/csm/nodestatus.client
xCSM now provides two xCAT commands designed to work in a CSM
cluster: addclusteruser and pushuser. These commands give you the
ability to quickly add users throughout the cluster and to setup
OpenSSH keys to allow users unprompted ssh access from any node in the
cluster to any other node in the cluster. To use addclusteruser and
pushuser do the following:
- Install xCAT on your management
server
(the requirement for xCAT
will be removed in a future release).
- Run the xCAT provided script: /opt/xcat/csm/sbin/csm2xcat
to setup the xCAT tab files using your CSM data as a starting point.
- Set nisdomain to NA in /opt/xcat/etc/site.tab
- Export your path to include /opt/xcsm/xcat/sbin
- Run addclusteruser. For example to add a user called clusterguest
run: addclusteruser -n clusterguest
- Then either use CFM to push out /etc/passwd and /etc/group
to the nodes, or run pushuser
[noderange] clusterguest
- Finally, run cfmupdatenode to push out the user's home directory
(which includes the needed OpenSSH keys): cfmupdatenode -n
[noderange]. Please
note you need CSM version 1.4.0.13 or later for this CFM functionality.
If you are using an earlier version of CSM, push the user's home
directory to the nodes via NFS or dcp.
References