For additional information about Platform LSF Version 7 Update 6, visit the Platform Computing Web site:
http://www.platform.com/Products/platform-lsf/features-benefits
To achieve the highest degree of performance and scalability, we strongly recommend that you use a powerful master host.
There is no minimum CPU requirement. For the platforms LSF is supported on, any host with at least 1 GB of physical memory can run LSF as master host. Swap space is normally configured as twice the physical memory. LSF daemons use about 20 MB of memory when no jobs are running. Active jobs consume most of the memory LSF requires.
DO NOT use the UNIX and Linux upgrade steps to migrate an existing LSF 7 Update 1 cluster to LSF Version 7 Update 6. Follow the manual steps in the document Migrating to Platform LSF Version 7 Update 6 on UNIX and Linux to migrate an existing LSF 7 Update 1 cluster to LSF Version 7 Update 6 on UNIX and Linux.
Visit the Platform Computing Web site for information about supported operating systems and system requirements for Platform LSF:
http://www.platform.com/Products/platform-lsf/technical-information
The HPC Portal is now provided as an alternative to the Platform Management Console on Linux 2.6 64-bit installations. The UNIX upgrade and migrations guides include instructions on how to install the HPC Portal, even if you have been using the Platform Management Console.
For all other platforms, you can install the Platform Management Console using the LSF Version 7 Update 5 PMC package:
If libraries need to be re-linked, applications need to be rebuilt if they use APIs that have changed in LSF Version 7 Update 6.
To take full advantage of new LSF Version 7 features, you should recompile your existing LSF applications with LSF Version 7.
Multi-phase resource reservations can be made for job memory requirements. rusage strings set in the queue-level or application-level RES_REQ parameter (lsb.queues or lsb.applications) or in the bsub -R option can contain multiple durations with multiple memory and decay requirements. The resource requirements are merged if they occur at more than one level, and must satisfy any queue limits set by RESRSV_LIMIT in lsb.queues.
Single-phase memory requirements and multi-phase memory requirements can merge together, and one or both can appear as alternative -R options for a job. Multi-phase memory requirements cannot be used with multiple -R strings.
Resource limits for consumable resources appearing in the rusage section can now be set at the queue level. These limits include maximum values, and optionally minimum values, which set the range of allowable resource requirements for job submissions to the queue.
Resource reservation limits are set in the parameter RESRSV_LIMIT in lsb.queues. Queue-level RES_REQ rusage sections must be within the range set by RESRSV_LIMIT or the queue-level RES_REQ is ignored. Merged job-level and application-level rusage sections must be within the range set by RESRSV_LIMIT or the job is rejected.
Scheduling of MultiCluster jobs under the job forwarding model can be configured to consider remote queue attributes in addition to remote resources. Remote queue attributes are collected every MC_PLUGIN_UPDATE_INTERVAL (lsb.params).
The new scheduler configuration settings allow several options or combinations thereof through the parameter MC_PLUGIN_SCHEDULE_ENHANCE in lsb.params. Jobs forwarded under the job-forwarding model are then scheduled depending on:
Some limitations apply to MultiCluster job forwarding scheduler:
When an advance reservation is active on a remote cluster, slots within the advance reservation are excluded from the number of available slots.
The submission cluster assumes all hosts in a hostgroup have the same boolean resources, as is required in hostgroup configuration.
The submission cluster assumes all slots within a hostgroup are of the same host type.
Users can now submit jobs using SSH X11 forwarding, which uses the SSH client they have installed. Administrators can configure lsf.conf with a value for LSB_SSH_XFORWARD_CMD.
Applies only to UNIX submission and execution hosts. This feature can be combined with bsub -I for an interactive session. The feature cannot be used with job arrays, chunk jobs, or user account mapping. You cannot select or modify SSH X11 forwarding options in esubs. This feature cannot be combined with -K, -IX, or -r options of bsub.
Users can now specify a job description of up to 4094 characters upon job submission using the new command option bsub -Jd, the wrapper command tssub -Jd, or the new esub parameter LSB_SUB_JOB_DESCRIPTION.
The job description can be modified for a job, job array, or job array element using bmod -Jd. The job description appears in the output from bjobs -l and bhist -l, and in the lsb.events and lsb.acct log files. Searches for specific job descriptions can be made using bjobs -Jd or bhist -Jd, and can include the wildcard character *.
Job exception notifications can now be extended to include more information by setting EXTEND_JOB_EXCEPTION_NOTIFY=Y in lsb.params. Full information now includes JOB_ID, RUN_TIME, IDLE_FACTOR, USER, QUEUE, EXEC_HOST, and JOB_NAME.
You can also set the format to truncate or not in the script LSF_SERVERDIR/eadmin by setting JOB_EXCEPTION_EMAIL_FORMAT to full or fixed (truncated).
The following configuration parameters and environment variables are new or changed for LSF Version 7 Update 6:
MC_PLUGIN_SCHEDULE_ENHANCE: MultiCluster job-forwarding model only. When defined as a valid value, the submission cluster scheduler considers specified remote queue information in addition to remote resource availability as for MC_PLUGIN_REMOTE_RESOURCE=Y in lsf.conf.
MC_PLUGIN_UPDATE_INTERVAL: Multicluster job-forwarding model only. Interval between remote queue information updates to the submission cluster. Disabled when set to zero.
EXTEND_JOB_EXCEPTION_NOTIFY: Job exception notifications can now be extended to include more information. Full information now includes JOB_ID, IDLE_FACTOR, RUN_TIME, USER, QUEUE, EXEC_HOST, and JOB_NAME.
MC_PLUGIN_REMOTE_RESOURCE: Multicluster job-forwarding model only. Now enabled when MC_PLUGIN_SCHEDULE_ENHANCE is defined as a valid value, as well as when MC_PLUGIN_REMOTE_RESOURCE=Y.
LSB_SSH_XFORWARD_CMD: For SSH X11 forwarding jobs, specifies the SSH command to run when a user runs bsub -XF. Accepts the full PATH and options of a regular SSH command.
The option -l for exited jobs now includes the detailed termination reason (if available) following the signal or exit code.
The option -l now includes information about SSH X11 forwarding jobs, and the job description, if applicable.
The new option -Jd retrieves jobs based on the specified job description.
The bmod command has new options -Jd which accepts a job description of up to 4094 characters, and -Jdn which removes the job description.
bmod -R rusage values for pending jobs now must satisfy the limits set by RESRSV_LIMIT in lsb.queues. bmod -R rusage values for running jobs now must satisfy the maximum limits set by RESRSV_LIMIT, but can be lower than the minimum limits.
bmod -R can contain multi-phase memory rusage values for pending jobs. bmod -R multi-phase memory rusage values for running jobs do not take effect until after the current reservation phase.
To switch a pending job to a new queue, the job’s rusage values now must satisfy the queues limits set by RESRSV_LIMIT in lsb.queues.
To switch a running job to a new queue, the job’s rusage values now must satisfy the maximum limits set by RESRSV_LIMIT, but can be lower than the minimum limits.
Jobs with multi-phase memory requirements can be switched to other queues at any time, without waiting for the current phase to finish.
The LSF 6.x passwd.lsfuser password file is not compatible with LSF 7. In LSF 6.x, if a domain name is defined with LSF_USER_DOMAIN in lsf.conf, LSF only saves the user name to the password entry in the passwd.lsfuser password file. In LSF 7, the user name part of the password entry in the passwd.lsfuser file is a fully qualified user name (domain_name\user_name,), even if LSF_USER_DOMAIN is defined in lsf.conf.
Workaround: If your cluster defines LSF_USER_DOMAIN in lsf.conf, you must upgrade the entire 6.x cluster to LSF 7, and have all users run lspasswd to reenter their password.
Without this workaround, LSF 7 daemons cannot find the 6.x password entry and 6.x daemons cannot see the password saved on LSF 7 servers.
This problem affects all LSF versions before Version 7, LSF 6.0, 6.1, and 6.2.
Backfill jobs can overlap exclusive compute unit reservations. Free slots within an exclusive compute unit reservation appear available when using bslots to schedule backfill jobs. Job slots used by the exclusive compute unit job do not appear available beyond the reservation start time.
When specifying a domain name in any LSF configuration file, use all uppercase characters. For example: LSF/lsfadmin instead of lsf/lsfadmin. Configuration settings will not be applied if the domain is in lowercase characters.
When a job has been suspended, it may be scheduled to run before a pending job with a higher priority. The higher priority pending job runs as soon as the next scheduling period starts.
Jobs submitted with CPUSET_TYPE=none are still considered CPUSET jobs, and do not support compound resource requirements. For example, the following job submission will not run:
When using ProPacks in a cluster with mixed host types, you must also specify "same[type]" in the resource requirement string or use %a to run applications on appropriate host types. Only setting the ProPack version number is not sufficient to identify the possible host types a job can run on.
If there are no PSET hosts in your cluster, the PSET plug in is not supported and should not be configured in lsb.modules.
When compiling an application with a LSF Version 7 Update 6 library, specify the option -ldl.
The SGI-MPI integration with LSF PAM has been enabled on Linux, but LSF_PAM_USE_ASH is not supported on linux2.6-glibc2.3-x86_64.
A Session Scheduler job suspended with bstop enters USSUP state and the job cannot be killed with bkill. The out-of-box TERMINATE_CONTROL=SIGINT configuration in Session Scheduler causes only SIGINT to be sent to the job from bkill. To be terminated, the job must receive the required SIGCONT, SIGINT, SIGTERM, and SIGKILL signals. You must run bresume to cause the job to receive the correct bkill signals.
When installing License Scheduler standalone, the installer removes EGO environment variables from cshrc.lsf and profile.lsf. Specify a different LSF_TOP from the LSF installation to install standalone License Scheduler.
In the resource plan, if you specify reclamation with a grace period, the grace period is ignored by LSF. All resources are reclaimed immediately.
LSF admin cannot start the PMC in EGO-decoupled mode. Since the PMC has already been started by root, the log files are owned by root. When the PMC is restarted by the LSF cluster administrator, admin does not own the existing log files resulting in the JAVA (tomcat) process stalling.
Integrating LDAP with LSF has some additional requirements:
If you did not set DERBY_DB_HOST in install.config, you can still enable the Derby database host after installation. See procedure that follows.
Access to the Platform FTP site is controlled by login name and password. If you cannot access the distribution files for download, send email to support@platform.com.
You must provide your Customer Support Number and register a user name and password on my.platform.com to download LSF.
To register at my.platform.com, click New User? and complete the registration form. If you do not know your Customer Support Number or cannot log in to my.platform.com, send email to support@platform.com.
Before installing Platform LSF Version 7, you must get a demo license key.
Contact license@platform.com to get a demo license.
Put the demo license file license.dat in the same directory where you downloaded the Platform LSF product distribution tar files.
Use the lsfinstall installation program to install a new LSF Version 7 cluster, or upgrade from and earlier LSF version.
See Installing Platform LSF on UNIX and Linux for new cluster installation steps.
See the Platform LSF Command Reference for detailed information about lsfinstall and its options.
DO NOT use the UNIX and Linux upgrade steps to migrate an existing LSF 7 cluster or LSF Version 7 Update 1 cluster to LSF Version 7 Update 6. Follow the manual steps in the document Migrating to Platform LSF Version 7 Update 6 on UNIX and Linux to migrate an existing LSF 7 Update 1 cluster to LSF Version 7 Update 6 on UNIX and Linux.
Platform LSF on Windows 2000, Windows 2003, and Windows XP is distributed in the following packages:
See Installing Platform LSF on Windows for new cluster installation steps.
To migrate your existing LSF Version 7 cluster on Windows to LSF Version 7 Update 6, you must follow the manual steps in the document Migrating Platform LSF Version 7 to Update 6 on Windows (lsf_migrate_windows_to_update6.pdf).
See Using Platform License Scheduler for installation and configuration steps.
Platform License Scheduler Version 7 Update 5 is the current release, compatible with Platform LSF Version 7 Update 6. The manual Using Platform License Scheduler is incorrectly labeled as Update 6 instead of Update 5.
Information about Platform LSF Version 7 is available in the LSF area of the Platform FTP site (ftp.platform.com/distrib/7.0/).
The latest information about all supported releases of Platform LSF is available on the Platform Web site at www.platform.com.
If you have problems accessing the Platform web site or the Platform FTP site, send email to support@platform.com.
my.platform.com—Your one-stop-shop for information, forums, e-support, documentation and release information. my.platform.com provides a single source of information and access to new products and releases from Platform Computing.
On the Platform LSF Family product page of my.platform.com, you can download software, patches, updates and documentation. See what’s new in Platform LSF Version 7, check the system requirements for Platform LSF, or browse and search the latest documentation updates through the Platform LSF Knowledge Center.
The Platform LSF Knowledge Center is your entry point for all LSF documentation. If you have installed the Platform Management Console, access and search the Platform LSF documentation through the link to the Platform Knowledge Center.
Get the latest LSF documentation from my.platform.com. Extract the LSF documentation distribution file to the directory LSF_TOP/docs/lsf.
The Platform EGO Knowledge Center is your entry point for Platform EGO documentation. It is installed when you install LSF. To access and search the EGO documentation, browse the file LSF_TOP/docs/ego/1.2.3/index.html.
If you have installed the Platform Management Console, access the EGO documentation through the link to the Platform Knowledge Center.
Platform’s Professional Services training courses can help you gain the skills necessary to effectively install, configure and manage your Platform products. Courses are available for both new and experienced users and administrators at our corporate headquarters and Platform locations worldwide.
Customized on-site course delivery is also available.
Find out more about Platform Training at www.platform.com/services/training, or contact Training@platform.com for details.
Contact Platform Computing or your LSF vendor for technical support. Use one of the following to contact Platform technical support:
When contacting Platform, please include the full name of your company.
See the Platform Web site at www.platform.com/company/contact-us for other contact information.
To get periodic patch update information, critical bug notification, and general support notification from Platform Support, contact supportnotice‑request@platform.com with the subject line containing the word "subscribe".
To get security related issue notification from Platform Support, contact securenotice‑request@platform.com with the subject line containing the word "subscribe".
© 1994-2009, Platform Computing Inc.
Although the information in this document has been carefully reviewed, Platform Computing Inc. (“Platform”) does not warrant it to be free of errors or omissions. Platform reserves the right to make corrections, updates, revisions or changes to the information in this document.
UNLESS OTHERWISE EXPRESSLY STATED BY PLATFORM, THE PROGRAM DESCRIBED IN THIS DOCUMENT IS PROVIDED “AS IS” AND WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. IN NO EVENT WILL PLATFORM COMPUTING BE LIABLE TO ANYONE FOR SPECIAL, COLLATERAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, INCLUDING WITHOUT LIMITATION ANY LOST PROFITS, DATA, OR SAVINGS, ARISING OUT OF THE USE OF OR INABILITY TO USE THIS PROGRAM.
This document is protected by copyright and you may not redistribute or translate it into another language, in part or in whole.
You may only redistribute this document internally within your organization (for example, on an intranet) provided that you continue to check the Platform Web site for updates and update your version of the documentation. You may not make it available to your organization over the Internet.
LSF is a registered trademark of Platform Computing Corporation in the United States and in other jurisdictions.
POWERING HIGH PERFORMANCE, PLATFORM COMPUTING, PLATFORM SYMPHONY, PLATFORM JOBSCHEDULER, and the PLATFORM and PLATFORM LSF logos are trademarks of Platform Computing Corporation in the United States and in other jurisdictions.
UNIX is a registered trademark of The Open Group in the United States and in other jurisdictions.
Linux is the registered trademark of Linus Torvalds in the U.S. and other countries.
Microsoft is either a registered trademark or a trademark of Microsoft Corporation in the United States and/or other countries.
Windows is a registered trademark of Microsoft Corporation in the United States and other countries.
Globetrotter and FLEXnet are registered trademarks or trademarks of Acresso Software Corporation in the United States of America and/or other countries.
Oracle is a registered trademark of Oracle Corporation and/or its affiliates.
Intel, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries.
Other products or services mentioned in this document are identified by the trademarks or service marks of their respective owners.