This release is only supported for the IBM FlashSystem 900, Machine Types and Models (MTM) 9840-AE2 and 9843-AE2
IBM FlashSystem 900 product resources guide users through the various features and components of the storage system, including usage and troubleshooting guides. To read about this storage system and learn how to use or troubleshoot, see IBM Knowledge Center for IBM FlashSystem 900 or visit the IBM Redbooks® website for the IBM FlashSystem 900 Product Guide.
The following explains the bug severity ranking used for key fixes and in the Release history:
| Severity | Description | |||
|---|---|---|---|---|
| S1 | Recommended upgrade for all users as soon as possible. | |||
| S2 | Recommended upgrade for all users at the next scheduled maintenance window | |||
| S3 | Recommended upgrade at the next scheduled maintenance window only for users experiencing the issue. All others may consider this to be an S4. | |||
| S4 | Upgrade at the next scheduled maintenance window. May be performed at the discretion of the user if the issue is having a negative impact. | |||
| S5 | Upgrade is not necessary. This would include a mostly cosmetic or minor annoyance fix. |
The current release is Program Temporary Fix (PTF) for IBM FlashSystem 900 customers and includes a minor fix and multiple security remediations.
After initial configuration of the hardware is complete, IBM strongly recommends that you make sure that your IBM FlashSystem firmware is up-to-date. Visit IBM Fix Central using the link below to see if any updates are available for your system.
The following fix is made available with release 1.3.0.10. To view fixes from earlier releases, see the Release history. Available firmware releases are listed on IBM Fix Central. For issue severity definitions, see the Bug severity legend.
S5
Multiple vulnerabilities are remediated with this release. Use the associated links below to find out more information about each vulnerability.
More information is available on these vulnerabilities through the following security bulletins:
The following information lists the features that come with the 1.3 release of IBM FlashSystem 900 product.
The following are features of all 1.3 releases and are therefore included in the latest release:
The following are features of all 1.2 releases and are therefore included in the latest release:
To stay up-to-date on current known issues, workarounds, downloads, and other documentation from support, please ensure that you have subscribed to My Notifications.
| Protocol | Description |
|---|---|
| SCSI-SAM-3 | SCSI Architecture Model (v3) |
| SCSI-SPC-3 | SCSI Primary Commands (V3) |
| SCSI-SBC-2 | SCSI Block Commands (V2) |
| SCSI-FCP-3 | Fibre Channel (FC) Protocol (V3) |
| SCSI-SRP | SCSI RDMA Protocol |
| FC-PH-3 | FC Physical and Signaling Interface (V3) |
| FC-AL-3 | FC Arbitrated Loop (V2) |
| IBTA-1.2 | InfiniBand (IB) Trade Association Architecture Specification (V1.2) |
Note: To test or demonstrate concurrent maintenance on canisters and batteries, use this featured document, which describes the recommended process for concurrent maintenance.
The following sections include a list of all fixes and improvements for previous FlashSystem 900 releases.
The following issues were fixed in release 1.3.0.9:
FLASH-22859 - Remediate vulnerabilities in Java™ CPU (CVE-2016-5546, CVE-2016-5548, CVE-2016-5549, CVE-2016-5547, and CVE-2016-2183).
FLASH-23390 - Remediate a vulnerability in Apache Tomcat (CVE-2017-5647).
FLASH-25364 - Staggered battery end of life is needed to ensure that both system batteries will not reach end of life simultaneously. (S1)
FLASH-25519 - Validation should be performed on rebuild/xverify to avoid out of bound addresses. (S2)
FLASH-23501 - Shutting the system down with the stopsystem -force command could cause a warmstart if any flash modules are in the failed state or a RAID controller is in the service state. (S3)
The following issue was fixed in 1.3.0.8:
FLASH-22860 - Remediate a vulnerability in Apache Struts Jakarta Multi-Part Parser Code Execution (CVE-2017-5638). More information is available on the IBM PSIRT Blog.
The following issues were fixed in release 1.3.0.7:
FLASH-20584, 20733 - Remediate vulnerabilities in Apache Tomcat (CVE-2016-3092, CVE-2016-5387, CVE-2016-5388, CVE-2016-5385, CVE-2016-5386, CVE-2016-1000110, CVE-2016-1000105, and CVE-2016-1000111).
FLASH-20585 - Remediate vulnerabilities in Apache Struts (CVE-2016-4430, CVE-2016-4431, CVE-2016-4433, CVE-2016-4436, CVE-2016-4438, and CVE-2016-4465).
FLASH-20737 - Remediate vulnerabilities in OpenSSH (CVE-2015-5352, CVE-2015-6563, and CVE-2015-6564).
FLASH-22261 - Continuous and repeated loss of access of AC power on a PSU may, in rare cases, result in the report of a critical temperature fault. Using the provided cable secure mechanisms is highly recommended in preventing this issue. (S1)
FLASH-21881 - HIPER (Highly Pervasive): In rare cases, when both a rebuild read fails and a data reconstruction fails, a SCSI read should fail. (S1)
FLASH-21210 - Nodes have the potential to warm start after initializing. (S2)
FLASH-22264 - Cluster goes down due to a dead management PCIe link. (S2)
FLASH-21574 - The CLI allows the input of carriage return characters into certain fields after cluster creation resulting in invalid cluster VPD. (S3)
FLASH-22262 - Adjusted InfiniBand (IB) ASIC timeouts to prevent erroneous system data stalls if the IB ASIC becomes unresponsive. (S3)
FLASH-22328 - The DMP with error code 1114 for battery fault does not wait for a low charge battery FRU to charge. (S3)
FLASH-21975 - CLI and GUI don't get updated with the correct flash module firmware version after flash module replacement. (S4)
FLASH-22263 - The lsnodevpd command displays an incorrect FRU Number. (S4)
FLASH-22333 - The GUI system image does not always show USB drives installed when ports are active. (S4)
FLASH-22350 - Some text in some GUI fix procedures is not translated. (S4)
FLASH-20587 - The CLI command lsdrive drive_id output does not reflect an updated 'firmware_level' field after a system firmware upgrade. (S4)
The following issues were fixed in release 1.3.0.6:
FLASH-16577 - Remediate vulnerabilities in Apache Tomcat (CVE-2015-5345, (CVE-2015-5346, CVE-2015-5351, CVE-2016-0706, CVE-2015-0714, CVE-2016-0763, (CVE-2015-5174).
FLASH-17238 - Remediate vulnerabilities in Apache Struts (CVE-2016-0785, CVE-2016-2162).
FLASH-17242, 18087 - Remediate vulnerabilities in OpenSSL (CVE-2016-0797, CVE-2016-0705, CVE-2016-2107).
FLASH-17956 - Remediate a vulnerability in NSS (CVE-2016-1978).
Remediate a vulnerability in Java (CVE-2016-0475).
FLASH-18049 - Improve internal Flash checking to prevent access loss. (S1)
FLASH-18132 - Issues result when the same call home manager processes run simultaneously. (S1)
FLASH-18134 - PSoC issues eventually lead to both canisters going into service state. (S1)
FLASH-18067 - Internal error handling causes loss of access. (S1)
FLASH-17914 - Degraded components are still used in the thermal algorithm. (S2)
FLASH-18053 - Upgrade failed with the message 'Unable to communicate with Systemmgr.' (S2)
FLASH-18062 - Fix interface error reporting. (S2)
FLASH-15900 - A rare scenario finds sequential fail logic to be too aggressive.(S3)
FLASH-17916 - Internode communication issue causes CCU to stall. (S3)
FLASH-19024 - Rare UTDE packet internal error causes warmstart and CCU stall. (S3)
FLASH-17011 - Missing internal system notification causes CCU to hang. (S3)
FLASH-18406 - Export to CSV does not work on the GUI performance page. (S4)
FLASH-16724 - Update system page reports the current software version is not supported. (S5)
The following issues were fixed in release 1.3.0.5:
FLASH-15905 - Marking 'Array Mdisk is not protected by sufficient spares' event as fixed should not work. (S3)
FLASH-14603 - Remediate a vulnerability in Java (CVE-2015-4872).
FLASH-15060 - Remediate multiple vulnerabilities related to Network Security services (NSS) (CVE-2015-7181, CVE-2015-7182, CVE-2015-7183).
FLASH-15835 - Remediate a vulnerability in OpenSSL (CVE-2015-3194).
FLASH-13795 - Remediate a vulnerability in Apache Struts (CVE-2015-5209).
FLASH-13574 - Remediate a vulnerability in cross-site request forgery (CSRF) (CVE-2015-7446).
The following issues were fixed in release 1.3.0.4:
FLASH-14845, 14844 - Remediate vulnerability in PAM or Pluggable Authentication Module (CVE-2015-3238).
FLASH-13574 - Remediate Cross-Site Request Forgery or CSRF vulnerability.
FLASH-13535 - Remediate vulnerability in SSL/TLS.
FLASH-13369, 13368 - Remediate vulnerability in nss-softokn (CVE-2015-2730).
FLASH-13706 - HIPER (highly pervasive): Potential undetected data corruption may occur due to a low probability race condition. The race condition has been observed on a system with a specific workload that is doing 1 to 2 GB/s of read operations with 250 MB/s of write operations. The write operations were less than 4K in size. (S1)
FLASH-15207 - Repeated interface panics due to a bad interface cable can cause unnecessary component failures. (S2)
FLASH-14793 - A flash module can become unresponsive when array certify is running while hardware errors are being found. (S2)
FLASH-15489 - The nodes warmstart after being powered on due to an error in call home. (S3)
FLASH-13411 - Use of the command svcinfo lshostsubvolumemap causes the CLI to go down temporarily. (S3)
FLASH-13263 - VPD mismatch due to 8 Gb to 16 Gb conversion causes node asserts on upgrades to 1.3 firmware levels. (S3)
FLASH-15372 - Stalled upgrade reports an erroneous 'Failed to upgrade' error. (S3)
FLASH-15143 - The maximum number of host port objects decreases by one going from 1.2 to 1.3 code. (S3)
FLASH-14685 - Code upgrade stalls with internal error. (S3)
FLASH-12079 - Node timeout results in flash failure. (S3)
FLASH-12463 - Gateway to node CRC errors result in flash failure. (S3)
FLASH-11546 - Flash card failures occurred due to unexpected power off. (S3)
FLASH-13325 - Mitigation for flash module encryptor error. (S3)
FLASH-15254 - Improve signal integrity between canisters. (S3)
FLASH-15315 - The telephone number field length in the GUI does not match the requirements of SVC products. (S5)
FLASH-13484 - CLI help documentation for the ping command does not include new parameter. (S5)
The following issues were fixed in release 1.3.0.3:
FLASH-10534 - HIPER (highly pervasive): Potential undetected data corruption may occur when using Write Same commands. Direct attached FlashSystem 840 and 900 products can overwrite a buffer when Write Same commands are executing with heavy Input/Output usage. This is considered a highly pervasive problem involving firmware 1.2.x.x and 1.3.0.2. (S1)
FLASH-12982 - Issuing rmvdisk -force to remove a VDisk causes a node failover when host mappings exist. (S2)
FLASH-12837 - A node assert takes place when trying to add a node. (S2)
FLASH-12500 - Remediate vulnerabilities in Java (CVE-2015-2613, CVE-2015-2601, CVE-2015-2625, and CVE-2015-1931). (S2)
FLASH-11827 - When trying to install some packages, an 'Error in verifying the signature of the update package' message is produced. (S3)
The following issues were fixed in release 1.3.0.2:
FLASH-10119, 13429 - HIPER (highly pervasive): Potential undetected data corruption may occur from interface error. FlashSystem 840 and 900 products can write inconsistent data to a host. This is considered a highly pervasive problem involving firmware versions 1.1.x.x and 1.2.x.x. (S1)
FLASH-11635 - Remediate vulnerabilities in OpenSSL (CVE-2015-1788, CVE-2015-1789, CVE-2015-1791, and CVE-2015-3216). (S1)
FLASH-12653 - Vulnerability in SSL/TLS discovered on REST API port (CVE-2015-2808). (S1)
FLASH-12537 - Canister is marked as 'failed' because it came online before completing the upgrade. (S2)
FLASH-11481 - The node throws an assertion exception for exceeded temperature on an unused drive. (S2)
FLASH-11958 - Stats can fill the /dumps folder to capacity, which disables the node from booting. (S2)
FLASH-12689 - An unexpected canister powering off can, in some cases, cause loss of access to data due to interface failure. (S2)
FLASH-12492 - Not able to rekey if encryption was enabled after the initial array creation and a node failover has occurred on firmware version 1.1.3.x or 1.2.x.x. (S2)
FLASH-12271 - The RAID controller was falsely marked as 'failed' instead of a flash module in a particular double flash module failure scenario. (S2)
FLASH-12219 - An unexpected canister powering off can cause the other canister to warm start. (S2)
FLASH-12061 - The controller panics when no type is set on an unresponsive interface. (S2)
FLASH-9320 - Simultaneous double flash fails can result in the incorrect component being marked as 'failed.' (S2)
FLASH-11595 - Interface incorrectly fails during a sequential double flash module failure scenario. (S2)
FLASH-11567 - Spikes in Input/output latency occur due to encryption validation. (S3)
FLASH-12387 - GUI incorrectly uses the '-force' option to reboot or power off a canister or system. (S3)
FLASH-12331 - iSCSI Check Condition sense data is invalid. (S3)
FLASH-12299 - iSCSI 'Desired Data Length' incorrectly exceeds 'MaxBurstLength.' (S3)
FLASH-12298 - iSCSI packet with garbled parameters incorrectly causes port to go offline. (S3)
FLASH-12296 - iSCSI duplicate 'InitiatorName' key is not rejected at login as it should be. (S3)
FLASH-12236 - iSCSI target does not discard command with invalid CmdSN. (S3)
FLASH-10687 - Archive stats are wrong after a canister power off. (S3)
FLASH-10170 - 'Abort Task Set' incorrectly compares sequence numbers. (S3)
FLASH-12237, FLASH-11964, FLASH-11962, FLASH-11943 - iSCSI improvements made for path failures. (S3)
FLASH-12175 - Incorrect memory free error for canceled UNMAP commands. (S3)
FLASH-12174 - Interface does not UNMAP data in a particular scenario. (S3)
FLASH-12173 - Interface allows host to surpass the UNMAP limit on block descriptors. (S3)
FLASH-11944, FLASH-11943 - High traffic on FC and iSCSI systems can cause a single command to stall. (S3)
FLASH-11933 - Traffic on iSCSI can stall if something is put in a queue. (S3)
FLASH-10276 - Improvements needed for link speed for iSCSI. (S3)
FLASH-12227 - Unexpected event for drive failure and replacement should actually be a quorum error. (S3)
FLASH-11468 - GUI shows canister offline while CLI shows canisters online. (S3)
FLASH-11184 - 'Neighbor table overflow' spamming causes Ethernet connectivity issues. (S3)
FLASH-12283 - Stale interface logins are not removed on failure as expected. (S3)
FLASH-11337 - Fault LED comes on after drive replacement and the resetleds command is issued.
FLASH-11051 - GUI becomes unresponsive. (S3)
FLASH-10111 - A link error between the drive and RAID controller gets incorrectly propagated and incorrectly fails the RAID controller. (S3)
FLASH-11477 - Flash module reports the incorrect temperature on node timeout, which results in a false critical temperature failure. (S3)
FLASH-11874 - A quick canister reseat can lead to the canister reporting 'degraded.' (S4)
FLASH-11359 - A nonconcurrent upgrade fails due to an issue in the full system boot upgrade. (S4)
FLASH-9930 - The GUI should allow the user to cancel if upgrade is in 'prepared' state. (S4)
FLASH-9241 - The GUI does not report the correct output for the lsupdate command. (S4)
FLASH-10516 - Improve logging. (S4)
FLASH-9152 - The status LED is incorrectly lit when canister is off. (S4)
FLASH-12393 - Some system stats incorrectly continue to update. (S4)
FLASH-10157 - Improve management controller packet handling. (S4)
FLASH-6114 - The svcinfo lsnode CLI command incorrectly displays different port information than the lsportfc command. (S4)
FLASH-5869 - The 'fc_io_port_WWPN' field of the sainfo lsservicestatus command is inconsistent between protocols. (S4)
FLASH-4288 - The 'node_code_build' field of the sainfolsservicestatus command does not display the complete build number. (S4)
FLASH-9962 - Issuing lsdumps -prefix with an invalid directory causes a node failover. (S4)
FLASH-10377 - Make battery output improvements. (S5)
FLASH-12047 - Improve system manufacturing tests. (S5)
FLASH-10034 - Improve system logs. (S5)
FLASH-12392 - GUI snaps are missing files from the 'cimom' directory. (S5)
Use the following sections to perform firmware upgrades for your systems to the current release.
Warning: Please read all the instructions below before upgrading.
If you are upgrading to this release and your system is healthy, you can perform a Concurrent Code Upgrade (CCU). A CCU is a non-disruptive upgrade and is the preferred upgrade method. For general instructions on performing upgrades, refer to the FlashSystem Knowledge Center.
The following upgrade paths are supported for this release. Note that customers with AE3 hardware have fewer supported upgrade paths than customers with AE2 hardware.
| From | From/To | To | ||
|---|---|---|---|---|
| 1.2.0.x | --> | 1.2.1.10 | --> | 1.3.0.10 |
| 1.2.1.x | --> | 1.3.0.10 | ||
| 1.3.0.x | --> | 1.3.0.10 |
CCU is a non-disruptive upgrade, which means that the system remains online throughout the process and that you can continue to access data normally. As a precaution, it is recommended that the upgrade occur during a time of reduced traffic. During the upgrade, the interface adapters in each canister are taken offline temporarily to be upgraded. This might impact performance or throughput. The impact is more noticeable under heavy load conditions. With a properly configured multi-path configuration, access to your data is always maintained.
To ensure a successful, non-disruptive upgrade, you should verify that your interface ports are all online and all the system hardware is functioning normally. Ideally, you should have the following:
lsenclosurebattery or the management GUI under Monitoring --> Systems to verify battery status. Note: If the battery status is 'reconditioning,' the firmware upgrade will not be allowed to start until after reconditioning completes. If the battery status is 'reconditioning required,' then you may proceed with the upgrade and perform reconditioning on the battery later. Note also that battery reconditioning can take up to 24 hours to complete. Running the ugprade test utility is a required step before concurrent upgrade in firmware versions after 1.2.0.11. The utility checks for problems in the system that might prevent the upgrade from completing successfully and either warns the user or blocks the user from proceeding. IBM Support recommends that all users planning to upgrade run the utility a full day in advance so that any issues called out by the utility can be remedied without delaying the planned upgrade.
To view checks that the upgrade utility makes before an upgrade, see the the release notes for the latest upgrade test utility posted along with each available firmware package on IBM Fix Central.
Important: Before you begin the upgrade, we recommend that you perform a backup of your data and a backup of the FlashSystem configuration. To back up the configuration, log into the cluster management IP address and issue the following command using admin-level authority:
svcconfig backup
Optionally, you can copy the configuration backup file from the FlashSystem to your workstation using secure copy (scp) on Linux or PuTTY secure copy (pscp.exe) on Windows as in the following examples:
(Using Linux)
scp superuser@cluster_ip:/dumps/svc.config.backup.* .
(Using Windows)
pscp -unsafe superuser@cluster_ip:/dumps/svc.config.backup.* .
Note: Do not ignore the periods shown above at the end of each command. In addition, replacement of italicized descriptions within angle brackets with appropriate information is required.
Posted along with the release notes and upgrade files on Fix Central are md5sum text files. These files exist for each update file so that the user can verify that the update file was downloaded correctly.
It is highly recommended that the upgrade be performed using the web-based cluster management interface known as the management GUI. Instructions are available for performing a CCU in IBM Knowledge Center. Search for 'IBM FlashSystem 900,' then navigate to Upgrading the system. Included is information on retrieving software packages, using the update test utility, and automatically updating using either the GUI or the CLI.
Use the following sections to troubleshoot problems that may occur during the upgrade process.
If the upgrade takes more than two hours to complete, it may have stalled. Upgrade status is viewed by issuing lsupdate CLI command or by going to Settings --> System --> Update System in the GUI.
Both show a 'Stalled' status. In most cases, this can be resolved by aborting the upgrade and reattempting the upgrade after the system downgrades to its original level.
To abort the upgrade, issue the applysoftware -abort CLI command or click the 'Stop Upgrade' button in the GUI.
After the system is downgraded, you can reattempt your upgrade from the GUI or CLI. If the upgrade stalls repeatedly or if you have alerts which cannot be cleared, contact IBM Support.
You may get a battery or quorum alert during upgrade due to required reconfiguration. These alerts should be automatically cleared when the upgrade is completed. They may be visible from the Events view of the management GUI if the filter is set to 'Show All,' but they should no longer appear in the Recommended Actions, Unfixed Messages, or Alerts views. If you see unfixed battery or quorum alerts after an upgrade is complete, contact IBM Support.
If the upgrade has failed or stopped due to a hardware failure, you will see the 'Hardware Failed' status.
If you suspect a hardware failure, issue the lsupdate command to confirm the state of your system. This command shows that the system is in a hardware_failed state and the event log contains a 'System upgrade suspended' event. You may resume the upgrade by issuing the applysoftware -resume -force command for the following conditions:
lsenclosurebatteryIf the upgrade cannot be resumed or you have other alerts which cannot be cleared, contact IBM Support.
The battery reconditioning feature calibrates the gauge that reports the amount of charge on the batteries.
On systems that have been installed for 10 months or more or systems that have experienced several power outages, the recommendation to run 'battery reconditioning' will appear in the event log shortly after upgrading.
This is normal. Use the management GUI to run a DMP for this error or see the FlashSystem Knowledge Center to view how to properly issue the chenclosureslot command in reference to this issue.
Use the following link to access the Knowledge Center page for battery reconditioning.
Call IBM at 1-800-IBM-SERV (1-800-426-7378). To find contact information for a specific region, visit the IBM directory of worldwide contacts.
IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of IBM Corporation in the United States, other countries, or both. These and other IBM trademarked terms are marked on their first occurrence in this information with the appropriate symbol, indicating US registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available here.
The following terms are trademarks of other companies:
Other product and service names might be trademarks of IBM or other companies.