Guidelines for good operational procedures

Good operational procedures make problem diagnosis easier and minimize the risk of damaging data.

Consider the following guidelines:

  • If Db2 hangs during distributed processing, it is possible that the problem is with VTAM® or the network, rather than Db2. For more information about procedures on diagnosing problems in a distributed environment, see Diagnosing distributed data facility (DDF) failures.
  • If a Db2 hangs in the data sharing environment, the steps that are listed in this section can be used to resolve the problem. Follow the extra steps for data sharing, where indicated.
  • Keep space available on the active logs. If all active log data sets are full and archiving has not been completed, Db2 functions do not complete. Db2 cannot be stopped in this situation.
  • Verify that all archive mount requests have been properly satisfied.
  • Be sure that IRLM address spaces have a higher dispatching priority than any allied, Db2, or IMS address spaces. If IRLM does not get enough CPU time, a Db2 subsystem can experience bad performance or wait conditions.
  • Whether the entire operating system seems to be affected or only Db2 and some of its users, request an z/OS® console dump. It should include:
    • The common service area (CSA)
    • The Db2 system services and database services address spaces and, for distributed processing only, the distributed data facility address space.

      If the problem occurs in the data sharing environment and involves more than one member of the data sharing group, take dumps of these address spaces for each member of the data sharing group.

    • z/OS master scheduler address space
    • IRLM address space.

      If the problem occurs in the data sharing environment and involves more than one member of the data sharing group, create a dump file of this address space for each member of the data sharing group. Always include the XESDATA keyword in the SDATA parameters when IRLM is a member of a SYSPLEX GROUP.

    • The address space of at least one involved user
    • GRSTRACE (global resource serialization trace)
    • Cross-system extended services (XES), if the problem occurs in the data sharing environment and involves more than one member of the data sharing group.
    To obtain a z/OS console dump:
    1. If an allied address space is hung:
      • Determine whether the relevant allied ASID getting CPU or if it is swapped out. If it is swapped out:
        Take a console dump of the z/OS master (ASID=0001). Order is important. The z/OS master must be dumped first to ensure good dump data. After the z/OS master ASID=0001 is dumped, then dump:
        • Allied ASID
        • DB2MSTR
        • DB2DBM1
        • IRLM
      • If the relevant ASID is not swapped out, the z/OS master is not required, so dump only:
        • Allied ASID
        • DB2MSTR
        • DB2DBM1
        • IRLM
    2. If Db2 is hung, take a console dump of:
      • DB2MSTR
      • DB2DBM1
      • IRLM
      • XES, if the problem occurs in the data sharing environment and involves more than one member of the data sharing group.

      SDATA information should include GRSQ and RGN added to the SDATA PARAMETER defaults on the dump command.

    Request the console dump as soon as it becomes evident that Db2 or a transaction is not functioning normally.

  • If you suspect that Db2 is looping, start the Db2 global trace, if it is not already active. Refer to Printing and analyzing global traces for information about the Db2 global trace facility.
  • If a performance problem is suspected, consider using the Db2 performance trace to obtain more information. Refer to Performance trace for information about running the performance trace and choosing the events to be traced. Refer to Using the performance trace for diagnosis for an example of using the performance trace to diagnose a WAIT problem.
  • After the console dump completed, try to break the wait or loop by issuing the command
    -CANCEL THREAD(token)
    to cancel any hung or looping threads.

    If the hung thread cannot be canceled, or if Db2 is hung, try the following commands, in order, until one of them succeeds (ssnm is the Db2 subsystem name):

    1. -STOP DB2 MODE(QUIESCE)
    2. -STOP DB2 MODE(FORCE)
    3. MODIFY irlmproc,ABEND
    4. CANCEL ssnmMSTR
    5. CANCEL ssnmDBM1
    6. FORCE ssnmMSTR

    Use the FORCE command only as a last resort. FORCE can put IRLM in an indefinite state, and you might need to re-IPL z/OS before you can restart Db2.

    If all attempts to clear the problem fail, then a re-IPL of z/OS is necessary. Taking a stand-alone dump at this point provides little diagnostic value, because the preceding attempts to clear the problem might delete important diagnostic information.

    It is very important to take a console dump BEFORE attempting to clear the problem.

  • If X'04E' or X'04F' abends that occurred before the WAIT/LOOP condition, obtain their corresponding SVC dumps. Follow the instructions in Printing and analyzing dumps and any recommendations of IBM® Support.
  • It is important to save the SYS1.LOGREC entries during the abnormal condition and for at least one hour that proceeds it. It is likewise important to save and print the console log for a similar period, preferably back to the -START DB2 command if this is not excessively long.