Diagnosing a wait by using the SYSZDSN3.ERLYOLRH ENQ

If Db2 and, possibly, z/OS® are unresponsive, the holding of ENQ resource SYSZDSN3.ERLYOLRH can provide important clues about the location, but not the cause, of a wait. Normally this resource is held for short periods of time. If a number of tasks are holding or waiting for the resource longer than a few seconds, then the tasks should be examined.

About this task

SYSZDSN3.ERLYOLRH.ERLY_block_address is the full name of the ENQ resource. ERLY_block_address is a 32-bit binary address. If more than one Db2 subsystem is active, the ERLY_block_address value is unique for each subsystem. Be sure to look at the correct ENQ resource name.

Procedure

To locate the resource and analyze the status of each holding task:

  1. The holders of the ENQ resource can be located by the following methods:
    • Various software monitor products can display the status of ENQ resources. However, if TSO or the z/OS consoles are unresponsive, not all of them are working.
    • If z/OS is unresponsive, a stand-alone dump should have been taken before you restart the system. Find the SYSZDSN3.ERLYOLRH ENQ resource in the GRSTRACE portion of the dump. If you are printing the dump, include the GRSTRACE verb. Locate the ENQ resources in the section that is entitled, OUTPUT FROM GRSTRACE VERB. If you are using the dump online using IPCS, then invoke the GRSTRACE verb directly.
    • Issue the following z/OS console command, which can also be issued by way of the TSO OPER command:
      D GRS,RES=(SYSZDSN3,*)
  2. If the ENQ resource is not held, skip the rest of this section. If it is held, the next step should be to determine if a problem exists in GRS. Review any messages and accompanying return and reason codes you receive.
    If any match those codes in the following table, it indicates a possible GRS problem.
    Table 1. Messages and codes for GRS problems
    Message Return Code Reason Code
    IDC3009I 4 Any
    184 Any
    IEC161 28 184
    52 Any
  3. In the GRSTRACE display of the SYSZDSN3.ERLYOLRH ENQ resource, an exclusive request at the top of the list prevents any shared request from getting the resource.
    • Shared holders that are higher in the list than an exclusive request must all release the resource before the exclusive request can be satisfied.
    • In Db2, a task holds this resource while it is running in the Db2 subsystem interface code. Tasks hold the resource in SHARED mode, except for the Db2 job step tasks and a z/OS master scheduler task, which is transmitting an End-of-Memory (EOM) condition on a Db2 address space. These tasks request the ENQ in EXCLUSIVE mode.
  4. If all of the tasks are holding the resource in SHARED mode, then one or more are waiting in some Db2 module.
    • If you are viewing the ENQ resource online with a software monitor, the -DISPLAY THREAD command might be used to determine the status of the corresponding threads. Match the ASIDs in the list of tasks that are holding the resource with the ASIDs of the threads. For an interpretation of the -DISPLAY THREAD fields and the actions to take, refer to Initial procedure for the WAIT/LOOP keywords. Discontinue using this procedure unless the wait state cannot be cleared.
    • If you are viewing the ENQ resource from a dump, it should be a stand-alone dump. It is necessary to look at the status of each task that holds the resource. The address and module name of the wait point must be determined for each task. Refer to Locating the waiting CSECT below.
  5. If any task is requesting or holding the resource in EXCLUSIVE mode, then the Db2 subsystem is in the process of abnormal termination. Remember the jobname, ASID, and TCB address of the task. As stated above, it is either a Db2 job step or z/OS master scheduler task.
    • If the task is the first on the list, it is holding the ENQ resource. The Db2 subsystem interface code is attempting to complete subsystem abend termination, but has been unable to do so.
    • If the task is not first on the list, then it is waiting to obtain the resource in EXCLUSIVE mode, but is unable to do so because of one or more SHARED holders. The process of Db2 subsystem abend issues an ABEND with code X'04F' against the allied task of each thread that runs Db2 code. The presence of SHARED holders indicates that the abend was unsuccessful on those holders.
    • In any case, under this condition, Db2 cannot be recovered. If a stand-alone dump and restart the system has not been requested, do so now.
  6. Using the stand-alone dump, locate the address space and task of the EXCLUSIVE holding or waiting task and any shared holding tasks. Determine the address and module name of the wait point for each task.
    If the dump is printed, use the z/OS TCB Summary to locate each address space and task in the dump. If you are viewing the dump by using IPCS, use IPCS procedures to locate each.

    Any shared waiting tasks that follow the EXCLUSIVE holding task need not be examined. They have not caused the wait state.