Resolving a resynchronization failure

An example of how to resolve a resynchronization failure by using the CEMT transaction is described.

This example uses the following commands:
  • CEMT INQUIRE CONNECTION
  • CEMT INQUIRE UOWLINK
  • CEMT INQUIRE UOW
  • CEMT INQUIRE UOWENQ
  • SET CONNECTION NOTPENDING
CICS Explorer graphicTip: In CICS Explorer, the ISC/MRO Connections view provides a functional equivalent to the INQUIRE and SET CONNECTION commands. See ISC/MRO Connections view in the CICS Explorer product documentation.
A transaction on system IYLX1 (which involves function shipping requests to system IYLX4) is failing with a SYSIDERR error. A CEMT INQUIRE CONNECTION command on system IYLX1 shows the following:
Figure 1. CEMT INQUIRE CONNECTION: connections owned by system IYLX1
  INQUIRE CONNECTION
  STATUS:  RESULTS - OVERTYPE TO MODIFY
   Con(ISC2) Net(IYLX2   )     Ins Rel Vta Appc         Unk
   Con(ISC4) Net(IYLX4   ) Pen Ins Acq Vta Appc     Xno Unk
   Con(ISC5) Net(IYLX5   )     Ins Acq Vta Appc     Xok Unk
To see more information about this connection, put the cursor on the ISC4 line and press ENTER. See Figure 2.
Figure 2. CEMT INQUIRE CONNECTION: details of connection ISC4
  INQUIRE CONNECTION
  RESULT - OVERTYPE TO MODIFY
    Connection(ISC4)
    Netname(IYLX4)
    Pendstatus( Pending )
    Servstatus( Inservice )
    Connstatus( Acquired )
    Accessmethod(Vtam)
    Protocol(Appc)
    Purgetype(              )
    Xlnstatus(Xnotdone)
    Recovstatus( Nrs )
    Uowaction(              )
    Grname()
    Membername()
    Affinity(              )
    Remotesystem()
    Rname()
    Rnetname()
Note: VTAM® is the previous name for z/OS® Communications Server.

Although the Connstatus of connection ISC4 is Acquired, the Xlnstatus is Xnotdone. The exchange lognames (XLN) flow for this connection has not completed successfully. (When CICS® systems connect they exchange lognames. These lognames are verified before resynchronization is attempted, and an exchange lognames failure means that resynchronization is not possible.) For function shipping, a failure for the connection causes a SYSIDERR. Synchronization level 2 conversations are not allowed on this connection until lognames are successfully exchanged. (This restriction does not apply to MRO connections.)

The reason for the exchange lognames failure is reported in the CSMT log. A failure on a CICS Transaction Server for z/OS system can be caused by the following:
  • An initial start (START=INITIAL) of the CICS TS for z/OS system, or of a partner.
    Note: A cold start (START=COLD) of a CICS TS for z/OS system preserves resynchronization information (including the logname) and does not, therefore, cause an exchange lognames failure.
  • Use of the CEMT SET CONNECTION NORECOVDATA command.
  • A system logic or operational error.

The Pendstatus for connection ISC4 is Pending, which means that there is resynchronization work outstanding for the connection; this work cannot be completed because of the exchange lognames failure.

At this stage, if you are not concerned about loss of synchronization, you can force all indoubt UOWs to commit or back out by issuing the SET CONNECTION NOTPENDING command. However, before you do so, you can investigate the outstanding resynchronization work that exists before we clear the pending condition.

You can use a CEMT INQUIRE UOWLINK command to display information about UOWs that require resynchronization with system IYLX4:
Figure 3. CEMT INQUIRE UOWLINK: UOWs that require resynchronization with system IYLX4
  INQUIRE UOWLINK LINK(IYLX4)
  STATUS:  RESULTS - OVERTYPE TO MODIFY
   Uowl(016C0005) Uow(ABD40B40C1334401) Con Lin(IYLX4   )
      Coo Appc Col Sys(ISC4)            Net(..GBIBMIYA.IYLX150 M. A....)
   Uowl(01680005) Uow(ABD40B40C67C8201) Con Lin(IYLX4   )
      Coo Appc Col Sys(ISC4)            Net(..GBIBMIYA.IYLX151 M. F@b..)
   Uowl(016D0005) Uow(ABD40B40DA5A8803) Con Lin(IYLX4   )
      Coo Appc Col Sys(ISC4)            Net(..GBIBMIYA.IYLX156 M. .!h..)
To see more information for each UOW-link, press enter alongside it. For example, the expanded information for UOW-link 016C0005 shows the following:
Figure 4. CEMT INQUIRE UOWLINK: detailed information for UOW-link 016C0005
  I UOWLINK LINK(IYLX4)
  RESULT - OVERTYPE TO MODIFY
    Uowlink(016C0005)
    Uow(ABD40B40C1334401)
    Type(Connection)
    Link(IYLX4)
    Action(              )
    Role(Coordinator)
    Protocol(Appc)
    Resyncstatus(Coldstart)
    Sysid(ISC4)
    Rmiqfy()
    Netuowid(..GBIBMIYA.IYLX150 M. A....)

The Resyncstatus of Coldstart confirms that system IYLX4 has been started with a new logname. The Role for this UOW-link is shown as Coordinator, which means that IYLX4 is the syncpoint coordinator.

You could now use a CEMT INQUIRE UOW LINK(IYLX4) command to show all UOWs that are indoubt and which have system IYLX4 as the coordinator system:
Figure 5. CEMT INQUIRE UOW LINK(IYLX4): all UOWs that have IYLX4 as the coordinator
  INQUIRE UOW LINK(IYLX4)
  STATUS:  RESULTS - OVERTYPE TO MODIFY
   Uow(ABD40B40C1334401) Ind Shu Tra(RFS1) Tas(0000674)
      Age(00003560) Ter(X150) Netn(IYLX150 ) Use(CICSUSER) Con Lin(IYLX4   )
   Uow(ABD40B40C67C8201) Ind Shu Tra(RFS1) Tas(0000675)
      Age(00003465) Ter(X151) Netn(IYLX151 ) Use(CICSUSER) Con Lin(IYLX4   )
   Uow(ABD40B40DA5A8803) Ind Shu Tra(RFS1) Tas(0000676)
      Age(00003462) Ter(X156) Netn(IYLX156 ) Use(CICSUSER) Con Lin(IYLX4   )
To see more information for each indoubt UOW, press enter on its line. For example, the expanded information for UOW ABD40B40C1334401 shows the following:
Figure 6. CEMT INQUIRE UOW LINK(IYLX4): detailed information for UOW ABD40B40C1334401
  INQUIRE UOW LINK(IYLX4)
  RESULT - OVERTYPE TO MODIFY
    Uow(ABD40B40C1334401)
    Uowstate( Indoubt )
    Waitstate(Shunted)
    Transid(RFS1)
    Taskid(0000674)
    Age(00003906)
    Termid(X150)
    Netname(IYLX150)
    Userid(CICSUSER)
    Waitcause(Connection)
    Link(IYLX4)
    Sysid(ISC4)
    Netuowid(..GBIBMIYA.IYLX150 M. A....)

This UOW cannot be resynchronized by system IYLX4. Its status is shown as Indoubt, because IYLX4 does not know whether the associated UOW that ran on IYLX4 committed or backed out.

You can use the CEMT INQUIRE UOWENQ command to display the resources that have been locked by all shunted UOWs (those that own retained locks):
Figure 7. CEMT INQUIRE UOWENQ: resources locked by all shunted UOWs
  INQUIRE UOWENQ OWN RETAINED
  STATUS:  RESULTS
   Uow(ABD40B40C1334401) Tra(RFS1) Tas(0000674) Ret Tsq Own
      Res(RFS1X150                  ) Rle(008) Enq(00000008)
   Uow(ABD40B40C67C8201) Tra(RFS1) Tas(0000675) Ret Tsq Own
      Res(RFS1X151                  ) Rle(008) Enq(00000008)
   Uow(ABD40B40DA5A8803) Tra(RFS1) Tas(0000676) Ret Tsq Own
      Res(RFS1X156                  ) Rle(008) Enq(00000008)
You can filter the INQUIRE UOWENQ command so that only enqueues that are owned by a particular UOW are displayed. For example, to filter for enqueues owned by UOW ABD40B40C1334401:
Figure 8. CEMT INQUIRE UOWENQ: resources locked by UOW ABD40B40C1334401
  INQUIRE UOWENQ OWN UOW(*4401)
  STATUS:  RESULTS
   Uow(ABD40B40C1334401) Tra(RFS1) Tas(0000674) Ret Tsq Own
      Res(RFS1X150                  ) Rle(008) Enq(00000008)
To see more information for this UOWENQ, press enter alongside it:
Figure 9. CEMT INQUIRE UOWENQ: detailed information for UOWENQ ABD40B40C1334401
  INQUIRE UOWENQ OWN UOW(*4401)
  RESULT
    Uowenq
    Uow(ABD40B40C1334401)
    Transid(RFS1)
    Taskid(0000674)
    State(Retained)
    Type(Tsq)
    Relation(Owner)
    Resource(RFS1X150)
    Rlen(008)
    Enqfails(00000008)
    Netuowid(..GBIBMIYA.IYLX150 M. A....)
    Qualifier()
    Qlen(000)

With knowledge of the application, it may now be possible to decide whether updates to the locked resources should be committed or backed out. In the case of UOW ABD40B40C1334401, the locked resource is the temporary storage queue RFS1X150. This resource has an ENQFAILS value of 8, which is the number of tasks that have received the LOCKED response due to this enqueue being held in retained state.

You can use the SET UOW command to commit, back out, or force the uncommitted updates made by the shunted UOWs. Next, you must use the SET CONNECTION(ISC4) NOTPENDING command to clear the pending condition and allow synchronization level 2 conversations (including the function shipping requests which were previously failing with SYSIDERR).

You can use the XLNACTION option of the CONNECTION definition to control the effect of an exchange lognames failure. In this example, the XLNACTION for the connection ISC4 is KEEP. This meant that:
  • The shunted UOWs on system IYLX1 were kept following the cold/warm log mismatch with IYLX4.
  • The APPC connection between IYLX1 and IYLX4 could not be used for function shipping requests until the pending condition was resolved.

An XLNACTION of FORCE for connection ISC4 would have caused the SET CONNECTION NOTPENDING command to have been issued automatically when the cold/warm log mismatch occurred. This would have forced the shunted UOWs to commit or back out, according to the ACTION option of the associated transaction definition. The connection ISC4 would then not have been placed into Pending status. However, setting XLNACTION to FORCE allows no investigation of shunted UOWs following an exchange lognames failure, and therefore represents a greater risk to data integrity than setting XLNACTION to KEEP.



resresf.html | Timestamp icon Last updated: Monday, 5 June 2017