Interim Fix number: PK75441 Product: WebSphere Application Server Fix (APAR): PK75441 Status: Fix Release: 6.0.2.23, 6.0.2.25, 6.0.2.27, 6.0.2.29, 6.0.2.31, 6.0.2.33, 6.1.0.19, 6.1.0.21, 7.0.0.1 Operating System: AIX,HP-UX,i5/OS,Linux,Linux pSeries,Linux zSeries,Solaris,Windows Supersedes Fixes: CMVC Defect: 561563, 572768 Abstract: Filetransfer can fail randomly, causing a cell to be unable to synchronize properly until the cell is restarted. Description/symptom of problem: PK75441 resolves the following problem: ERROR DESCRIPTION:? Filetransfer will randomly fail, and it causes the cell to be unable to sync properly until the cell is restarted Symptom: [10/16/08 16:01:07:049 EDT] 0000003d FileDocument E ADMR0105E: The system is unable to write document cells/myCell /xdcatalognodes.config: java.io.FileNotFoundException: File download failed: java.io.IOException: File download failed [404].Message: 404 Not Found for http://mycorp.com:9060/FileTransfer/transfer/ download%2Fcells%2FmyCell%2Fxdcatalognodes.config1687.tmp? compress=true&deleteOnCompletion=true. at com.ibm.websphere.management.filetransfer.client. FileDownloadInputStream.getStream(FileDownloadInputStream.java :226) at com.ibm.websphere.management.repository.RepositoryInputStr eam.read(RepositoryInputStream.java:133) at java.security.DigestInputStream.read(DigestInputStream.jav a:161) at java.io.FilterInputStream.read(FilterInputStream.java:113) at com.ibm.ws.management.repository.FileDocument.copyDocument (FileDocument.java:445) at com.ibm.ws.management.repository.FileDocument.write(FileDo cument.java:288) at com.ibm.ws.management.repository.FileRepository.modifyInte rnal(FileRepository.java:1154) at com.ibm.ws.management.repository.FileRepository.updateInte rnal(FileRepository.java:1354) at com.ibm.ws.management.repository.FileRepository.update(Fil eRepository.java:1286) at com.ibm.ws.management.repository.FileRepository.modify(Fil eRepository.java:1073) at com.ibm.ws.management.repository.FileRepository.modify(Fil eRepository.java:1065) at com.ibm.ws.management.sync.NodeSyncTask.processFolderSyncU pdate(NodeSyncTask.java:465) at com.ibm.ws.management.sync.NodeSyncTask.doSync(NodeSyncTas k.java:296) at com.ibm.ws.management.sync.NodeSyncTask.run(NodeSyncTask.j ava:181) at java.lang.Thread.run(Thread.java:810) Here is a trace snippet showing the root cause: [10/30/08 10:11:12:648 EDT] 00000093 FileTransferS 3 sendFile: Method = GET [10/30/08 10:11:12:648 EDT] 00000094 FileTransferS 3 sendFile: Method = GET [10/30/08 10:11:12:648 EDT] 00000095 FileTransferS 3 sendFile: Method = GET [10/30/08 10:11:12:648 EDT] 00000095 FileTransferS 3 The repository root=[/opt/WAS61/profiles/dmgr/config/temp] [10/30/08 10:11:12:648 EDT] 00000093 FileTransferS 3 The repository root=[/opt/WAS61/profiles/dmgr/config/temp] [10/30/08 10:11:12:648 EDT] 00000094 FileTransferS 3 The repository root=[/opt/WAS61/profiles/dmgr/config/tem] [10/30/08 10:11:12:648 EDT] 00000095 FileTransferS > sendFile Entry /download/cells/myCell/xdcatalognodes.config5734.tmp [10/30/08 10:11:12:648 EDT] 00000093 FileTransferS > sendFile Entry /download/cells/myCell/xdcatalognodes.config5740.tmp [10/30/08 10:11:12:648 EDT] 00000094 FileTransferS > sendFile Entry /download/cells/myCell/xdcatalognodes.config5735.tmp Note the three strings "The repository root = ". The 'p' gets stripped off in the final one due to the thread concurrently updating the servlet instance variable. LOCAL FIX:? PROBLEM SUMMARY:? USERS AFFECTED: All users of WebSphere Application Server V6.0.2, V6.1 and V7.0 PROBLEM DESCRIPTION: Filetransfer can fail randomly, causing a cell to be unable to synchronize properly until the cell is restarted. RECOMMENDATION: None Under certain conditions, the repository root value in FileTransferServlet can become corrupted. Here is a sample log from a WebSphere Extended Deployment system that shows one symptom of the problem: [10/16/08 16:01:07:049 EDT] 0000003d FileDocument E ADMR0105E: The system is unable to write document cells/myCell/xdcatalognodes.config: java.io.FileNotFoundException: File download failed: java.io.IOException: File download failed [404].Message: 404 Not Found for http://xdblade.mycompany.com:9060/FileTransfer/transfer/download %2Fcells%2FmyCell%2Fxdcatalognodes.config1687.tmp?compress=true& deleteOnCompletion=true. at com.ibm.websphere.management.filetransfer.client.FileDownloadInp utStream.getStream(FileDownloadInputStream.java:226) at com.ibm.websphere.management.repository.RepositoryInputStream.re ad(RepositoryInputStream.java:133) at java.security.DigestInputStream.read(DigestInputStream.java:161) at java.io.FilterInputStream.read(FilterInputStream.java:113) at com.ibm.ws.management.repository.FileDocument.copyDocument(FileD ocument.java:445) at com.ibm.ws.management.repository.FileDocument.write(FileDocument .java:288) at com.ibm.ws.management.repository.FileRepository.modifyInternal(F ileRepository.java:1154) at com.ibm.ws.management.repository.FileRepository.updateInternal(F ileRepository.java:1354) at com.ibm.ws.management.repository.FileRepository.update(FileRepos itory.java:1286) at com.ibm.ws.management.repository.FileRepository.modify(FileRepos itory.java:1073) at com.ibm.ws.management.repository.FileRepository.modify(FileRepos itory.java:1065) at com.ibm.ws.management.sync.NodeSyncTask.processFolderSyncUpdate( NodeSyncTask.java:465) at com.ibm.ws.management.sync.NodeSyncTask.doSync(NodeSyncTask.java :296) at com.ibm.ws.management.sync.NodeSyncTask.run(NodeSyncTask.java:18 1) at java.lang.Thread.run(Thread.java:810) The root cause is illustrated by the trace below (with trace spec com.ibm.ws.management.filetransfer.servlet.*=all): [10/30/08 10:11:12:648 EDT] 00000093 FileTransferS 3 sendFile: Method = GET [10/30/08 10:11:12:648 EDT] 00000094 FileTransferS 3 sendFile: Method = GET [10/30/08 10:11:12:648 EDT] 00000095 FileTransferS 3 sendFile: Method = GET [10/30/08 10:11:12:648 EDT] 00000095 FileTransferS 3 The repository root=[/opt/WAS61/profiles/dmgr/config/temp] [10/30/08 10:11:12:648 EDT] 00000093 FileTransferS 3 The repository root=[/opt/WAS61/profiles/dmgr/config/temp] [10/30/08 10:11:12:648 EDT] 00000094 FileTransferS 3 The repository root=[/opt/WAS61/profiles/dmgr/config/tem] [10/30/08 10:11:12:648 EDT] 00000095 FileTransferS > sendFile Entry /download/cells/myCell/xdcatalognodes.config5734.tmp [10/30/08 10:11:12:648 EDT] 00000093 FileTransferS > sendFile Entry /download/cells/myCell/xdcatalognodes.config5740.tmp [10/30/08 10:11:12:648 EDT] 00000094 FileTransferS > sendFile Entry /download/cells/myCell/xdcatalognodes.config5735.tmp Notice that the third repository root value is truncated, because a single field is being updated by multiple threads at the same time. PROBLEM CONCLUSION:? The code was fixed by modifying the method that sets the value of repository root to be thread safe. The fix for this APAR is currently targeted for inclusion in fixpacks 6.0.2.35, 6.1.0.23 and 7.0.0.3. Please refer to the Recommended Updates page for delivery information: http://www.ibm.com/support/docview.wss?rs=180&uid=swg27004980