CMVC FREQUENTLY ASKED QUESTIONS: CMVCD, NOTIFYD, CMVCARCHIVE/RESTORE, USER EXITS, SERVER ERRORS, DATABASE ISSUES Document Number TR 29.2332 Angel Rivera, Lee Perlov, Clifford Meyers CMVC/TeamConnection Development IBM Software Solutions Research Triangle Park, North Carolina Copyright (C) 1997 IBM Corp. All rights reserved. ii CMVC FAQ: server ABSTRACT This technical report provides answers to frequently asked questions made by CMVC users with respect to the following CMVC topics: o cmvcd daemons o notifyd daemon and mail notification o cmvcarchive and cmvcrestore o user exits o server errors o database issues ITIRC KEYWORDS o CMVC o cmvcd o notifyd o user exits o cmvcarchive o cmvcrestore ABSTRACT iii iv CMVC FAQ: server ABOUT THE AUTHORS ANGEL RIVERA Mr. Rivera is an Advisory Software Engineer and team lead for the development of CMVC 2.3. He joined IBM in 1989 and since then has worked in the development and support of library systems. Mr. Rivera has an M.S. in Electrical Engineering from The Univer- sity of Texas at Austin, and B.S. in Electronic Systems Engi- neering from the Instituto Tecnologico y de Estudios Superiores de Monterrey, Mexico. LEE PERLOV Mr. Perlov is a Staff Software Engineer in the TeamConnection/CMVC development group. He started working for IBM in 1985 in Gaithersburg, Md, working in the Federal Systems Division on various projects for the United States intelligence community. He then moved to RTP to work on library development and support. Mr. Perlov received a B.S.Acc degree in Accounting from the Uni- versity of Florida in 1983. He also completed two years of grad- uate work in the Department of Computer Science at the University of Florida. CLIFFORD MEYERS Mr. Meyers is an advisory programmer in the TeamConnection/CMVC development group. He joined IBM in 1985 as support for AIX/370, AIX/ESA, and CMVC, before accepting his current assignment in 1997. Mr. Meyers has an A.S. in Business Administration from SUNY Cobleskill and a B.S. in Computer Science from SUNY Postdam. ABOUT THE AUTHORS v vi CMVC FAQ: server CONTENTS ABSTRACT . . . . . . . . . . . . . . . . . . . . . . . . . III ITIRC KEYWORDS . . . . . . . . . . . . . . . . . . . . . iii ABOUT THE AUTHORS . . . . . . . . . . . . . . . . . . . . . . V Angel Rivera . . . . . . . . . . . . . . . . . . . . . . . v Lee Perlov . . . . . . . . . . . . . . . . . . . . . . . . v Clifford Meyers . . . . . . . . . . . . . . . . . . . . . . v FIGURES . . . . . . . . . . . . . . . . . . . . . . . . . . IX 1.0 INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . 1 1.1 Acknowledgements . . . . . . . . . . . . . . . . . . . 1 2.0 CMVCD DAEMONS . . . . . . . . . . . . . . . . . . . . . 3 2.1 What is the relationship between the CMVC client and the CMVC server? . . . . . . . . . . . . . . . . . . . . . 3 2.2 How does the CMVC server process the transactions? . . 4 2.3 What is the flow of a server transaction? . . . . . . 5 2.4 What are the interactions between the server and the database? . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.5 What is the purpose of the Sequence table? . . . . . . 7 2.6 How to customize the default defect names . . . . . . 7 2.7 How to debug problems with the CMVC daemons (part 1) 8 2.8 How to debug problems with the CMVC daemons (part 2) 10 2.9 Server daemon fails to start when specifying multiple daemons . . . . . . . . . . . . . . . . . . . . . . . . . 12 2.10 Use the monitor tool to supervise the activity of the server daemons . . . . . . . . . . . . . . . . . . . . . 13 2.11 Do not change the file permissions nor the ownership of cmvc files . . . . . . . . . . . . . . . . . . . . . . 14 2.12 When stopping a CMVC family, use the stopCMVC script 14 2.13 How to deal with CMVC processes showing . 15 2.14 stopCMVC: conflict with DB2 instance and CMVC family names . . . . . . . . . . . . . . . . . . . . . . . . . . 16 3.0 NOTIFYD AND MAIL NOTIFICATION . . . . . . . . . . . . 17 3.1 What is the interaction between the CMVC server and notifyd? . . . . . . . . . . . . . . . . . . . . . . . . 17 3.2 Diagnosing and correcting typical mail problems . . 18 3.3 How can I manipulate the text of the message to new users ? . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.4 How to diagnose problems with notification (notifyd) 19 3.5 Subdirectories queue and queue/messages in the CMVC family account . . . . . . . . . . . . . . . . . . . . . 20 3.6 notifyd will not start when using Oracle . . . . . . 20 4.0 CMVCARCHIVE AND CMVCRESTORE . . . . . . . . . . . . . 23 4.1 What important details should I know about cmvcarchive and cmvcrestore? . . . . . . . . . . . . . . . . . . . . 23 Contents vii 4.2 What additional details should I know about cmvcarchive? . . . . . . . . . . . . . . . . . . . . . . 24 4.3 What are the prerequisites for cmvcarchive? . . . . 26 4.4 What are the files created by cmvcarchive? . . . . . 26 4.5 What additional details should I know about cmvcrestore? . . . . . . . . . . . . . . . . . . . . . . 28 4.6 What are the prerequisites for the CMVC family to be restored? . . . . . . . . . . . . . . . . . . . . . . . . 28 4.7 If having problems with cmvcarchive, then migrate to CMVC 2.3.0.22 or higher . . . . . . . . . . . . . . . . . 29 4.8 CMVC is not reclaiming file system space when doing cmvcarchive . . . . . . . . . . . . . . . . . . . . . . . 29 4.9 cmvcarchive is taking a long time to run . . . . . . 30 4.10 Several problems with cmvcarchive . . . . . . . . . 31 4.11 How to find out a missing path id (due to cmvcarchive) . . . . . . . . . . . . . . . . . . . . . . 32 5.0 USER EXITS . . . . . . . . . . . . . . . . . . . . . . 35 5.1 What are user exits? . . . . . . . . . . . . . . . . 35 5.2 Showing a small user exit example . . . . . . . . . 36 5.3 What are the typical problems with user exits? . . . 37 5.4 Warning on using CMVC line commands inside CMVC user exits . . . . . . . . . . . . . . . . . . . . . . . . . . 37 5.5 New parameters for user exits in CMVC commands, starting with 2.3.0.22 . . . . . . . . . . . . . . . . . 38 5.6 Apparent problem when passing more than 10 parameters to a user exit . . . . . . . . . . . . . . . . . . . . . 39 5.7 The temp file for user exit 2 FileCheckIn is no longer available . . . . . . . . . . . . . . . . . . . . . . . . 39 5.8 Are there any real examples of UserExits shipped with CMVC? . . . . . . . . . . . . . . . . . . . . . . . . . . 41 6.0 ERROR HANDLING AND AUDIT LOG . . . . . . . . . . . . . 43 6.1 What details should I know about error management? . 43 6.2 Where are the error messages from the CMVC daemon and databases? . . . . . . . . . . . . . . . . . . . . . . . 43 6.3 How to get more information from DB2 error numbers . 44 6.4 Error message: 0010-063 with keyword bind() . . . . 45 6.5 CMVC error message 0010-063: Not owner, chown() function . . . . . . . . . . . . . . . . . . . . . . . . 47 6.6 Error message 0010-063 - Broken pipe, write () function . . . . . . . . . . . . . . . . . . . . . . . . 47 6.7 How to deal with the 0010-250 error message . . . . 47 6.8 CMVC error message: 0010-256 . . . . . . . . . . . . 48 6.9 CMVC error message: 0010-381 (followed by 0010-256) 49 6.10 What details should I know about audit logging? . . 51 6.11 After running the sample cmvclog.cleanu, cmvcd does not log . . . . . . . . . . . . . . . . . . . . . . . . . 51 7.0 DATABASE ISSUES . . . . . . . . . . . . . . . . . . . 53 7.1 Indexes that improve performance in levels and in file changes . . . . . . . . . . . . . . . . . . . . . . . . . 53 7.2 Reorganizing and rebinding a DB2 database. . . . . . 54 7.3 Running DB2 RUNSTATS is taking a very long time . . 55 viii CMVC FAQ: server 7.4 Need to reboot the host if system hangs during db2stop 55 7.5 backupCMVC fails with DB2 error DB2106E . . . . . . 56 7.6 How to display the SQLCA reason code when there is a DB2 problem . . . . . . . . . . . . . . . . . . . . . . . 56 7.7 How to dump the Sybase transaction log due to the notify daemon . . . . . . . . . . . . . . . . . . . . . . 57 APPENDIX A. BIBLIOGRAPHY . . . . . . . . . . . . . . . . . 59 A.1 How to get electronic copies of manuals and technical reports . . . . . . . . . . . . . . . . . . . . . . . . . 60 A.1.1 IBM Intranet . . . . . . . . . . . . . . . . . . 60 A.1.2 Internet . . . . . . . . . . . . . . . . . . . . 60 APPENDIX B. COPYRIGHTS, TRADEMARKS AND SERVICE MARKS . . . 63 FIGURES 1. Updated parameter list for user exits . . . . . . . . 40 2. Sample Bourne shell using positional parameters . . . 41 3. Sample Korn shell using positional parameters . . . . 42 Contents ix x CMVC FAQ: server 1.0 INTRODUCTION This technical report provides answers to frequently asked questions made by CMVC users with respect to the following CMVC topics: o cmvcd daemons o notifyd daemon and mail notification o cmvcarchive and cmvcrestore o user exits o server errors and audit log o database issues 1.1 ACKNOWLEDGEMENTS Many of the questions and answers that are compiled in this tech- nical report were obtained from the CMVC forum in the IBMPC con- ferencing disk, and from the CMVC6000 forum in the IBMUNIX conferencing disk. We want to thank the main participants in these electronic forums for their support! Also, we want to thank Dodde Stark for editing this technical report. Introduction 1 2 CMVC FAQ: server 2.0 CMVCD DAEMONS 2.1 WHAT IS THE RELATIONSHIP BETWEEN THE CMVC CLIENT AND THE CMVC SERVER? QUESTION: What is the relationship between the CMVC client and the CMVC server? ANSWER: o Because all CMVC client requests depend in some way on the interaction with the database, a connection between the CMVC server and the database must be made before work for a client can begin. o The CMVC server is implemented as one or more background server processes (cmvcd daemons). Each of these server proc- esses can service one client request at a time. The number of server processes that should be started depends on the number of clients that are likely to make requests at the same time. On the one hand, starting many server processes wastes system resources and shared memory on the CMVC server. On the other hand, starting few server processes may cause some client request rejections (a connection refused message). When you start the server processes, you indicate the number you want to start. The CMVC server will fork (spawn) an additional child process for each of the parent server proc- esses that have been requested. Therefore, if you query the number of cmvcd processes running for the CMVC family's, account you will see double the number that you requested. You can use the CMVC Activity Monitor to see the server proc- esses that are servicing requests. For more details, see 2.10, "Use the monitor tool to supervise the activity of the server daemons" on page 13. o When the parent server processes are initialized, they issue a series of TCP/IP socket calls to establish the server-side socket connection. Also during initialization, each child cmvcd daemons 3 server process establishes a connection to the database and then it waits for client requests. The operating system kernel decides which server process gets the client's socket connection. During server process initialization, each child server process connects to the database causing the creation of a shadow database process for each child server process. For example, type ps -u from the UNIX prompt on the CMVC server to see the server processes and the shadow data- base processes. The CMVC clients locate the CMVC server processes by using the network address of the machine on which the server proc- esses are running and the port number on which the server processes are listening. 2.2 HOW DOES THE CMVC SERVER PROCESS THE TRANSACTIONS? QUESTION: How does the CMVC server process the transactions? ANSWER: The requests from a client are handled by the transaction processor in the server. A CMVC transaction is the accomplishment of one unit of work. One command line invocation (directly or indirectly via the GUI) results in one or more transactions. o A server process (cmvcd) will always complete its transaction regardless of whether the client connection is still intact. - The client connection can be broken by entering CTRL-C from the client's machine. - When the transaction completes, the packets of informa- tion will be discarded if the client-server connection is not available. - The client's request will still be effective since the server will have completed the requested unit of work. o A transaction can be terminated at the server side by 'recy- cling' a server process. - To recycle a server daemon process, enter 'kill -1 pid', where pid is the process id of the server daemon process. 4 CMVC FAQ: server This will roll back the transaction within the database and prepare the server daemon process for the next request. - A transaction may complete before you have time to termi- nate or recycle it. Once a transaction is complete it can only be undone by using CMVC actions that would reverse the effect of the completed transaction. Some transactions can never be undone (for example, File -destroy, Level -commit, Level -complete). 2.3 WHAT IS THE FLOW OF A SERVER TRANSACTION? QUESTION: What is the flow of a server transaction? ANSWER: The following is the typical sequence of steps that the CMVC server takes when servicing a user request: 1. Make a shared memory entry for the CMVC action, which can be looked at by using the monitor tool. 2. Initialize the return arguments and message buffer. 3. Query the objects in the database. This includes validating the existence of the objects and related objects, obtaining object information and locking required objects. 4. Check the user's access authority to perform the action. 5. Test the validity of the action. For example, testing that the requested state change for an object is valid. 6. Make modifications, insertions, deletions to the database and in some cases the version control system. 7. Commit the changes to the database. 8. Notify the users of the action. 9. Make an audit entry for successful CMVC actions. cmvcd daemons 5 10. Return the arguments and the message buffer to the client. 2.4 WHAT ARE THE INTERACTIONS BETWEEN THE SERVER AND THE DATABASE? QUESTION: What are the interactions between the server and the database? ANSWER: o For the storage and fast retrieval of user data (other than files), the CMVC server makes use of an underlying relational SQL database. Each child server daemon process has a connection to the database and makes requests using embedded SQL. The CMVC server depends on a type of database transaction that begins implicitly when the first database request is made. The transaction is explicitly terminated by a commit or a rollback command. o The majority of the locking that occurs is done by the data- base by default. For example, when a database user updates a record, that record remains locked for that user until it is explicitly committed or rolled back. When the default locking is not sufficient, then table locks are used. For example, the default locking is not sufficient when a database record is to be modified based on its current content, because another user could modify the record first and in this way, it could invalidate the action. Unique indexes on tables are used to ensure that duplicate records are not created. o Deadlock conditions are detected and resolved by the data- base. If the database detects a deadlock condition it will return an error to one of the deadlock participants. The CMVC server treats a deadlock error like any other database error. 6 CMVC FAQ: server 2.5 WHAT IS THE PURPOSE OF THE SEQUENCE TABLE? QUESTION: What is the purpose of the Sequence table? ANSWER: Similar to most database applications, the information about CMVC objects is stored in a number of tables. Database identifiers are used to link the information in the various tables. These database identifiers are controlled by sequence numbers within the CMVC Server. The CMVC server has 3 sequence numbers which are stored in the Sequence table: o defect It controls the next number assigned to a defect or a feature. o source It controls the next number assigned to a file during cre- ation time. o general It controls the next number assigned to all other database objects. 2.6 HOW TO CUSTOMIZE THE DEFAULT DEFECT NAMES QUESTION: I would like to customize the default names used for defects and features: o I would like to start at a different number, such as 20000. o I would like to add some alphanumeric characters to the default defect number, such as "ABC" in "ABC100". ANSWER: The CMVC family administrator could do the following customizations for the defect/feature names: o To change only the starting sequence number, you can modify the value for the "defect" entry in the "sequence" table. Take note of the current value, and then enter a new value cmvcd daemons 7 that is greater than the current value, in order to avoid conflict with existing defect names. o You can use a user exit to modify the generated defect/feature name by following the suggestions in the Redbook GG24-4178 "Did you say CMVC?", section 4.7.2.2, page 96 and section C.2 "User exit to generate defect or feature number", page 185. 2.7 HOW TO DEBUG PROBLEMS WITH THE CMVC DAEMONS (PART 1) QUESTION: How to debug problems with the CMVC daemons (part 1) ANSWER: This section summarizes the steps that you can take to better diagnose the problems with the CMVC daemons, in case they seem to suddenly die or behave in an unusual manner. For extra sug- gestions due to rare combination of events, see 2.8, "How to debug problems with the CMVC daemons (part 2)" on page 10. First of all, you will need to enable the syslog facility, as explained in the technical report 29.2298 "CMVC frequently asked questions: hints and tips on using databases and OEM platforms", section "Where are the error messages from the CMVC daemon and databases?" Due to the nature of the CMVC daemons, the messages generated by CMVC and by the database are logged in the system log "syslog". Thus, if a CMVC daemon suddenly dies, then it is very likely that there will be a message at the bottom of the syslog that will reveal more details on the cause of the problem. Then, it is suggested that you perform the following sequence of events (probably off-shift to minimize the impact to other users): 1. Reboot your host server in order to eliminate any problems specific to the operating system such as unflushed buffers, unreleased shared memory, etc. 2. Determine that the syslog is up and running and that can log the warning and error messages issued by the CMVC daemons. You may do this verification by issuing the following command that will cause a security warning: su root password: xxxx # Give a wrong password in order to fail the "su" command. 8 CMVC FAQ: server One good reason to issue the following security warning is that you will have a baseline in your syslog, and you can use it as reference point in case subsequent actions cause the logging of many messages by the CMVC daemons, the database system or the operating system: a. Create a security warning that should be logged in the syslog if it is properly configured. One such security warning is to try to login as user "root" with the wrong password (you have to try from a user that is not root): su root b. Look at the bottom of the syslog file to see if this warning was logged. In AIX, the default syslog file is /var/spool/syslog: tail /var/spool/syslog c. If you see a warning message such as "BAD su from &ellip" at the bottom of the syslog, then the CMVC warning/error messages will be logged too. 3. Logon as the CMVC family administrator. 4. Stop the family daemons: stopCMVC yourCMVCfamily 5. Run the "tail -f" command on the syslog to find out which is the timestamp of the last message in the syslog. This will serve as a point of reference in the syslog. 6. Start the cmvc daemons with just 2 daemons: cmvcd yourCMVCfamily 2 7. Do "ps -ef | grep cmvcd" to see that you have 4 entries: 2 are for the initial 2 parent daemons, and the other 2 are for the new 2 children daemons. 8. Start the "monitor 1" CMVC command (to refresh every second). This command will show you the status of each of the daemons, and if they are processing requests. For more information on the monitor command, see 2.10, "Use the monitor tool to supervise the activity of the server daemons" on page 13. 9. From another screen or terminal, issue several CMVC commands, such as "Report -view Users". 10. From another screen or terminal, look at the bottom of the audit/log file. This is useful when the transaction happens very fast and it is not possible to see it in the monitor screen: cmvcd daemons 9 tail -f audit.log 11. At the same time look at the monitor output, and it might be possible to see when a daemon is processing the request. In some cases the request is processed so fast that the monitor will not show it. However, one good indicator is the number of hits in the top portion of the monitor output, it should increment by 1 every time you issue the Report command. 12. Press any key when looking at the monitor output. This will stop it. 13. Do "ps -ef | grep cmvcd", Do you see still 4 entries? 14. Stop again the family daemons: stopCMVC yourCMVCfamily 15. Look at the tail of the syslog: are there any warning or error messages since the last time? If there are messages, then try to solve them. If there are database errors, then consult the documentation for your database. 2.8 HOW TO DEBUG PROBLEMS WITH THE CMVC DAEMONS (PART 2) QUESTION: How to debug problems with the CMVC daemons (part 2) ANSWER: This section summarizes some secondary steps that you can take to better diagnose the problems with the CMVC daemons, in case they seem to suddenly die or behave in an unusual manner. In case the normal procedure mentioned in 2.7, "How to debug problems with the CMVC daemons (part 1)" on page 8 does not provide additional clues, try the following suggestions: o Whenever you explicitly specify 1 or more cmvcd daemons, they are brought up in background. However, if the above sequence fails to provide you with more clues, then you may try to bring up your family in the foreground: cmvcd familyName 0 o Ensure that your family is properly defined in the /etc/hosts and /etc/services in server host. The family should be 10 CMVC FAQ: server uniquely identified and the port number used by the family should not be used by another application. o It might be possible that another application is hard-coding the same port number used by your family. If you are using a port number that would be easier to specify such as a round number "3000", try to change it to another number such as "3001" or "3124". o If the cmvcd fails only when specifying multiple daemons, then you may take a look at 2.9, "Server daemon fails to start when specifying multiple daemons" on page 12. o Determine when was the last time the system was rebooted by using the following command: uptime Overtime, it might be possible that some buffers, files, processes were not released properly by applications running on this host. A reboot of the system will usually clean up these unreleased resources. NOTE: Thus, it is recommended to reboot the system period- ically, at least once a month. o If it is difficult to reproduce a problem with the current set of daemons, it is recommended to stop the CMVC daemons and restart them in order to have a new set of daemons to reproduce the problem: 1. Stop the CMVC daemons by using the stopCMVC command (which should take less than 5 minutes to perform) and then restart the daemon and try to issue CMVC commands. In case the stopCMVC seems to hang, then do the fol- lowing: - Identify the process id of all the active CMVC proc- esses: ps -ef | grep cmvcd ps -ef | grep notifyd - Kill all the CMVC processes (cmvcd and notifyd) by using: kill -15 process-id-1 process-id-2 ... The signal 15 is the preferred signal to kill the daemons because it causes a graceful termination. cmvcd daemons 11 If "kill -15" does not seem to kill the processes, then use "kill -9" which forces the termination and does not allow for a graceful stop. - Issue the previous "ps -ef" command to verify that the CMVC processes are no longer running. - Issue the following command to release other CMVC resources, such as shared memory, semaphores and queues: stopCMVC familyName Notice that at this point there should not be any active CMVC processes running. 2. Restart the daemons: cmvcd familyName 2 2.9 SERVER DAEMON FAILS TO START WHEN SPECIFYING MULTIPLE DAEMONS QUESTION: The CMVC server daemon fails to start when specifying multiple daemons but it works fine when specifying only one daemon. ANSWER: For more information on debugging problems with the CMVC family server daemon see 2.7, "How to debug problems with the CMVC daemons (part 1)" on page 8. Because only "multiple daemons" is not working, you may look at the following: o Check for processes that are getting killed by the system. Use the "errpt" command to see if there are processes getting killed. Hint: the output from "errpt -a" provides much more detail. o If you do not have your system set up to allow enough proc- esses for each user, use "smit" to change the system config- uration to allow for more concurrent processes (at least 30). o Ensure that you have enough licenses for the database. You need at least 1 license for each daemon that is run. The CMVC Server Administration manual recommends that you should have an extra database license and family daemon for 12 CMVC FAQ: server miscellaneous commands/queries that are run by the family administrator. o Ensure that you have enough space in /var, /tmp, and the file system that contains the family account (such as /home). You want 3 times the largest possible file to be checked-in for /tmp, at least 2-3 Mb for mail, syslog, and other files that grow in /var, and tons of space for the family account. o Reboot your machine and then cleanup /tmp, if needed. 2.10 USE THE MONITOR TOOL TO SUPERVISE THE ACTIVITY OF THE SERVER DAEMONS QUESTION: Use the monitor tool to supervise the activity of the server daemons. ANSWER: Use the CMVC tool "monitor 1" to see the activity of all the server daemons. This tool identifies each server by its process id (the second number from the left), and what is the request that each server is performing. To exit the monitor tool simply press the enter key. To recycle a daemon that is doing a task that is not appropriate, such as a query of all the files (1=1) which is usually not needed and takes too many resources, or a task that was killed by the client and thus there is no point for the server to keep working on it, then issue the signal -1 to gentle recycle the daemon: kill -1 process-id If the signal -1 does not seem to work, then try the signal -15 and if this one does not work, then try -9. cmvcd daemons 13 2.11 DO NOT CHANGE THE FILE PERMISSIONS NOR THE OWNERSHIP OF CMVC FILES QUESTION: A customer modified the ownership of the executable of cmvcd in /usr/lpp/cmvc/bin from root:system to bin:bin (probably by mistake) and this action caused a LOT of problems that were dif- ficult to diagnose, such as messages in the syslog about not being able to open the audit/log, error message 0010-063 with chown() when creating or checking a file, and problems with fchown() during a release extract. ANSWER: NOTE: Please, do not change the file permissions or the owner- ship of any of the CMVC executables. For your information, the cmvcd must have: -rwsr-xr-x 1 root system /usr/lpp/cmvc/bin/cmvcd* While the other executables must be: -rwxr-xr-x 1 bin bin /usr/lpp/cmvc/bin/Report* 2.12 WHEN STOPPING A CMVC FAMILY, USE THE STOPCMVC SCRIPT QUESTION: When stopping a CMVC family, use the stopCMVC script. ANSWER: The recommended process for stopping a CMVC family is to use the stopCMVC script, which takes care of using "kill -15" against each family cmvcd daemon and the notifyd daemon, and clears up the shared memory, semaphores and process queues used by the family. NOTE: Please do not use "kill -9" to stop a cmvcd daemon because it does not give the daemon the opportunity to do a graceful shutdown, as does "kill -15". In case stopCMVC does not kill all the cmvcd processes, it might be possible that a Release/Level -extract action is in progress or is hanging. This action (and other similar actions) needs to do something as a particular user id and not as the family id; therefore, the cmvcd server working on this action spawns a new process owned by user "root" and changes the ownership to the appropriate user id. That is the reason the owner of the 14 CMVC FAQ: server /usr/lpp/cmvc/bin/cmvcd file is "root:system" and the file per- missions have the s-bit as follows: "-rwsr-xr-x". Therefore, the primary reasons that many families are seeing cmvcd processes hanging around and owned by root is that they are: o Started by an OS/2 client. o Then, the OS/2 client process is killed by the user, the box hangs, etc. o The OS/2 NFS daemon either hangs or is killed as a result. o The server daemon hangs because it cannot perform a umount. o Because the killing of a client process does NOT kill the corresponding server process (CMVC does not have a mechanism for communicating a kill message), there is no way for the client user to get rid of this daemon. One suggestion is to run a series of "cron" jobs in this order: 1. As each family: run stopCMVC against each family 2. As root: grep for any cmvcd processes still remaining, then kill them using first "kill -15" and if that does not work then use "kill -9". 3. Do your normal backup for the CMVC family account and the corresponding database. You could use the backupCMVC script as a starting base. 4. Restart the cmvcd and notifyd daemons. 2.13 HOW TO DEAL WITH CMVC PROCESSES SHOWING QUESTION: I have restarted my families using cmvcd and after a while I checked the processes for the family by using "ps -fu familyName". The resulting list does no longer contain the cmvcd processes but a lot of entries like sfw 21240 1 0 0:00 What can I do ? ANSWER: If the parent process for all the defunct processes is 1, it may mean that inittab has not exited. Check in /etc/inittab for all cmvcd daemons 15 entries with 'wait' or 'once'. Then run 'who -a' and look for these processes. They should all say 'exit=0'. Investigate (or kill) any that have not exited. When you kill the hanging process, all the defunct ones will probably go away. 2.14 STOPCMVC: CONFLICT WITH DB2 INSTANCE AND CMVC FAMILY NAMES QUESTION: We are using CMVC with DB2. We created a new DB2 instance called 'cmvc23'. Next, I created CMVC family called 'cmvc23'(same DB2 instance). I ran "db2start", "cmvcd cmvc23 3" and "notifyd". There are no problems at this point. However, when I ran "stopCMVC cmvc23", besides killing the CMVC notifyd daemon and cmvcd daemons, all the DB2 instance's daemons were killed at same time. Is this a bug or not? ANSWER: Your problem is caused by the stopCMVC script and it occurs because the name of the DB2 instance is the same as the family name. The stopCMVC script uses "ps -ef" to identify the CMVC daemons that are logged, it is possible to misidentify some other proc- esses that have the "cmvc" string in them, such as a user "cmvcdemo", "cmvca" (for user CMVC "A"), "cmvc23", a DB2 instance called "cmvc", etc. You will have to set the DB2 instance with a different name to overcome the problem. 16 CMVC FAQ: server 3.0 NOTIFYD AND MAIL NOTIFICATION 3.1 WHAT IS THE INTERACTION BETWEEN THE CMVC SERVER AND NOTIFYD? QUESTION: What is the interaction between the CMVC server and notifyd? ANSWER: The CMVC Server's notification subsystem uses the UNIX sendmail facility to deliver mail to users. This subsystem consists of the notifyd daemon (a background process) and its connection to a database shadow process. The server daemon processes initiate a notification during the processing of most CMVC successful actions. A message and a limited set of delivery instructions are placed in the CMVC family's $HOME/queue/messages and $HOME/queue direc- tories respectively. The notifyd daemon periodically reads the $HOME/queue directory looking for instruction files to process. After reading the instruction files, a query is made to the database to resolve the notification requests into a list of addresses suitable for the sendmail command. The actual message to be used is obtained from the $HOME/queue/messages subdirectory. Any workstation or host that is able to receive mail in SMTP (simple mail transfer protocol) format can receive mail from a CMVC family. The notifyd daemon instructs the sendmail command to place the mail into the sendmail queue. The mail will be delivered the next time the sendmail queue is read, which is usually within half an hour. After the notifyd daemon has processed the sendmail command, the message and instruction files are removed from the family's UNIX account on the CMVC server. notifyd and mail notification 17 3.2 DIAGNOSING AND CORRECTING TYPICAL MAIL PROBLEMS QUESTION: Diagnosing and correcting typical mail problems ANSWER: The following are typical mail problems: o The mail for specific users is not being routed either because of a name resolution timeout or because the user is unknown. The family administrator should periodically check the mail queue for the CMVC family account, or should forward the mail from the CMVC family account to the CMVC administrator account in case it is different. Any mail that is not being delivered will eventually be rerouted back to the originator after the retry expiry period is reached. - Read the undelivered mail messages. The header shows the intended destination. - If 'Deferred: A route the remote host is not available' is displayed in the header, then this means that the user was identified but the connection to the host is tempo- rarily unavailable. The only way to avoid this situation is to chose a route that is guaranteed to be available all the time. - If 'Host Unknown' is displayed in the header, then this means that the host is not known on the network. This situation must be corrected immediately otherwise junk mail will accumulate. The family administrator should contact the person for whom this mail was intended and modify the user's mail address in the CMVC User ID. If the user no longer exists then the CMVC notification list and User ID for this user should be deleted. - Undelivered mail should be removed from the family's mail queue. o The mail is arriving but it is delayed. 18 CMVC FAQ: server If user's are complaining that the mail that they receive is outdated, then the sendmail queue processing time interval should be checked. Based on our experience, it is recommended to accept the default value of 30 minutes for sendmail to wake up and process the queue. We have noticed that a shorter interval tends to have a negative impact in the performance of heavily used families. The value can be modified in the /etc/rc.tcpip file and the sendmail daemon on the server can be restarted. 3.3 HOW CAN I MANIPULATE THE TEXT OF THE MESSAGE TO NEW USERS ? QUESTION: How can I manipulate the text of the message to new users ? ANSWER: The current implementation of the notification of CMVC messages does not provide the flexibility to alter the actual text. However, you can use a user exit for the action User Create and send an extra customized message to the new user. You could include in this message information such as: o Welcome to the CMVC family. o Notes about the installation of the CMVC client or related files. o How to modify the /etc/hosts or /etc/services file in order to use the short family name. o How to get answers to frequently asked questions about CMVC functions or the CMVC family. o How to contact the help desk in case of problems/questions. You can use the sample user exit checkPubsImpactDefectOpen1 which is available in 2.3.0.22. 3.4 HOW TO DIAGNOSE PROBLEMS WITH NOTIFICATION (NOTIFYD) QUESTION: How to diagnose problems with notification (notifyd). ANSWER: o Check that the notifyd daemon is running. notifyd and mail notification 19 o Check if the sendmail daemon is running in the CMVC server. o Check if the appropriate mail daemon/process is running in the CMVC client. If a client host is not powered on or if the mail daemons are not active, then the sendmail daemon from the CMVC server will give up the delivery of mail to that client (after a certain period). 3.5 SUBDIRECTORIES QUEUE AND QUEUE/MESSAGES IN THE CMVC FAMILY ACCOUNT QUESTION: There are many file names like "i.F8MIc5" with old dates in the subdirectory $HOME/queue of the CMVC family. What are these files? Why are still there? Can they be deleted? ANSWER: The queue and queue/messages subdirectories of a CMVC family account contain files that the CMVC notifyd daemon will use to send notifications from users. In our latest levels of CMVC, we fixed a bug in which certain CMVC actions (such as file add), created these queue files with the wrong ownership, which caused the notifyd to ignore them and leaving them in the queue subdirectories. The workaround is to change the file ownership of these files to the CMVC family account, then the next time that the notifyd is run, it will process and delete them. 3.6 NOTIFYD WILL NOT START WHEN USING ORACLE QUESTION: We run CMVC 2.3 on AIX 3.2.5. In the last few months, we noticed that lately, when we start cmvcd, the notifyd daemon no longer starts. In the syslog we get the following errors: /usr/lpp/cmvc/bin/notifyd61843Ù: cmvcmgr: 0010-061 Database error -1034 (ORA-01034: ORACLE not available ORA-07320: smsget: shmat error when tr), has occurred. If the error is the result of an invalid column name, then check the syntax of the command. If you cannot resolve the problem, contact the family administrator. 20 CMVC FAQ: server /usr/lpp/cmvc/bin/notifyd61843}: cmvcmgr: 0010-062 The database cannot be initialized. To solve the database problem, contact the family administrator. The database administration team tell us that this means that notifyd cannot see that the listener is running, when in fact it is running because all other applications on that box are working perfectly. We believe that the team put a fix on the system that has broken notifyd as this used to work fine. Does anyone have any ideas what the problem could be? ANSWER: In this case, it was discovered that due to a problem with the database, some changes were done to the Oracle executables, and it happens that the file permissions were changed incorrectly. They are supposed to be 6751 or -rwsr-s--x and without that, notifyd will not run. As soon as we changed the file permissions to the correct ones, notifyd started with no complaints! notifyd and mail notification 21 22 CMVC FAQ: server 4.0 CMVCARCHIVE AND CMVCRESTORE 4.1 WHAT IMPORTANT DETAILS SHOULD I KNOW ABOUT CMVCARCHIVE AND CMVCRESTORE? QUESTION: What important details should I know about cmvcarchive and cmvcrestore? ANSWER: o Make a complete backup of the CMVC family database and file system before doing any archiving/restoring activities. o The cmvcarchive and cmvcrestore commands are executed from the CMVC server. The CMVC family administrator must log in to the CMVC fami- ly's account on the CMVC server to run the programs. o The executables are found in /usr/lpp/cmvc/bin/cmvcarchive and cmvcrestore. o They are interactive, menu-driven programs. However, it is possible to create an answers file and execute the commands with the answer file via an "at" or a "cron" job. o The prerequisites must be met before running the programs. o Feedback messages are displayed to monitor the program status and progress. o The messages "Archive complete." and "Restore complete." indicate the successful completion of the programs. o The time to complete an archive or restore varies according to the size of the CMVC objects to be handled and on the system resources. In any case, these commands take significantly longer to execute than the every day commands from CMVC. o They cannot run unless the cmvcd server daemons are stopped or are running in maintenance (read-only) mode. cmvcarchive and cmvcrestore 23 Use the -m flag to start the cmvcd server daemons in mainte- nance mode. In this mode, the daemons will only accept Report and File -extract commands. o Keep in mind that cmvcd, cmvcarchive and cmvcrestore make use of system semaphores to guarantee that they are the only pro- grams accessing the database in update mode. Use IPCS and IPCS -S commands to check and ____ _____________________ clear semaphores when necessary. Under normal conditions, the semaphores will be set and cleared automatically. If the user receives a message indicating that the programs cannot be run (either because the cmvcd server daemons are running in regular mode or another cmvcarchive/cmvcrestore program is running), and if they are not running, then the semaphore has not been cleaned up from the last execution. In this case, use the IPCS -S command to clear _____________________ the semaphore whose id is owned by the CMVC family's account. IPC status from /dev/mem as of Mon Jun 14 11:40:34 EDT 1993 T ID KEY MODE OWNER GROUP Message Queues: Shared Memory: m 0 0x0d050207 --rw------- root system m 1 0x58067039 --rw-rw-rw- gray staff m 2 0x52564801 --rw-rw---- informix informix m 3 0x000000dc --rw------- cmvcdemo system Semaphores: s 0 0x4d07004b --ra-ra---- root system s 3 0x58067039 --ra-ra-ra- gray staff s 4 0x52564801 --ra-ra---- informix informix s 5 0x6c090c7b --ra------- cmvcdemo system 4.2 WHAT ADDITIONAL DETAILS SHOULD I KNOW ABOUT CMVCARCHIVE? QUESTION: What additional details should I know about cmvcarchive? ANSWER: o Archive options: - Estimate storage required for archive. Estimates the amount of local file system storage required to archive levels or releases. In many cases the value specified will overestimate the amount of storage required because an average size for defect and feature notes and file version remarks is used to calcu- late the estimate. 24 CMVC FAQ: server - Archive levels of a release up to and including a speci- fied level. When archiving levels, the users indicate the level they want to archive and the cmvcarchive program will archive that level and all levels committed prior to that level. In a sense, this is a baseline in which all work com- pleted prior to that time is archived. - Archive one or more releases. o Archive to a local file system or a device. Local file systems must be empty and accessible to the cmvcarchive program. Devices must be raw devices (for example, in AIX /dev/rmt0). When archiving to a device, files containing data being archived will be written in either the home directory of the family or a user-specified working directory prior to being written to the device. When archiving to a local file system, the version control files are cleaned up during the archive operation. When archiving to a device, the version control files are cleaned up during the restore operation. o Optional removal of archived objects from CMVC. Removal of objects allows the user to reclaim database and file system space. Users should verify the archive VERY CAREFULLY prior to selecting the remove option. The remove option takes more time to complete than the normal archive. Archived objects are deleted from the database. Defects and Features associated with archived tracks are deleted if no other tracks are associated with them and all verification records have been marked. Level maps for archived levels are deleted from the $HOME/maps directory on the server. Version control files are deleted from the $HOME/vc tree if a release archive is performed and all common/shared files are archived. o Archive prerequisites must be satisfied. o Creates files containing data required for restore. o Recommendations: - Backup your CMVC family before attempting to do a CMVC archive. - Perform an archive without the delete action and verify the restore. If the restore is successful, then perform the archive with the delete action. cmvcarchive and cmvcrestore 25 4.3 WHAT ARE THE PREREQUISITES FOR CMVCARCHIVE? QUESTION: What are the prerequisites for cmvcarchive? ANSWER: o General Prerequisites: - CMVC Server at V2R1 or later. - cmvcd not running or running in maintenance mode. o Level Archive Prerequisites: - Level exists in the complete state for the release. - Previously committed levels must be in complete state. - Common/shared files are not locked (PVCS systems). - Tracks that are levelMembers are complete. - The levels that are not being archived must not include any tracks that are being archived (for example, a track is included in one level that is being archived and in another level that is not being archived). o Release Archive Prerequisites: - Releases exist; levels for the releases are complete. - No files are locked for the releases. - All tracks are complete for the releases. - Sizing records for the releases are marked. 4.4 WHAT ARE THE FILES CREATED BY CMVCARCHIVE? QUESTION: What are the files created by cmvcarchive? ANSWER: The following files are created by cmvcarchive: o CMVCarchive.info It contains information related to the archive (for example, date and time, objects archived). It is used during the restore phase to validate prerequisites. o CMVCarchive.dbData 26 CMVC FAQ: server It contains archived object data. It is used during the restore phase to populate the database. o CMVCarchive.export It is created when database=ORACLE. It contains export of the Notes and Versions database tables of the family; it is imported during restore. o CMVCarchive.notesData It is created when database=INFORMIX/SYBASE: - INFORMIX -> holds notes data for archived defects and features. - SYBASE -> holds bulk copy of the Notes database table. It is imported during restore. o CMVCarchive.versionsData It is created when database=INFORMIX/SYBASE: - INFORMIX -> holds versions data for the deltas of archived files. - SYBASE -> holds bulk copy of the versions table. It is imported during restore. o CMVCarchive.users It contains the list of CMVC logins in archived data. It is used upon restore to verify the first superuser of the family. o CMVCarchive.Vccleanup It is created when archiving to a device. It contains a list of deltas to be removed from the version control files upon restore. Format of file is: sourceId : fileType SID SID SID sourceID : fileType SID where - sourceID is 0-6 digit number representing the location of the file in the version control tree (for example, 5 = $HOME/vc/0/0/0/0/s.05) - fileType = 0 for text; 1 for binary - SID = delta that is to be removed from the SCCS/PVCS file during the restore cmvcarchive and cmvcrestore 27 o CMVCarchive.PVCS_clean It is created in the family's home directory when PVCS is used by the family and the user selects the 'remove archived objects' option. The user will have to manually delete the version control files listed in this file because the cmvcarchive program does not have the permission to do so. o CMVCarchive.CPIOlist It is created with archiving to a device. It contains a list of files to be cpio'ed to the device (by using the "cpio" Unix utility). This file will also be cpio'ed to the device. Also used during restore to determine where the files get deposited when extracted from the device. 4.5 WHAT ADDITIONAL DETAILS SHOULD I KNOW ABOUT CMVCRESTORE? QUESTION: What additional details should I know about cmvcrestore? ANSWER: o Restore options: - Restore from a local file system. - Restore from a device. o All archived data is restored (that is, there is no selection mechanism). o Data is restored to a NEW CMVC family. ___ o Restore prerequisites must be satisfied. o The database and the family's file systems are populated with archived data. o The configurable fields are set. 4.6 WHAT ARE THE PREREQUISITES FOR THE CMVC FAMILY TO BE RESTORED? QUESTION: What are the prerequisites for the CMVC family to be restored? ANSWER: 28 CMVC FAQ: server o CMVC Server at V2R1 or later. o cmvcd not running or running in maintenance mode. o The restored family will be new (mkfamily and mkdb executed only). o The first super user is not a member of the archived data (check CMVCarchive.users) o No configurable fields are set for the new family. o The $HOME/vc and maps directories are empty. o The $HOME/vc file system owned by the family id until after the cmvc restore. PVCS users must switch the owner of $HOME/vc to the PVCS owner AFTER the restore is done. 4.7 IF HAVING PROBLEMS WITH CMVCARCHIVE, THEN MIGRATE TO CMVC 2.3.0.22 OR HIGHER QUESTION: If having problems with cmvcarchive, then migrate to CMVC 2.3.0.22 or higher. ANSWER: Because we have implemented many fixes to cmvcarchive/cmvcrestore between the refreshes 2.3.0.18 and 2.3.0.22, it is recommended that you should upgrade to the latest refresh of CMVC if you are planning to use these functions. 4.8 CMVC IS NOT RECLAIMING FILE SYSTEM SPACE WHEN DOING CMVCARCHIVE QUESTION: CMVC is not reclaiming file system space when doing cmvcarchive. ANSWER: This is a clarification of the term "reclaiming space" when doing a cmvcarchive: cmvcarchive and cmvcrestore 29 In order to remove an archive from the vc tree, NO rows in the Files table can point to the archive (or archive directory in the case of binary files). The utility cmvcarchive will delete an archive if, and only if, ALL entries in the Files table that point to a given archive are specified. In other words, you have to delete all releases that point to an archive file at the same time in order to get rid of the archive. Because this is not an easy thing, few archives are actually deleted. 4.9 CMVCARCHIVE IS TAKING A LONG TIME TO RUN QUESTION: I am running cmvcarchive and it is taking a long time to run. ANSWER: Archiving to tape is much slower that archiving to disk. We have had a few instances where tape archives have failed after long periods because of problems in reading/writing from/to the tape. We do not know of any instances where the archive has just hung; we know that in doing an archival of extremely large releases, the action has taken several days to run. Because it takes so long, we have added some code to warn you when deletion begins (if you are using that option), so you know when you have reached the point of no return. It also turns out that the beginning of the deletion phase is about the halfway point. If you do not want to keep all of the releases installed together at any one time, then archive them separately. This will also use less space and will reduce the development downtime, so that you might be able to archive to disk, then tar off the files to tape as a separate step. We wish that we could make things faster, but the queries and other processing are simply very time consuming. 30 CMVC FAQ: server 4.10 SEVERAL PROBLEMS WITH CMVCARCHIVE QUESTION: 1. I recently performed an archive of a CMVC release and had a failure during the archive. One suspicion is that disk space was becoming an issue with the system. 2. The cmvcarchive stated that it needed 161 MB to archive. The file system had 250 MB of space so I proceeded. I suspect that the database files need additional space to archive also. 3. Because CMVC is down during the cmvcarchive process, what error logs are used?. 4. I am having problems when specifying multiple arguments to cmvcarchive, because I seem to be reaching a limit as to how many releases I can input or at least how many arguments I can specify. ANSWER: 1. The tool cmvcarchive performs a little of everything in CMVC. As a result, there are lots of things that can go wrong: o Running out of space in /tmp (for the SCCS commands). o Running out of space in the database transaction log (this is one thing cmvcarchive does inefficiently). o Running out of space in real memory (for the query). o Running out of space in the target directory. This is a difficult one to watch for without a background process, since cmvcarchive cleans up space as soon as it fails. However, the problems are displayed on the screen. 2. Our experience is that the size estimate has always been larger than necessary. This is not a guarantee, but we have not yet seen an underestimate of the required space. 3. Errors are written to standard error, that is, to the screen. 4. The problem that you are experiencing is actually in the shell, not with cmvcarchive. In order to circumvent the lim- itations of the number of characters entered at a prompt, you need to redirect the input from a file: a. Do the steps manually to determine what you normally enter. b. Put your answers in a file. Below is the contents of one such file: cmvcarchive and cmvcrestore 31 2 2 plink 1 1 /tmp/plink c. Run your cmvcarchive as follows: cmvcarchive < answers 2>&1 | tee cmvcarchive.out NOTE: It is recommended to run it via an "at" or a "cron" job to execute it automatically during off-shift hours to minimize the impact to other users of the family. 4.11 HOW TO FIND OUT A MISSING PATH ID (DUE TO CMVCARCHIVE) QUESTION: When running cmvcarchive there is an error about a missing path id. How to overcome the problem? ANSWER: To overcome the problem do the following: 1. Using SQL from UNIX command line: Create a file with name xxx.sql (.sql required by Oracle). Here is a query (and exit) that will find all occurrences of missing Path table records (again, Oracle syntax): select f.id, f.pathId, f.baseName, r.name from Files f, Releases r where f.releaseId = r.id and r.name = '&CMVC_RELEASE' and f.pathId not in (select p.id from Path p where p.id = f.pathId); quit; There are important notes to consider: o Using Oracle syntax, the user will be prompted for the value of CMVC_RELEASE when the query is run. o You can change the syntax of the above query to look for a list of releases by replacing: r.name = '&CMVC_RELEASE' With: 32 CMVC FAQ: server r.name in (&CMVC_RELEASES) Of course, you will then need to put apostrophes (') around each release in the set. o Run the query (where xxx is the sql file created previ- ously) sqlplus user/password @xxx or sqldba lmode=y @xxx 2. Fix the problem: a. Logon as the CMVC family administrator. b. Connect to the database, such as: sqlplus user/password Without this step, you need to add the familyname in front of each table (for example, "select * from Fam.Files") c. Issue the following insert command (one single line): insert into Path (id, name) values (21799, 'src/dubparms.h'); "src/" is enough of a path d. Verify that the file is now available in FileView with: Report -view FileView -where "pathId=21799 and releaseName=r020" e. Run an archive check. cmvcarchive and cmvcrestore 33 34 CMVC FAQ: server 5.0 USER EXITS 5.1 WHAT ARE USER EXITS? QUESTION: What are user exits? ANSWER: User exits are programs or shell scripts written by users (such as the CMVC family administrator) that are executed by the CMVC server during the course of a CMVC action. User exits provide the ability to extend the functionality of CMVC to trigger non-CMVC processing during the handling of CMVC transactions. There is no support for user exits in the CMVC clients. The user exits do NOT allow interaction with the end-user. Family administrators can configure user exits for most of the CMVC actions. o To establish user exits, the family administrator must modify the USEREXITS file found in the $HOME/CONFIG directory on the CMVC server. When modifying the USEREXITS file: - Read the comments in the file and follow the definition format. - Only define one user exit per exit id. If you define multiple user exits for the same CMVC action and exit ID, then only the last one in the file will take effect. - The format of the definition file is: CMVCaction ExitID UEprogram UEparameters #Comments o The user exit programs themselves must be placed in the $HOME/BIN directory on the CMVC server. When writing user exit programs or shell scripts: User exits 35 - Remember to return a 0 exit status if everything is OK and you want the CMVC action to continue. - Remember to return a non-zero exit status if everything is not OK and if you do not want the CMVC action to con- tinue (only for exit ID's of 0 and 1) - If the user exit program is not behaving as you expected, try echoing or printing all of the input arguments that the user exit is receiving from the CMVC server. This will help you determine if you are interpreting the correct input arguments. - It is not recommended that you invoke another CMVC command from within your user exit program. This could lead to a deadlock situation. Some workarounds are to issue CMVC commands as an "at" job (in that way there will be no conflict with the current CMVC command being executed) and to execute shell scripts that use SQL statements to perform database queries directly to the database. o If you created or modified user exits while the cmvcd daemons are running, then you will have to restart the cmvcd daemons in order to get them to acknowledge the user exits. The cmvcd server processes read and initialize the user exit points when they start up. 5.2 SHOWING A SMALL USER EXIT EXAMPLE QUESTION: Showing a small user exit example ANSWER: The following is an example of the userExits definition file in the $HOME/CONFIG directory on the CMVC server: # # FileAdd User Exits # FileAdd 0 Check_Date Monday, Feb 3, 1991 FileAdd 1 Check_Date Monday FileAdd 2 Check_Date 02/03/91 # 36 CMVC FAQ: server The following is an example of the 'Check_Date' user exit. This particular example is a shell script that just echoes the input arguments that it receives. #!/usr/bin/ksh # # Check_Date User Exit # print "This is the Check_Date CMVC/6000 User Exit print "User Exit Name = $0" for i in $* do print $i done exit 0 5.3 WHAT ARE THE TYPICAL PROBLEMS WITH USER EXITS? QUESTION: What are the typical problems with user exits? ANSWER: Some situations that may cause problems with user exits are: o $HOME/bin is not in the CMVC family's PATH. o The user exit programs in $HOME/bin are not executable. o The user exit programs in $HOME/bin are not owned by the CMVC family. o The cmvcd daemons were not restarted after creating or modi- fying the $HOME/config/userExits file. o If using Bourne shell, the "shift" statement is not used to get the positional parameters greater than 9. o If using Korn shell, use the syntax ${1} to ${10}. 5.4 WARNING ON USING CMVC LINE COMMANDS INSIDE CMVC USER EXITS QUESTION: Warning on using CMVC line commands inside CMVC user exits ANSWER: User exits 37 Even though the CMVC Server manual implies in page 88 (V2.3 edition) that it is OK to use the CMVC Report command inside user exit, it is possible to cause deadlocks in these situations if there are not enough available CMVC daemons to process the Report command. Thus, to avoid these deadlocks, ensure that you have enough data- base licenses for the proper amount of CMVC daemons. 5.5 NEW PARAMETERS FOR USER EXITS IN CMVC COMMANDS, STARTING WITH 2.3.0.22 QUESTION: New parameters for user exits in CMVC commands starting with CMVC 2.3.0.22. ANSWER: The user exit parameters passed in the following CMVC commands have been updated. Below is the new parameter, trackList, shown with the parameter before and after it in the updated list. The new parameter, trackList, is a string that contains the track names of all features/defects specified in the CMVC command. See "Appendix H: User Exit Parameters" of the "CMVC Server Admin- istration and Installation" manual for the previous version of the complete parameter list. Also, as stated in Appendix H, the new parameters for ExitID 3 are the same as ExitID 0. The new parameter has NOT been added to the end of the parameter list, but it has been inserted between existing parameters; for example, with the command File -create (action FileAdd), in user exit id 0, the new parameter "trackList" has been inserted between "remarks" and "fileMode"; thus, the list of parameters for this user exit before CMVC 2.3.0.22 was: 1 file path name 2 temp file on server 3 release name 4 component name 5 file type 6 remarks 7 fileMode 8 effective CMVC user ID 9 real Unix login 10 verbose flag But after CMVC 2.3.0.22 is: 38 CMVC FAQ: server 1 file path name 2 temp file on server 3 release name 4 component name 5 file type 6 remarks 7 trackList 8 fileMode 9 effective CMVC user ID 10 real Unix login 11 verbose flag The user exits that are affected are shown below. 5.6 APPARENT PROBLEM WHEN PASSING MORE THAN 10 PARAMETERS TO A USER EXIT QUESTION: Some CMVC family administrators report that there is a problem when passing more than 10 parameters to a user exit. Some of these parameters appear to be wrong, particularly the ones that correspond to the positional parameters 10, 11, and so on. ANSWER: The Bourne shell only supports positional parameters $0 through $9. In order to access parameters 10 or greater, you must use the "shift" command. Shift will move $2 to $1, $3 to $2, etc. ($0 is the name of the command being invoked and never shifts). Also, you can shift more than one position at a time. For example, you can use $1 through $9, then issue the command "shift 9" and process variables 10 through 19 using the variables $1 through $9 again. The Korn shell supports positional parameters greater than $9 using braces. For example, you can access the eleventh posi- tional parameter as ${11}. 5.7 THE TEMP FILE FOR USER EXIT 2 FILECHECKIN IS NO LONGER AVAILABLE QUESTION: The temp file for user exit 2 FileCheckIn is no longer available. ANSWER: User exits 39 CMVC Command CMVC Exit After Parm., New Parameter, Before Parm. Action ID File -create FileAdd 0 remarks, trackList, fileMode" File -create FileAdd 1 remarks, trackList, fileMode File -create FileAdd 2 remarks, trackList, fileMode File -checkin FileCheckIn 0 remarks, trackList, common flag File -checkin FileCheckIn 1 remarks, trackList, common release name File -checkin FileCheckIn 2 remarks, trackList, common release name File -delete FileDelete 0 force flag, trackList, common flag File -delete FileDelete 1 component name, trackList, effective CMVC user ID File -delete FileDelete 2 component name, trackList, effective CMVC user ID File -link FileLink 0 SID, trackList, effective CMVC user ID File -link FileLink 1 component name, trackList, effective CMVC user ID File -link FileLink 2 component name, trackList, effective CMVC user ID File -migrate FileMigrate 0 component name, trackList, SID File -migrate FileMigrate 1 component name, trackList, SID File -migrate FileMigrate 2 component name, trackList, SID File -recreate FileRecreate 0 force flag, trackList, common flag File -recreate FileRecreate 1 force flag, trackList, common flag File -recreate FileRecreate 2 force flag, trackList, common flag File -rename FileRename 0 force flag, trackList, common flag File -rename FileRename 1 force flag, trackList, common flag File -rename FileRename 2 force flag, trackList, common flag File -undo FileUndo 0 force flag, trackList, common flag File -undo FileUndo 1 force flag, trackList, common flag File -undo FileUndo 2 force flag, trackList, common flag Release -link ReleaseLink 0 date, trackList, effective CMVC user ID Release -link ReleaseLink 1 date, trackList, effective CMVC user ID Release -link ReleaseLink 2 date, trackList, effective CMVC user ID Figure 1. Updated parameter list for user exits Even though the CMVC manual describes that the user exit ID 2 for FileCheckIn passes the temporary file name, at this stage the real file is gone! However, when using the user exit ID 1 is fine, that is, the real file is still available. 40 CMVC FAQ: server #!/usr/bin/sh # # This is a Bourne shell example - positional parameters # ------------------------------------------------------------------ # Giving legible names to the input parameters # It is nice to list ALL the parameters, it helps with the # debugging and overall understanding. # NOTE: It is important to deal with the "shift" command to # handle the input positional parameters greater than 9 # ------------------------------------------------------------------ # print "parameters passed: " # print "1 is component name: " $1 # print "2 is prefix: " $2 prefix=$2 # print "3 is severity ID: " $3 # print "4 is reference: " $4 # print "5 is environment name: " $5 # print "6 is remarks: " $6 # print "7 is level name: " $7 # print "8 is abstract: " $8 # print "9 is release name: " $9 shift 9 # print "10 is configurable field string: " $1 string=$1 # print "11 is defect number: " $2 defectNumber=$2 Figure 2. Sample Bourne shell using positional parameters 5.8 ARE THERE ANY REAL EXAMPLES OF USEREXITS SHIPPED WITH CMVC? QUESTION: Are there any real examples of UserExits shipped with CMVC? ANSWER: Some CMVC users would like to do some extra activities during the processing of CMVC commands at the CMVC server host and the users can exploit the feature called "user exits" (there are no user exits in the CMVC clients). In CMVC 2.3 we added in the directory /usr/lpp/cmvc/samples several samples of user exits. For more details, you can take a look at the file "server.samples" in the mentioned directory. User exits 41 #!/usr/bin/ksh # # This is a Korn shell example - positional parameters # ------------------------------------------------------------------ # Giving legible names to the input parameters # It is nice to list ALL the parameters, it helps with the # debugging and overall understanding. # ------------------------------------------------------------------ # print "parameters passed: " # print "1 is component name: " ${1} # print "2 is prefix: " ${2} prefix=$2 # print "3 is severity ID: " ${3} # print "4 is reference: " ${4} # print "5 is environment name: " ${5} # print "6 is remarks: " ${6} # print "7 is level name: " ${7} # print "8 is abstract: " ${8} # print "9 is release name: " ${9} # print "10 is configurable field string: " ${10} string=$10 # print "11 is defect number: " ${11} defectNumber=$11 Figure 3. Sample Korn shell using positional parameters 42 CMVC FAQ: server 6.0 ERROR HANDLING AND AUDIT LOG 6.1 WHAT DETAILS SHOULD I KNOW ABOUT ERROR MANAGEMENT? QUESTION: What details should I know about error management? ANSWER: When a cmvcd or a database error occurs, a message is displayed in the server's syslog (/usr/spool/syslog) or on the server console. The database performs a rollback of the command and the server process is reset so that it can receive additional client requests. The client does not know that the database error has occurred. It only knows that the connection to the server was lost (a message is displayed on the client side indicating that the connection is lost). Reports are a special case of database error since the user is able to type any SQL selection query. In this case, the database error is returned to the client so that the user can correct the error. When errors occur due to problems other than the database, a message is generated and returned to the client. The message is displayed through stderr. The CMVC server and line command errors are identified by the 0010-XXX message number. The CMVC GUI errors are identified by the 2059-XXX message number. 6.2 WHERE ARE THE ERROR MESSAGES FROM THE CMVC DAEMON AND DATABASES? QUESTION: Where are the error messages from the CMVC daemon and databases? ANSWER: The error messages from the CMVC daemon and the database are logged in the syslog. Error handling and audit log 43 You will need to enable the syslog facility, as explained in the technical report "CMVC frequently asked questions: handling OEM platforms and databases", section "Where are the error messages from the CMVC daemon and databases?" 6.3 HOW TO GET MORE INFORMATION FROM DB2 ERROR NUMBERS QUESTION: Sometimes there are DB2 error numbers shown in the syslog or in CMVC error messages. Is it possible to get more on-line informa- tion about them? ANSWER: To obtain more on-line information about a DB2 error number, such as "-818", you may use the following DB2 command: db2 ? sql---- Where ---- are the 4 digits of the error number. In case the number has only 3 digits, such as 818 or -818, then simply add a zero to the left of the number, such as: db2 ? sql0818 In this example, the output is: SQL0818N A timestamp conflict occurred. Explanation: The timestamp generated by the precompiler at precompile time is not the same as the timestamp stored with the package at bind time. This problem can be caused by the following: o Precompile, compile, and link without doing an application bind. o Precompile and bind without doing the program compile and link. o Bind the application using a bind file that resulted from a different precompile of the program than the precompile that produced the object module linked into the application module. o Bind an application with the same name as an existing plan and execute the existing (old) application. The statement cannot be processed. 44 CMVC FAQ: server User Response: Bind the application again, using the bind file for the program that matches the object module. Or, execute the program that corresponds to the package stored in the database. If installing the sample database, record the number and text of this message and then contact your technical service representative. sqlcode: -818 sqlstate: 51003 By the way, the CMVC utility "db2bind" can be used to bind the package files (*.bnd) to the database for CMVC. 6.4 ERROR MESSAGE: 0010-063 WITH KEYWORD BIND() QUESTION: Error Message: 0010-063 Error, Address already in use, occurred when the CMVC Server/6000 attempted to process the bind() function. ANSWER: 1. Situation A: dealing with a new family When creating and starting a new family, if this error appears, the most likely cause is that the port number for the new family is already being used by another service. Please take a look at the /etc/services file and ensure that the port number for the new family is unique and not in use by another service. Try to avoid "nice and round" numbers such as "3000", and try to use "odd" numbers such as "3125". We have encountered a couple of situations where a port number with a "nice and round" number was hard-coded in an application instead of being defined in the /etc/services" file. 2. Situation B: dealing with existing families, part 1 We have seen these errors when there is a conflict with the shared memory. We have added some cleaning up code in the stopCMVC sample in CMVC 2.3.0.18. In the meantime, please do the following steps: Error handling and audit log 45 a. Ensure that all the CMVC daemons are down: stopCMVC familyName b. Ensure that there are no "orphan" cmvcd daemons: ps -ef | grep cmvcd If there are "orphan" cmvcd daemons, kill them with "kill -15". c. Invoke the CMVC Activity Monitor which shows the activity of the CMVC daemons (at this point there should be none) and performs the cleaning of the shared memory (which is what we are trying to accomplish in this step). monitor 1 Press enter to terminate the monitor command. d. Restart your family: cmvcd yourFamily 2 e. Verify that the daemons are running: ps -ef | grep cmvcd In this example there should be 4 entries with cmvcd. 3. Situation C: dealing with existing families, part 2 When using an existing family and the situation above is not applicable, then this error may happen when stopping and restarting the cmvcd daemons; the most likely explanation is: What is probably happening is that you tried to restart the daemons, but the nfsmounthelper process is still attempting to do the NFS write. Therefore, the port or socket is already reserved for the communications between server and client. a. You have to re-ipl the server to fix the problem because you cannot restart the daemons and cannot unmount the file system in /tmp. b. You might have reasonable luck stopping NFS, perhaps via /etc/nfs.clean. c. Then "kill -1 1" to kick off the orphaned NFS process that got attached to init when you stopped NFS. d. Then restart NFS. If using AIX, this can be accomplished by either through SMIT or /etc/rc.nfs. 46 CMVC FAQ: server 6.5 CMVC ERROR MESSAGE 0010-063: NOT OWNER, CHOWN() FUNCTION QUESTION: When trying to create or checking a file, the following error gets placed in the syslog: 0010-063 Error, Not owner, occurred when the CMVC server attempted to process the chown() function, ... ANSWER: It appears that on some installations, /usr/lpp/cmvc/bin/cmvcd is owned by bin.bin instead of root.system. If you have this problem, please apply the following command (as user root): chown root.system /usr/lpp/cmvc/bin/cmvcd 6.6 ERROR MESSAGE 0010-063 - BROKEN PIPE, WRITE () FUNCTION QUESTION: The following message is received several times during a week in the /var/spool/syslog file: 0010-063 Error, Broken pipe, occurred when the CMVC server software attempted to process the write() function. ANSWER: One possible explanation is that the user closes the CMVC GUI by using the Close item from the UNIX pull down system menu (upper left corner), instead of using the function within the CMVC File->Closed selection. By the way, the UNIX Close function will cause the pipe to break. 6.7 HOW TO DEAL WITH THE 0010-250 ERROR MESSAGE QUESTION: CMVC client shows this error message: 0010-250 A connection cannot be established with family XXXX at node YYYY on port PPPP. The error is: Connection timed out. Error handling and audit log 47 ANSWER: This error message is ONLY issued when the CMVC client is trying to connect to the CMVC server via the TCP/IP sockets. Some possible workarounds are: o If this behavior is consistent: Probably the cmvcd server daemons are not running. Logon to the CMVC server and check the status by issuing: ps -ef | grep cmvcd Also you can use the monitor command: monitor 1 If the CMVC daemons are no longer running, check the errors in the syslog (in AIX, the default name is /var/spool/syslog) o If this behavior is intermittent: It might be possible that there are some problems with the underlying network and name server. You may issue the 'host' TCP/IP command in AIX and OS/2 (or the cmvchost command for SunOS, Solaris and HP-UX) for the server node (on the command line) before starting your client. For example: host serverName Then repeat the host command and this time use the IP address for this server which was shown in the previous command. If you do not get a good reply from 'host' for the machine name and IP address, the network still is having problems. 6.8 CMVC ERROR MESSAGE: 0010-256 QUESTION: CMVC error message: 0010-256 ANSWER: The 0010-256 error message is a generic (almost a catch-all) error message. The CMVC 2.3 Server manual says the following about this error (but there is additional information that was not included in the manual which is shown at the bottom of this note): 48 CMVC FAQ: server 0010-256 An error occurred when the CMVC server software tried to process function . It is possible that the network or the CMVC server is experiencing problems. EXPLANATION: The CMVC client software and the CMVC server software may be incompatible. Or your database file system may be full. This message is displayed on the CMVC client and on the CMVC server console, or entered into the syslog file if the syslogd daemon is running on the CMVC server. USER RESPONSE: o Verify that the version of the CMVC client software is the same as the version of the CMVC server software and that the port number designated to the CMVC family is unique on the network. o Check the amount of free space in the file systems; verify that your database file system has sufficient space for your CMVC server operations. o Message 0010-257 is also displayed on the CMVC client and on the console of the CMVC server, or entered in the syslog file if the syslog daemon is running. o Please check for: - Is your $HOME/vc tree full? - Is your database full? Check for the database specific files, such as rollback segments, table space, index space, etc. - Is your system table full? - Is the /tmp full? o Take a look at additional messages recorded in the syslog (see last page of the Chapter 14 from the CMVC 2.3 Server manual). The syslog recorded by syslogd usually gives you good hints. If you have not got it set up yet, you need to set it up and run the CMVC command again to catch the error in the syslog. 6.9 CMVC ERROR MESSAGE: 0010-381 (FOLLOWED BY 0010-256) QUESTION: When executing a CMVC command, I get the following 2 error mes- sages: Error handling and audit log 49 0010-381 The CMVC client software is not compatible with the CMVC server software for this action. Contact the family administrator to upgrade the CMVC client software installation. 0010-256 An error occurred when the CMVC server software tried to process function txInit. It is possible that the network or CMVC server is experiencing problems. ANSWER: Some possible answers, in the recommended order are shown below: 1. These errors will be shown when there is another application that is using the same port number as the CMVC family. Ensure that the port number for the CMVC family is not dupli- cated in the /etc/services file. Try to use another port number for the CMVC family and the CMVC_FAMILY environment variable, to eliminate the possi- bility that an application is using the same port number without registering it in the /etc/services file. 2. This is often a network problem in disguise. a. Access the CMVC family and issue "monitor 1" to monitor the commands that are issued against the family server. b. Then, from the client try a simple command: Report -testServer c. Verify from the monitor output if the number of hits was increased and if the command was shown in the output. 3. Sometimes, this is a system/database problem. Take a look at the tail of the syslog and see if there are any messages, if so, please take care of those messages. 4. There is a possibility that if you do not have enough space in the file system of the server or in the client, the 0010-381 error may show up. Check that the file space for the CMVC family account direc- tory is not greater than 90%! (As reported by the file system monitor command: "df" on AIX or "bdf" in HP-UX). Even though there was enough space for the database tables, and the home directory is not "full", the problem detected by 0010-381 might be present, and you may need to allocate more space. 5. If all the above fails, then the best guess is to reboot the system in order to clear up any possible dangling processes, unreleased shared memory and semaphores. 50 CMVC FAQ: server 6.10 WHAT DETAILS SHOULD I KNOW ABOUT AUDIT LOGGING? QUESTION: What details should I know about audit logging? ANSWER: Successful CMVC actions are logged in an audit log, which is found in the $HOME/AUDIT directory on the CMVC server. The name of the log file is LOG Unsuccessful transactions are not logged. The audit log can be used as a debugging aide to retrace the sequence of events that have successfully completed. Regular maintenance of the audit log is recommended as it will grow infinitely. See the CMVC Administration and Installation manual for maintenance tips. Regularly check the amount of free space in the file system that contains the audit log and allocate new storage if required. 6.11 AFTER RUNNING THE SAMPLE CMVCLOG.CLEANU, CMVCD DOES NOT LOG QUESTION: After I ran the /usr/lpp/cmvc/samples/cmvclog.cleanu command, my cmvcd does not log entries into the new $HOME/audit/log anymore. ANSWER: After you run the sample shell script to clean up your audit log file, remember to bring down cmvcd and then restart it in order to log entries into the new log file. Error handling and audit log 51 52 CMVC FAQ: server 7.0 DATABASE ISSUES 7.1 INDEXES THAT IMPROVE PERFORMANCE IN LEVELS AND IN FILE CHANGES QUESTION: Indexes that improve performance in levels and in file changes. ANSWER: One of the CMVC support groups has provided us with additional indexes that they used to improve performance for Level and File related commands. These new indexes were added to CMVC in 2.3.0.18. These indexes are NOT automatically added to existing CMVC databases, they are automatically added only to new databases. Thus, you may need to manually add these indexes. o Added in Level 017 -> GA of 2.3.0.18 (Level check and commit) create index XChaFilTrkVerTyp on Changes (fileId, trackId, versionId, type); commit; create index XLLvlTrk on LevelMembers (levelId, trackId); commit; create index XTrackIdStat on Tracks (id, state); commit; o Added in Level 006 -> GA of 2.3.0.0 (File information) create index XFilPid on Files (pathId); commit; create index XFilesSrcId on Files (sourceId); commit; Database issues 53 7.2 REORGANIZING AND REBINDING A DB2 DATABASE. QUESTION: What is the best way to reorganize and to rebind a DB2 database? ANSWER: After any major changes done to the CMVC database (such as migrating from a previous version/release of CMVC, performing massive file creations, or just after a period of heavy use), it is important to perform the following process for a DB2 database in order to update the statistics for the indexes. In that way, the performance of the database will not be affected: 1. Ensure that the CMVC family daemons are down. 2. Run REORGCHK. 3. Run REORG. You can run the CMVC "db2reorg" command discussed below. 4. Run RUNSTATS. 5. Finally, bind the DB2 plans to the CMVC executables by running the CMVC shell script "db2bind". For more information on the commands REORGCHK, REORG and RUNSTATS, see the DB2/6000 Administration Guide manual, Chapter "Performance Considerations", Section "Reorganizing Tables and Running Statistics". The following shell scripts are included that facilitate the maintenance of DB2 databases used with CMVC families: o db2reorg You can invoke the CMVC shell script "db2reorg" to run the DB2 REORG command for the CMVC family. This shell script was added in 2.3.0.18. The CMVC db2reorg shell script provides the following help: db2reorg Purpose: Run the DB2 REORG command for the CMVC family db2reorg Usage: db2reorg CMVCfamily workPath Notes: 1. CMVCfamily is typically the name of the family account 2. workPath is typically a temporary directory, such as /tmp o db2bind is needed to synchronize DB2 plans with database and CMVC executable files. 54 CMVC FAQ: server The CMVC db2bind shell script is needed to synchronize (bind) the DB2 plans with the DB2 database for the CMVC family and the CMVC executables. This means that whenever there is an update to the CMVC executables, it is necessary to invoke this shell script. The CMVC db2bind shell script provides the following help: db2bind Purpose: Bind CMVC executables to DB2 family instance db2bind Usage: db2bind Notes: 1. Assumes $CMVC_HOME points to cmvc directory (e.g. /usr/bin/cmvc) 2. is typically the name of the family account 3. Database output is stored in db2bind.out 7.3 RUNNING DB2 RUNSTATS IS TAKING A VERY LONG TIME QUESTION: When running DB2 RUNSTATS, it takes a very long time, even for a CMVC family that is very small. Is this OK? ANSWER: The most likely reason for this problem is that you are executing RUNSTATS (or related DB2 utilities such REORG) when the CMVC family daemons are still running. You should stop the CMVC family before you use these DB2 utili- ties. Also, you need to use the CMVC db2bind shell script after you run these DB2 utilities. 7.4 NEED TO REBOOT THE HOST IF SYSTEM HANGS DURING DB2STOP QUESTION: The CMVC family was successfully stopped but when issuing "db2stop" the system hangs. ANSWER: If after a successful shutdown of the CMVC family by using stopCMVC, you try to stop DB2 by using "db2stop" and the system hangs, then you may need to reboot your system in order to reset it. Database issues 55 7.5 BACKUPCMVC FAILS WITH DB2 ERROR DB2106E QUESTION: I am using the backupCMVC script to backup our families. When I run the script the first family in the list backs up OK, however, when it comes to the next family backup the DB2 command fails with the following error: DB21016E The command line processor encountered a system error while sending the command to the back end process. ANSWER: You can add the following statement in the backupCMVC, at the end of the 'for i in ${CMVC_family_list}' loop: su - ${i} -c "db2 terminate" That breaks the connection for one database and you can continue on with the second database. 7.6 HOW TO DISPLAY THE SQLCA REASON CODE WHEN THERE IS A DB2 PROBLEM QUESTION: How to display the SQLCA reason code when there is a DB2 problem ANSWER: There are some DB2 systems errors, such as SQL0980 (shown as SQL -980), that provide more information, such as a reason code, on a data structure called SQLCA. Table 4, Appendix B "Reason Codes" from the DB2 Messages and Problem Determination Guide, says that for the SQL0980 message, the reason code is included in the second array element of the SQLERRD field in the SQLCA structure. Th SQL Reference manual, page 239, Appendix B, SQLCA, describes the SQLERRD field as an array of 6 integers, and the 2nd element is reserved for internal return code. This data structure is not available to CMVC, but the end user needs to do the following in order to get the extra information. The SQLCA can be viewed from the DB2 command line processor by 56 CMVC FAQ: server setting the DB2OPTIONS=-a in the profile of the CMVC family account, and then running: db2 list command options The SQLCA will be shown at the end of the output and the 2nd field of the SQLERRD structure will contain the reason code. An example is shown below: SQLCA Information sqlcaid : SQLCA sqlcabc: 136 sqlcode: 0 sqlerrml: 0 sqlerrmc: sqlerrp : sqlerrd : (1) 0 (2) 0 (3) 0 (4) 0 (5) 0 (6) 0 sqlwarn : (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) sqlstate: 7.7 HOW TO DUMP THE SYBASE TRANSACTION LOG DUE TO THE NOTIFY DAEMON QUESTION: The Sybase log segment fills up because CMVC notifyd is doing a BEGIN TRANSACTION and is not rolling it back or committing it. The open transaction prevents Sybase from truncating the syslog when a dump transaction database_name to the device is issued. This results in the logsegment filling up over time, which causes cmvcd to hang daemons for Sybase 10 or terminate daemons for Sybase 4.9. ANSWER: When notifyd attaches to Sybase, a begin transaction is issued, and this transaction is kept open during the entire life span of notifyd. This pending transaction prevents Sybase from trun- cating the transaction log at the time of a transaction dump. Because the transaction log is not truncated, the disk space is consumed until all of the space for the log device has been used up. This workaround is a significant deviation from normal practices. The work around is to run a cron job which: 1. Kills notifyd for a given family. 2. Performs a transaction dump. For details, see "How to dump the transaction log in Sybase" the section about Sybase in the technical report "CMVC fre- Database issues 57 quently asked questions: handling OEM platforms and data- bases". 3. Restarts notifyd. This will allow Sybase to dump the transaction log. The only disadvantage is that during the interval when notifyd is down, there will be no mail notification. To reduce this exposure window, it is recommended to perform this workaround at times of low user activity during the day. 58 CMVC FAQ: server APPENDIX A. BIBLIOGRAPHY For more information on how to use CMVC, you can consult the fol- lowing manuals and publications: SC09-1596-01 IBM CMVC Client Installation and Configuration SC09-1597-01 IBM CMVC User's Reference SC09-1631-02 IBM CMVC Server Administration and Installation SC09-1633-00 IBM CMVC Concepts SC09-1635-01 IBM CMVC Commands Reference The following Redbooks offer practical advice on CMVC: GG24-4178-00 Did you say CMVC? GG24-4345 CMVC: Customer's Perspective The following technical reports describe in detail useful hints on using CMVC: 29.2169 How to use CMVC with National Language Support (NLS) and Double-Byte Character Sets (DBCS) 29.2180 How to do routine tasks in the OEM Platforms for CMVC 29.2183 Using CMVC for Products with Multiple National Language Versions (NLVs) 29.2232 How to do migration tasks with CMVC 29.2244 How to build and package the CMVC client for Windows 3.1 29.2245 How to build and package the CMVC client for OS/2 29.2253 Comparison between CMVC 2.3 and TeamConnection 2 29.2254 Migrating from CMVC 2.3 to TeamConnection 2 29.2268 CMVC frequently asked questions: license management with NetLS/iFOR 29.2269 How to build and package the CMVC server/Client for UNIX 29.2296 CMVC frequently asked questions: files, releases, levels, and tracks 29.2297 CMVC frequently asked questions: version control and database synchronization issues 29.2298 CMVC frequently asked questions: hints and tips on using databases and OEM platforms 29.2299 CMVC frequently asked questions: client for UNIX, OS/2 and Windows 3.1 29.2321 Comparison between TeamConnection 2 and CMVC 2.3 (2nd Edition) 29.2322 CMVC/TeamConnection: How to cope with dynamic IP addresses 29.2323 CMVC frequently asked questions: handling file, release and level extract problems Appendix A. Bibliography 59 A.1 HOW TO GET ELECTRONIC COPIES OF MANUALS AND TECHNICAL REPORTS Many of the manuals and technical reports mentioned in this docu- ment can be downloaded as follows: o From the IBM intranet (only for IBM employees). o From the Internet (open to everyone). A.1.1 IBM Intranet ___________________ A.1.1.1 Web Home Page You can access the CMVC Service/Development Home Page at: http://tc-cmvc.raleigh.ibm.com/cmvc A.1.1.2 FTP You can download the code from our internal FTP site, by doing: 1. ftp tc-cmvc.raleigh.ibm.com 2. login as 'anonymous' and for password give your email address. 3. cd e:\cmvc\doc\tr 4. binary 5. get fileName 6. quit A.1.2 Internet _______________ A.1.2.1 Web Home Page Not available. 60 CMVC FAQ: server A.1.2.2 FTP You can download the code from our external FTP site, by doing: 1. ftp ftp.software.ibm.com 2. login as 'anonymous' and for password give your email address. 3. cd ps/products/cmvc/doc 4. binary 5. get fileName 6. quit Appendix A. Bibliography 61 62 CMVC FAQ: server APPENDIX B. COPYRIGHTS, TRADEMARKS AND SERVICE MARKS The following terms used in this technical report, are trademarks or service marks of the indicated companies: +---------------------+-------------------------------------------+ | TRADEMARK, | COMPANY | | REGISTERED | | | TRADEMARK OR | | | SERVICE MARK | | +---------------------+-------------------------------------------+ | IBM, AIX, OS/2 | IBM Corporation | | CMVC, TeamConnection| | | DB2/6000, DB2 | | +---------------------+-------------------------------------------+ | PostScript | Adobe Systems Incorporated | +---------------------+-------------------------------------------+ | HP-UX | Hewlett-Packard Company | +---------------------+-------------------------------------------+ | Informix | Informix Inc. | +---------------------+-------------------------------------------+ | PVCS | InterSolv, Inc. | +---------------------+-------------------------------------------+ | Windows | Microsoft Corporation | +---------------------+-------------------------------------------+ | Oracle | Oracle Corp. | +---------------------+-------------------------------------------+ | NFS, SunOS, Solaris | Sun Microsystems Inc. | +---------------------+-------------------------------------------+ | Sybase | Sybase Inc. | +---------------------+-------------------------------------------+ | UNIX | X/Open Co., Ltd. | +---------------------+-------------------------------------------+ END OF DOCUMENT Appendix B. Copyrights, Trademarks and Service marks 63