Rational®TestManager
Release Notes
Release 2003.06.00
Copyright © 2003 Rational Software Corporation. All Rights Reserved.
This documentation (the "Work") is protected under the copyright laws of the United States and/or other jurisdictions, as well as various international treaties. Any reproduction or distribution of the Work is expressly prohibited without the prior written consent of Rational Software Corporation.
Legal notices are available in program files\Rational\doc\legal_information.html.
About These Release Notes
Support for Crystal Decisions Crystal Reports
Getting Started
- Installation Information and System Requirements
New and Changed Features
- Rational ManualTest
- Undo and Redo
- Find and Replace
- Step by Step Execution Mode
- Exporting Test Assets to Text Files
- Specifying the Expected Result
- Specifying Result Details
- Viewing the New Features in the Log
- Executing Manual Scripts via the Web
- Deleting Multiple Log Assets
- Enhanced High-Volume Testing
- Test Inputs View
- TestManager and XDE Tester
Guidelines and Restrictions
- Unsupported Platforms
- Windows Me
- Using Microsoft Excel Test Inputs
- Submitting Defects in the Test Log
- Data Code Pages: Character Set Compatibility with ClearQuest
- Using the Suspicion Feature in TestManager
- Ensuring the Performance of the Suspicion Feature
- Updating Suspicion
- Manually Updating Suspicion
- Test Script Playback
- VU
- Command Line
- ManualTest
- Visual Basic
- Generated Visual Basic and Java Test Scripts
- F5 (Refresh All) Does Not Work in Test Asset Workspace
- Resource Monitoring
- Report Generation
- SAP Limitations
- UCM on Windows NT with ClearCase 5.0
- Editing Reports with Long Path Names
- Performance Testing
- ODBC Limitations
- Testing CORBA Applications
Defects and Change Requests
- Problem with Exporting and Importing Test Case Designs
- Problem with Test Case Results Trend Reports
Late Changes to Documentation
- Wildcard References in Filter Operator Types Help Topic
- TestManager Extensibility TIClearFilter Function
Contacting Rational Customer Support
About These Release Notes
This release incorporates several enhancements to the Rational TestManager product. These Release Notes provide information that is not available in the printed documentation or the Help for Rational TestManager.
Support for Crystal Decisions Crystal Reports
IBM Rational TestManager Version 2003.06.00 supports Listing report formats authored with Crystal Reports 8.5. Crystal Reports is required to design, run, and view those reports. TestManager Version 2003.06.00 does not include copies of the Crystal Reports files nor the run-time libraries necessary to enable these capabilities.
For information about TestManager Listing reports, see the TestManager Help and the Rational TestManager User's Guide.
For instructions about how to license and install the appropriate Crystal Reports 8.5 software and the components necessary to create and run Listing reports in TestManager, contact Crystal Decisions directly. Any reference to Crystal Reports in the TestManager product documentation applies only to copies of Crystal software that you have rightfully licensed.
You can purchase licenses for Crystal Reports and obtain more information about licensing the run-time components required to run and view Listing reports in TestManager from the Crystal Decisions, Inc. Web site (www.crystaldecisions.com), or by contacting them by telephone:
Getting Started
Installation Information and System Requirements
The Rational Desktop Products Installation Guide, Rational Server Products Installation Guide, and Rational Suite Upgrade Guide contain complete installation and upgrade instructions and system requirements for Rational products for Windows. They are available electronically as PDF files on the Rational Solutions for Windows Online Documentation CD, which also includes a version of the Acrobat Reader.
Refer also to the Rational Testing Products Release Notes for important information about installing Rational testing products and creating test datastores.
New and Changed Features
For details on new features, click Help > Contents and Index > What's New in Rational TestManager 2003.06.00.
Rational ManualTest
Undo and Redo
Undo and Redo are now supported in the Manual Test Grid Editor, the Manual Test Text Editor, and the Manual Test Execution window. You access the Undo and Redo options from the Edit menu. You can also press Ctrl+Z or Ctrl+Y to undo or redo tasks, respectively.
Find and Replace
Find and Replace are now supported in the Manual Test Grid Editor and the Manual Test Text Editor.
You access the Find and Replace options from the Edit menu. You can also press Ctrl+F or Ctrl+H to find and replace text.
Step by Step Execution Mode
Step by Step execution mode allows you to run a Manual Test Script in a small modeless dialog box outside of the Manual Test application window. This allows you to minimize the Manual Test application so that it does not interfere with the application under test. To run in Step by Step mode, click Tools>Options and select Run in Step by Step mode.
Exporting Test Assets to Text Files
You can export manual test scripts from the ManualTest application to text files.
Specifying the Expected Result
You can specify the expected result of a step or verification point in both the text and grid editors. In the Grid Editor, "Expected Result" is a new column in the grid. In the text editor, the expected result appears as a new icon.
In the new Expected Result column, you can associate a file with a step or verification point to describe the expected result. For example, you can reference a bitmap capture of a screen that illustrates an expected result. In the Grid Editor, this is accessed through a new column in the grid. The file's full path is stored with the manual test.
Note: Rational recommends that you use Uniform Naming Convention (UNC) path names for expected result files.
Specifying Result Details
The Comments column in the Manual Test Execution Grid has been renamed to Result Details. In addition to the column name change, the dialog box for entering the comment (Result Details) has been changed to allow you to specify the Result Details File. For example, you could attach screen shots of failed steps or verification points.
Note: The Notes, Description, Expected Result, and Expected Result File, columns are read-only during execution.
Viewing the New Features in the Log
The first 64 characters of the description of each step or verification point are now written to a log when a manual test is run.
Now that you can associate file references with manual script steps and verification points during development and execution of the tests, you can gain access to these files when viewing the results. The Expected Result File and Result Details File for each step and verification point can be accessed from the tree view within the Test Log Viewer.
Executing Manual Scripts via the Web
The Expected Result and Result Details fields now appear during Manual Script Execution using Web.
ManualTest Web supports the latest Netscape versions. However, in Netscape 7.0 and higher, Expected Result files display links as path names only.
Deleting Multiple Log Assets
You can select and delete multiple test logs at the same time from the Results view of the Test Asset Workspace. Assets covered by this functionality include Builds, Test Log Folders, and Test Logs.
Enhanced High-Volume Testing
You can customize your system settings for high-volume testing on a single Windows-based computer (600 to 1000 virtual testers or more). To do this, you must edit the registry to increase desktop heap memory (memory used for dynamic memory allocation). For more information, see Appendix A in the Rational TestManager User's Guide.
Test Inputs View
The Test Inputs view in TestManager now mirrors the Rational RequisitePro folder structure. This change is a result of performance improvements. TestManager no longer uses the COM layer to communicate with RequisitePro. Instead, it communicates directly to the RequisitePro database. Rational recommends that you organize your RequisitePro requirements in a way that makes them easy to find in the Test Inputs view.
TestManager and XDE Tester
XDE Tester is the newest version of the product introduced as Rational RobotJ in Rational Suites Enterprise Studio, Rational Suites TestStudio, and Rational TeamTest version 2002.05.02. XDE Tester is integrated with TestManager. XDE Tester is a stand-alone point product, but it maintains the same integrations when you have the 2003.06.00 versions of both XDE Tester and TestManager installed. In TestManager, there is a new XDE Tester icon in the tool bar. It may appear differently in places in the Rational TestManager User's Guide.
Note: You purchase and install XDE Tester separately from TestManager or any other Rational testing products.
Guidelines and Restrictions
Unsupported Platforms
TestManager does not support Citrix or Microsoft Windows Terminal Server.
Windows Me
When you open a manual test script using the Grid Editor in the manual text editor, part of the text disappears, word wrap does not work, and the backspace and delete buttons do not work. To restore the missing text, you must click another step or verification point, and the text displays correctly.
Using Microsoft Excel Test Inputs
In TestManager you can view test inputs that have been defined in Microsoft Excel spreadsheets. If, while viewing those test inputs, you or another user creates a new test input or modifies an existing test input in Excel, you must refresh the TestManager Test Inputs view to see those changes in TestManager.
If you are changing the Date Modified column in an Excel spreadsheet and the TestManager datastore and spreadsheet are in different time zones, the Update Suspicion feature does not work properly.
When using an Excel spreadsheet as a test input source, be aware that if you insert or remove a row or column in the spreadsheet, you may need to update the Test Input Source configuration settings.
When an association is made between a test input and a test case, the value in the ID column is stored by TestManager in order to preserve the association. Any modifications to the spreadsheet that affect the specified columns (ID and Name), such as adding or removing rows or columns immediately around the specified columns or rows, may invalidate the spreadsheet as a source of test inputs and require that you update the associated test input source. Modifying the values in the specified column or rows can also result in a loss of connection between the test case and test input.
Submitting Defects in the Test Log
Data Code Pages: Character Set Compatibility with ClearQuest
You may see an error message about data code pages if your TestManager project is integrated with ClearQuest. The error message appears when you enter data into a tool and the characters you use are incompatible with the set of characters that a ClearQuest database accepts.
For example, say that you are working in a TestManager project that is integrated with a ClearQuest database, and the ClearQuest database accepts characters in English only. If you use English to enter TestManager data, the error message does not appear. However, the error message does appear if you use another character set, such as Japanese or Chinese.
If you see this error message, consult with your ClearQuest administrator or systems administrator to resolve the character set incompatibility.
Using the Suspicion Feature in TestManager
Ensuring the Performance of the Suspicion Feature
With the suspicion feature, you can define TestManager to inform you when requirements or test inputs change in a way that impacts test cases. TestManager detects changes in test inputs based on timestamps from the systems on which they reside. For suspicion to work correctly, you must synchronize the time on the systems. If the timestamps on the systems are out of synchronization by several minutes or more, the suspicion feature does not work. You must also set the time zone for each system, so that if the TestManager and test input systems are in different time zones, the suspicion feature still works correctly.
Updating Suspicion
Requisite Pro projects that use a Microsoft Access database may experience a delay between the entry of the data and the display of the modified data. This delay is caused by a defect in the Microsoft MDAC component. A similar delay is apparent from TestManager when Requisite Pro test inputs are involved. The most noticeable case is when suspicion is updated and cleared frequently in the Test Inputs view.
Manually Updating Suspicion
You can now manually update a suspect link between a test input and the associated test case by clicking View > Update Suspicion from the Test Inputs view or Test Plan window.
Test Script Playback
VU
Playback of VU test scripts is not supported for Windows Me or Windows 98, whether the system is a Local computer or an Agent computer.
To play back Oracle 7 with Oracle 7 libraries on HP-UX, you must set the following environment variable before invoking TestManager:
RT_MASTER_USE_STATE_FOR_SQLSet the variable in the Environment tab of the System Properties Control Panel.
Command Line
When running command line test scripts or custom test script types that use the command line adapter on Windows NT systems as Agents, TestManager may report test script failure even though the test scripts executed successfully. To avoid this, install Service Pack 6a for Windows NT 4 on the Agent.
If you write a test script on a Windows computer in an interpreted scripting language such as Perl or Python and execute that test script on a UNIX Agent, the test script might fail with an error message such as sh: filename: not found even though the file does exist and has the correct permissions. This problem is caused by the carriage return-line feed sequence that DOS text files use to mark line endings. The Notepad editor creates files of this type. To avoid this problem, use an editor that saves text files in the UNIX text file format that marks line endings with a line feed character.
ManualTest
When playing back manual test scripts from TestManager, be sure to exit Rational ManualTest to terminate the script. The alternative, terminating the user from the TestManager console, is not necessary and may lead to unexpected results.
Visual Basic
The Test Script Services (TSS) framework supports playback of test scripts either directly (by TestManager) or by an external proxy server. However, Visual Basic test scripts support only direct TestManager playback.
Generated Visual Basic and Java Test Scripts
Visual Basic and Java test scripts that are automatically generated (by the Robot Session Recorder or by Rational QualityArchitect) do not record script line numbers. As a result, the TestManager Script view feature (the Script check box under Tools > Options) does not work with these test scripts. If you manually add method calls to your Visual Basic and Java test scripts that pass test script line numbers to TestManager, the Script view works. The relevant methods are TSSMonitor.PositionSet (Visual Basic) and TSSMonitor.positionSet (Java).
F5 (Refresh All) Does Not Work in Test Asset Workspace
You can press the F5 key to refresh all in the Test Plan window and in the Test Input window. This key does not work in the Test Asset Workspace. You must right-click a folder in each Test Asset Workspace tab or click View > Refresh Workspace from the main menu to refresh all the Test Asset Workspace.
Resource Monitoring
Monitoring system resources during test runs is not supported for Windows Me or Windows 98 for Local or Agent computers.
To monitor resources on an Agent computer that is running AIX, you must install the bos.acct package on that Agent if it is not already installed.
Report Generation
TestManager provides a "previous" button allowing you to change the test log selection after you have selected the virtual tester, script, and commandID filters to be used in generating a report. This causes a problem because the previous test log is not closed and conflicts with the new test log. Workaround: if you need to change the test log selection after you have selected filters, start over rather than use the "previous" button.
SAP Limitations
The following list gives SAP testing limitations in TestManager:
- UNIX Agents are not supported for the SAP protocol.
- The personalizing option (the multicolored circle in the upper right of SAPGUI) is not supported.
- Multiple SAP sessions with a single client are not supported.
- Only the active screen is rendered, while SAPGUI sometimes renders inactive screens.
- The control box in table controls is not supported.
- Transaction IQ03 is not supported.
- Automatic logon load balancing, as performed by SAPLOGON.EXE and the SAP Session Manager, is not supported.
- The Manager's Desktop functionality under the "Human Resources" menu is not supported.
UCM on Windows NT with ClearCase 5.0
A possible problem has been identified on Windows NT 4.0 when ClearCase 5.0 is installed. After performing a Unified Change Management (UCM) datastore operation from the Rational Administrator (such as create a datastore, check-in all, add to or remove from source control), we suggest you shut down and restart Rational Administrator before attempting another such operation.
Editing Reports with Long Path Names
Crystal Reports is used to generate or edit Listing reports. You can run a report regardless of the length of its pathname. However, if you attempt to edit a report whose pathname is longer than 80 characters, the message "Dictionary not found" appears. This is a Crystal Reports restriction. If you create a new or customize an existing design layout, keep the root path to the test datastore as short as possible.
Performance Testing
ODBC Limitations
When playing back a SQL statement that generates multiple result sets, TestManager logs only the data for the first fetch. The other fetches show the number of rows returned, but do not log the data actually returned.
If a sqlfetch_cursor is run multiple times in succession with ALL_ROWS and EXPECT_ROWS 0, an error for the second sqlfetch_cursor is generated stating:
ERROR -1: {error} STATE=S1010, CODE=0, MSG=[Microsoft][ODBC Driver Manager] Function sequence errorWhen logging data returned from sqlfetch_cursor, sometimes the data is truncated and the next SQL command appears on the same line as a row of data. This is a logging problem only; the test script is executing correctly.
Testing CORBA Applications
To test CORBA applications with TestManager, you must have the following software installed:
- Windows 2000 or Windows NT Version 4.0 with at least Service Pack 4
- VisiBroker (C++ or Java) Version 3.3.
CORBA Playback
Before playback, you must import into the project the IDL for each interface referenced in a test script.
TestManager provides special IIOP monitoring states in the state histogram monitor window.
TestManager does not support playback of ASCII hex (opaque) data between computers of differing byte ordering. Byte ordering specification is marked in the IIOP protocol. It is possible that data may be transferred with big-endian byte order, even on Windows NT systems. This case is more likely to occur if the application is Java. The result is that data specified opaquely in these cases may not play back correctly, even though the recording computer and the playback computer have the same byte order.
Supported and Unsupported CORBA Data Types for VU Test Scripts
TestManager supports playing back the following CORBA data types:
- char
- octet
- short
- unsigned short
- long
- unsigned long
- float
- double
- boolean
- enum
- struct
- union
- array
- sequence
The following data types are not supported, and result in methods being scripted opaquely if they occur as a parameter data type in a method:
UNIX Agents
Only Solaris is supported for UNIX Agent playback of VU language CORBA/IIOP scripts recorded by Rational Robot. The VisiBroker bind modus is not supported for playback. This limitation means that "VisiBroker" cannot be used to locate object references at play back. Therefore, another method, such as reading IORs (Inter-operable Object Reference) from a file, or specifying them explicitly in VU test scripts, must be used to locate objects when playing back test scripts from a Solaris Agent.
Defects and Change Requests
Problem with Exporting and Importing Test Case Designs
There is a known problem with importing and exporting test case designs. If there are carriage returns in the Notes or Description boxes, the test case design does not import or export correctly.
To ensure that test case designs import and export correctly, when you create a test case design, do not enter carriage returns in the Notes or Description boxes.
However, if you need to have multiple lines of information in these boxes, you can do the following:
- In TestManager, right-click on a test case, and then click Design.
- in the Design Editor dialog box, in the Notes or Description boxes, enter multiple lines of text, and then click OK.
- In TestManager, right-click the test case again and click Properties.
- In the Test Case Properties dialog box, click the Implementation tab, and then click Import from Test Case Design.
- In the Manual Test Script Name box, type the name of a manual script and click OK.
- In Rational Manual Test, export the manual test to a text file.
Problem with Test Case Results Trend Reports
Test Case Result Trend reports and Test Case Result Distribution reports are returning erroneous data. Please contact Rational Customer Support and ask for the TestManager Trend Reporting fix.
Late Changes to Documentation
Wildcard References in Filter Operator Types Help Topic
The TestManager online Help topic "Filter Operator Types" has incorrect wildcard references (incorrectly referenced as asterisk(*) and question mark(?)). The text for the purpose of the Like operator should read as follows:
Allows the use of wildcard characters in the Value field to include items that have any alphanumeric text character(s) in particular character positions. The wildcard characters are percentage (%) for a string of characters and underscore (_) for a single character.
TestManager Extensibility TIClearFilter Function
This function is not documented in the current TestManager extensibility documentation.
The following is the text that will be added to the Rational TestManager Extensibility Reference manual for the next release:
TIClearFilter()
Clears filters for a test input source.
Syntax
HRESULT TIClearFilter(const TCHAR SourceID[TI_MAX_PATH], TCHAR ErrorDescription[TI_MAX_ERROR])
Element
Description
input. The handle identifying the connection to the test input source.
output. The message to be displayed to the TestManager user if there is an error.
Return Values
This function typically returns one of the following values:
- TI_SUCCESS. The function completed successfully.
- TI_ERROR. The function did not complete successfully.
- TI_ERROR_INVALID_SOURCEID. The input source was identified incorrectly.
Comments
TestManager makes this call when a user chooses to remove a filter. Your adapter should clear any filters that have been set.
Example
HRESULT TIClearFilter (const TCHAR SourceID[TI_MAX_PATH], TCHAR ErrorDescription[TI_MAX_ERROR]) { AFX_MANAGE_STATE(AfxGetStaticModuleState()); HRESULT rc = S_OK; CConnectionContext *pContext=0; // Figure out what instance of the adapter is in use. m_ProjectConnections.Lookup(SourceID, (void *&)pContext); // Make sure we've got a pContext if (pContext) { // Make sure the Instance of the adapter is connected and loaded if (pContext->m_lpDispatchProject) { //Clear your filters pContext->SetFilterSettings(""); } else rc = TI_NO_PROJECT; } else rc = TI_NO_CONTEXT; // Apply the filter pContext->LoadFilterOptions(); return rc; }Contacting Rational Customer Support
If you have questions about installing, using, or maintaining this product, contact Rational Customer Support as follows:
Note: When you contact Rational Customer Support, please be prepared to supply the following information:
|
Rational Software Corporation http://www.rational.com sales@rational.com info@rational.com |