Rational TestManager
Release Notes
Release 2002.05.00
Copyright © 2001 Rational Software and its subsidiaries. All rights reserved.
November 2001About These Release Notes
- Trademarks and Copyrights
Scope-Product Definition
Getting Started
- Installation Information and System Requirements
- Installation Issues
- Creating the Rational Test Data Store
- Message about required Internet Explorer (IE) upgrade
- Upgrading from the previous release of TestManager
- Evaluation artifacts
- Upgrading from earlier Rational Testing Products Releases
New and Changed Features
- General Enhancements
- Test impact analysis
- Enhanced filtering of test input sources
- New built-in test input type: Microsoft Excel
- Defining test script options
- New Test Scripts View
- Defining test tool options
- Support for Red Hat Linux Version 7.1 Agents
- New actions in the test script console and test input adapters
- Test Asset Management and Reporting Enhancements
- Importing and exporting test assets
- Changes to the test case reports
- Performance Testing Enhancements
- New reports
- Improved license allocation of virtual testers
Guidelines and Restrictions
- Using Excel Test Inputs
- Submitting defects in Test Log
- Updating suspicion from TestManager
- Embedded quotation marks in datapool columns
- Test Script Playback
- General
- VU
- Command Line
- Manual
- Visual Basic
- Generated Visual Basic and Java Test Scripts
- Supported Agents
- Playback Restrictions on UNIX Agents
- TSS ApplicationWait Call
- F5 (Refresh All) Does Not Work in Test Asset Workspace
- Resource Monitoring
- Report Generation
- SAP Limitations
- XML Files
- SQL Anywhere Server
- Rational Administrator
- Unified Change Management (UCM)
- Associating a Rose Test Input Source with a Project
- Editing Reports with Long Path Names
- Performance Testing
- Response vs. Time Reports
- Suspended Users in Transactors
- ODBC Limitations
- Testing CORBA Applications
- Rational TestManager Help
Contacting Rational Technical Support
About These Release Notes
These Release Notes provide information that is not available in the printed documentation or the Help for Rational TestManager.
Trademarks and Copyrights
A complete online trademark and copyright information page for all Rational products resides in the /doc subfolder of your installation folder. The file is called copyright.html, and its default location is C:/Program Files/Rational/doc.
Scope-Product Definition
This release incorporates several enhancements to the Rational TestManager product.
Getting Started
Installation Information and System Requirements
You will find complete installation instructions and system requirements in Rational Testing Products Installation Guide, which is available as a printed book and electronically as a PDF file on the Rational Solutions for Windows Online Documentation CD, which also includes a version of the Acrobat Reader.
Installation Issues
Creating the Rational Test Data Store
After installation, when creating the Rational test data store from Rational Administrator, do not specify a path containing non-ASCII characters. Also, do not use non-ASCII characters when naming any projects or log folders. If your test data store path contains non-ASCII characters, you will experience problems of varying severity, including abnormal test script termination, datapool access anomalies, and problems with verification points.
This is a particular issue when installing and using Rational products in non-English environments. This problem will be fixed in a patch.
Message about required Internet Explorer (IE) upgrade
You may see a message that indicates that you must upgrade to IE 5.0 Service Pack 2 before you can install the Rational testing products. This message is incorrect. The testing products support IE versions 5.0 and 4.0. Click on Ignore, and you will not be prevented from installing.
Upgrading from the previous release of TestManager
If you are upgrading to this release (2002.05) from the previous release of TestManager, you must remove the Nutcracker executable and restart your computer before installing the current version.
To remove the Nutcracker executable:
- Click Start > Settings > Control Panel.
- Click Add/Remove Programs.
- Select Nutcracker, and click Remove.
- Click OK.
- Restart your computer before installing the current version.
Evaluation artifacts
Evaluation artifacts are not included as a part of this installation procedure. As a result, when you select the Evaluator option on the Software Use page of the Rational Software Setup program, sample data will not be installed.
To obtain the sample data, download the Rational Suite Tutorial from http://www.rational.com/documentation.
Upgrading from earlier Rational Testing Products Releases
Datastores created in Rational Test Releases 2001A.04 or 2001.03 are not compatible with the latest version, 2002.05. Before TestManager or Robot Versions 2002.05 can connect to and operate against the test assets contained in the datastores, you must first use Rational Administrator to upgrade them as follows:
In the Administrator, select File > Upgrade Project Assets.
Select the .RSP file to upgrade the datastores associated with the project.
The program displays a progress dialog. When the process is complete, you can view the log generated during the upgrade.
After the datastore has been upgraded, it cannot be used with earlier versions of TestManager or Robot.
New and Changed Features
The following new features have been added to Rational TestManager. For details, click Help > Contents and Index > What's New in Rational TestManager 2002.05.00.
General Enhancements
Test impact analysis
If a test input changes, TestManager now informs you of the impact of that change on your test plan. For example, if a requirement changes in some meaningful way, you need to know which test cases in your test plans are affected so that you can update them to reflect requirement changes. TestManager automatically identifies and marks the affected test cases as "suspect" for ReqPro and Excel Test Input sources.
Enhanced filtering of test input sources
The Test Input Adapter Extensibility feature of TestManager now supports the ability to save test input source filters and Test Case Coverage report filters. Test input sources and Test Case Coverage reports are unfiltered by default. When you set a filter for a test input source, the last filter that you set persists in the Test Inputs window and Test Input Select dialog box. When you set a filter for a Test Input source, TestManager saves the filter with the test input source. When you set a filter for a Test Case Coverage report, TestManager saves the filter with the report.
New built-in test input type: Microsoft Excel
You can now use one or more Excel spreadsheets as test input sources. This test input adapter can be useful in many ways. Many teams use Excel to manage information in their projects. For example, some teams may document requirements or capture project planning information in an Excel spreadsheet. In either case, the new Excel test input adapter allows teams to track the testing effort with respect to their information in an Excel spreadsheet. Specifically, they could associate test cases with tasks defined in rows of a spreadsheet so that a coverage report could be generated to show progress or completion of those tasks.
Defining test script options
You can now define additional test script options controlling how test script execution adapters execute test scripts.
New Test Scripts View
You can now view all your test scripts sorted by type from the Test Scripts View. From this window you can perform test management operations on test scripts. You can also perform basic version control operations (check-in, checkout, etc.) on registered test scripts that support these operations.
Defining test tool options
You can now set test tool options to control how a particular test tool plays back a test script. You can define a test tool option in the test script execution adapter that you have written for a particular test tool.
Support for Red Hat Linux Version 7.1 Agents
You can now play back VU, Java, and Command Line, and custom test scripts on Red Hat Linux version 7.1 agents, as well as on version 6.x agents supported in earlier versions of TestManager.
New actions in the test script console and test input adapters
Test script console adapter and test input adapter developers can now add their own custom actions, or commands. You can run these custom actions or commands in TestManager.
Test Asset Management and Reporting Enhancements
Importing and exporting test assets
You can move or copy existing test assets between and within Rational projects, use third party tools to create importable versions of test assets, or use third party tools to analyze exported test assets. Importing and exporting Test Assets are subject to the following restrictions:
- Logs, log details, and test case results are not supported by TestManager's import functionality.
- TestManager does not support importing and exporting queries.
- Test Suite implementations are not imported or exported. Test Suite reference names can be imported or exported provided the reference name exists in the import destination Test Datastore.
Changes to the test case reports
You can now run test case reports that display test case suspicion status as presented by the test impact analysis capability. Using the test case suspicion feature you can track meaningful changes to test inputs so that you can update the associated test cases or their implementations.
Also, TestManager now has several new pre-defined test case coverage reports.
Performance Testing Enhancements
New reports
There are two new performance testing reports in TestManager: Command Data and Command Trace. Command Data reports let you identify the differences between expected and actual responses in a test run. They show the commands sent and the response data that is written to the test log. The Command Trace report lists the activity of a suite run and enables you to examine in detail unusual or unexpected events. This report formats raw data from the suite run without performing statistical analysis.
Improved license allocation of virtual testers
You can now combine multiple non-pooled virtual tester packs. Thus, if you had a 100 virtual tester pack and a 500 virtual tester pack, you could execute a single run of as many as 600 virtual testers.
Guidelines and Restrictions
Using Excel Test Inputs
In TestManager you can view test inputs that have been defined in Microsoft Excel spreadsheets. If, while viewing those test inputs, you or another user creates a new test input or modifies an existing test input in Excel, you must refresh the TestManager test input view to see those changes in TestManager.
If you are changing the "Date Modified" column in an Excel spreadsheet and the TestManager datastore and spreadsheet are in different time zones, the "Update Suspicion" feature does not work properly.
Submitting defects in Test Log
If after submitting a defect in the Test Log Window, you want to log into a different datastore or project, or log in as a different user, TestManager may not display the login dialog after restart. In this case you must either log off the operating system and log back in (if you are running on Windows NT or later), or bring up Task Manager, and terminate the CQINTSRV11.EXE process. You can then log in to another datastore or project, or log in as a different user.
Updating suspicion from TestManager
Requisite Pro projects that use a Microsoft Access database may experience a delay between the entry of the data and the display of the modified data. This delay is caused by a defect in the Microsoft MDAC component. A similar delay is apparent from TestManager when Requisite Pro test inputs are involved. The most noticible case is when suspicion is updated and cleared frequently in the Test Input View.
Embedded quotation marks in datapool columns
The TSSDatapool.DatapoolValue (Visual Basic) and TSSDatapool.datapoolValue() (Java) methods take a datapool column argument. If the value of the column specified by this argument contains double quotes (for example, "Name"), you must manually edit the datpool column and add a second pair of double quotes around the value (""Name"").
Test Script Playback
General
- Messages logged with the VU log_msg() function, or with an analogous TSS call, are truncated at 30 kb.
VU
- Playback of VU test scripts is not supported for Windows Me or Windows 98, whether the system is a Local computer or an Agent computer.
- To play back Oracle 7 with Oracle 7 libraries on HP-UX, you must set the following environment variable before invoking TestManager:
RT_MASTER_USE_STATE_FOR_SQLCommand Line
- When running command line test scripts or custom test script types that use the command line adapter on Windows NT systems as Agents, TestManager may report test script failure even though the test scripts executed successfully. To avoid this, install Service Pack 6a for Windows NT 4 on the Agent.
- If you write a test script on a Windows computer in an interpreted scripting language such as Perl or Python and execute that test script on a UNIX agent, the test script might fail with an error message such as sh: filename: not found even though the file does exist and has the correct permissions. This problem is caused by the carriage return - line feed sequence that DOS text files use to mark line endings. The Notepad editor creates files of this type. To avoid this problem, use an editor that will save text files in the UNIX text file format that marks line endings with a line feed character.
Manual
When playing back manual test scripts from TestManager, be sure to exit Rational ManualTest to terminate the script. The alternative, terminating the user from the TestManager console, is not necessary and may lead to unexpected results.
Visual Basic
Test Script Services (TSS) support test script playback from a proxy TSS server. However, TestManager does not provide Visual Basic bindings to those proxy TSS servers.
Generated Visual Basic and Java Test Scripts
Visual Basic and Java test scripts that are automatically generated (by the Robot Session Recorder or by QualityArchitect) do not record script line numbers. As a result, the TestManager Script view feature (the Script check box under Tools > Options) does not work with these test scripts. If you manually add method calls to your Visual Basic and Java test scripts that pass test script line numbers to TestManager, the Script view will work. The relevant methods are TSSMonitor.PositionSet (Visual Basic) and TSSMonitor.positionSet (Java).
Supported Agents
The following table lists supported agent operating systems and indicates which built-in test script types can be played back on each. Important note: effective with this release, Windows 95 is no longer a supported platform for the Rational Test Agent.
Playback Restrictions on UNIX Agents
- All versions of the test script service GetPath fail with a segmentation fault on any UNIX agent. The calls are:
- All versions of the test script service JavaApplicationStart fail with a segmentation fault on any UNIX agent. The calls are:
- A Java test script executed on a UNIX agent via rttsee that (1) extends com.rational.test.tss.TestScript and (2) includes the method call TSSServer.serverStart()fails with the message symbol avl_insert: referenced symbol not found. To avoid this and similar problems, do not base a test script on com.rational.test.tss.TestScript that you plan to execute with rttsee. The requirement to do this applies only to test scripts that are executed from TestManager.
TSS ApplicationWait Call
The test script service ApplicationWait includes an exitStatus argument. All versions of this call fail with a segmentation fault if a value of NULL is entered for exitStatus. The calls are:
- TSSApplicationWait() [C]
- ApplicationWait [tsscmd]
- TSSUtility.applicationWait() [Java. Note that this means the override method, which has no exitStatus argument, cannot be used.]
- TSSUtility.ApplicationWait [Visual Basic]
F5 (Refresh All) Does Not Work in Test Asset Workspace
You can press the F5 key to refresh all in the Test Plan window and in the Test Input window. This key does not work in the Test Asset Workspace. You must right-click a folder in each Test Asset Workspace tab or click View > Refresh Workspace from the main menu to refresh all the Test Asset Workspace.
Resource Monitoring
- Monitoring system resources during test runs is not supported for Windows Me or Windows 98, Local or Agent.
- To monitor resources on an Agent computer that is running AIX, you must install the bos.acct package on that Agent if it is not already installed.
Report Generation
TestManager provides a "previous" button allowing you to change the test log selection after you have selected the virtual tester, script, and commandID filters to be used in generating a report. This causes a problem because the previous test log is not closed and conflicts with the new test log. Workaround: if you need to change the test log selection after you've selected filters, start over rather than use the "previous" button.
SAP Limitations
The following list gives SAP testing limitations in TestManager:
- UNIX Agents are not supported for the SAP protocol.
- The personalizing option (the multicolored circle in the upper right of SAPGUI) is not supported.
- Multiple SAP sessions with a single client are not supported.
- Only the active screen is rendered, while SAPGUI sometimes renders inactive screens.
- The control box in table controls is not supported.
- Transaction IQ03 is not supported.
- Automatic logon load balancing, as performed by SAPLOGON.EXE and the SAP Session Manager, is not supported.
- The Manager's Desktop functionality under the "Human Resources" menu is not supported.
XML Files
The XML parser used with TestManager cannot parse XML files encoded in UTF-8 and with byte order marks. To work around this limitation, you can save the files choosing a different encoding (for example, ANSI, Unicode, and Unicode BE).
SQL Anywhere Server
During the installation of a SQL Anywhere Server, select Replace these files with English equivalents independent of the language environment in which you are working. Microsoft recommends this because selected files must be replaced with their English equivalents.
Rational Administrator
Unified Change Management (UCM)
If you use Unified Change Management (UCM) with Rational Suite under these conditions:
- You use Rational ClearQuest to manage UCM activities, and
- You place a RequisitePro project or a Rational Test datastore under configuration control
Make sure that you do not set the UCM policy, "Check Assignment Before Work On". (The default is for the policy not to be set.)
If you set the policy, you cannot work on an activity unless you are the owner of that activity. However, in the current implementation of UCM for RequisitePro and Rational Test, an entire team of workers share the same activity. Therefore, if the policy is set, only one person, not the entire team, can work in a RequisitePro project or in a Rational Test datastore.
If the policy is mistakenly set, the error message "Source control could not set activity." appears. To work around this error:
- Start the Rational Administrator and connect to a Rational project.
- Right-click the UCM project and click Properties.
- On the ClearQuest tab, unset the policy.
For further details, see the Rational Suite Administrator's Guide.
Associating a Rose Test Input Source with a Project
The Rational Administrator includes a Configure Project dialog box. When associating a Rational Rose modle with a Rational project, it is not automatically defined as a test input source for TestManager.
To register a Rational Rose model as a test input, use the Manage Test Input Types dialog box from TestManager. To reach this dialog, click Tools > Manage > Test Input Types.
Editing Reports with Long Path Names
Crystal Reports is used to generate or edit Listing reports. You can run a report regardless of the length of its pathname. However, if you attempt to edit a report whose pathname is longer than 80 characters, the message "Dictionary not found" appears. This is a Crystal Reports restriction. If you create a new or customize an existing design layout, keep the root path to the test datastore as short as possible (less than 80 characters).
Performance Testing
The issues described in this section are specific to performance testing.
Response vs. Time Reports
The Show Resource dialog box for data points in the Response vs. Time report for a Performance Testing Report, refers to the MASTER computer. This should read Local computer.
Suspended Users in Transactors
When you monitor a TestManager suite and suspend a virtual tester in an independent transactor, upon restarting, the tester will try to catch up on the missed transactions. If you suspend a virtual tester in a coordinated transactor, the remaining testers will compensate by attempting to maintain the desired transactor rate. If the rate cannot be maintained, as testers are restarted, they will try to catch up on the missed transactions.
ODBC Limitations
When playing back a SQL statement that generates multiple result sets, TestManager logs only the data for the first fetch. The other fetches show the number of rows returned, but does not log the data actually returned.
If a sqlfetch_cursor is run multiple times in succession with ALL_ROWS and EXPECT_ROWS 0, an error for the second sqlfetch_cursor is generated stating:
ERROR -1: {error} STATE=S1010, CODE=0, MSG=[Microsoft][ODBC Driver Manager] Function sequence errorWhen logging data returned from sqlfetch_cursor, sometimes the data is truncated and the next SQL command appears on the same line as a row of data. This is a logging problem only; the test script is executing correctly.
Testing CORBA Applications
To test CORBA applications with TestManager, you must have the following software installed:
- Windows 2000 or Windows NT Version 4.0 with at least Service Pack 4
- VisiBroker (C++ or Java) Version 3.3.
CORBA Playback
Before playback, you must import into the project the IDL for each interface referenced in a test script.
TestManager provides special IIOP monitoring states in the state histogram monitor window.
TestManager does not support playback of ASCII hex (opaque) data between computers of differing byte ordering. Byte ordering specification is marked in the IIOP protocol. It is possible that data may be transferred with big-endian byte order, even on Windows NT systems. This case is more likely to occur if the application is Java. The result is that data specified opaquely in these cases may not play back correctly, even though the recording computer and the playback computer have the same byte order.
Supported and Unsupported CORBA Data Types for VU Test Scripts
TestManager supports playing back the following CORBA data types:
- char
- octet
- short
- unsigned short
- long
- unsigned long
- float
- double
- boolean
- enum
- struct
- union
- array
- sequence
The following data types are not supported, and will result in methods being scripted opaquely if they occur as a parameter data type in a method:
UNIX Agents
Only Solaris is supported for UNIX Agent playback of VU language CORBA/IIOP scripts recorded by Rational Robot. The VisiBroker bind modus is not supported for playback.This limitation means that "VisiBroker" cannot be used to locate object references at play back. Therefore, another method, such as reading IORs (Inter-operable Object Reference) from a file, or specifying them explicitly in VU test scripts, must be used to locate objects when playing back test scripts from a Solaris Agent.
Rational TestManager Help
A paragraph in the Help topic, What's New in Rational TestManager 2002.05.00, contains a typographical error. A corrected version of the paragraph follows.
Enhanced filtering - The Test Input Adapter Extensibility feature of TestManager now supports the ability to save test input source filters and Test Case Coverage report filters. When you set a filter for a test input source, the last filter that you set affects test inputs that appear in the Test Inputs window and Test Input Select dialog box until you change the filter. When you set a filter for a Test Input source, TestManager saves the filter with the test input source. When you set a filter for a Test Case Coverage report, TestManager saves the filter with the report.
Contacting Rational Technical Support
If you have questions about installing, using, or maintaining this product, contact Rational Technical Support as follows:
Note: When you contact Rational Technical Support, please be prepared to supply the following information:
|
Rational Software Corporation http://www.rational.com sales@rational.com info@rational.com |