Rational TestMate Release Notes

Release 4.2.0

Thank you for choosing Rational TestMate!

Please read this document to learn about known product issues, important tips, and other TestMate product information. The Release Notes are also available online in HTML format from the Help > Manuals command. The online Release Notes are guaranteed to be the latest version of this document. If you find discrepancies between the hard-copy and online versions of the Release Note, please assume that the online document contains the most current information.

---

Objectives

These release notes provide information critical to installing and using Rational TestMate, including supported platforms and known issues with this release.

---

Scope - Product Definition

This document applies to Release 4.2.0 of Rational TestMate. These release notes describe:

• Hardware/Software Information
• What's New in TestMate 4.2.0
• Defects Resolved in Release 4.2.0
• Known Limitations

---

Hardware/Software Information

This section provides basic information on the platforms supported and the hardware and software requirements for installing and running Rational Apex and its add-ins.

Supported Platforms

This document uses the following terms to designate the various platforms on which Rational TestMate (hereafter TestMate) runs.

Term	Platform
AIX	IBM RS/6000 or PowerPC workstations running AIX 4.3.3.
Alpha or OSF/1	Digital Alpha workstations running release 4.0d, 4.0e, or 4.0f Compaq Tru64 (formerly known as OSF/1 or Digital UNIX).
HP-UX or HP	HP PA workstations running release 11.0 of HP-UX.
Solaris	Sun SPARC workstations running Solaris 2.6, 2.7, or 2.8.
SGI or IRIX	SGI workstation running IRIX 6.5

This release of TestMate requires Apex release 4.2.x. Refer to the Rational Apex Installation Guide for further information about hardware and software configurations.

---

Documentation

Rational Software Corporation provides the following documentation with this release.

The following documents are available in printed form as well as HTML format:

• Rational Apex Installation Guide
• Rational TestMate Release Note
• Rational TestMate User's Guide
• Rational TestMate Customization and Configuration Guide
• Rational TestMate Command Reference

These documents can be accessed through the Help > Manuals command from any directory viewer or the List of Documents entry in the button bar at the top of each HTML document.

Postscript versions of these manuals are also provided and can be printed using the Help > Print Manuals command.

If updates have not been issued, you may note discrepancies between the printed documentation and the online documentation. In these cases, the online documentation is the most up-to-date and should be followed.

The following are provided in on-line form only:

• Rational TestMate Online Reference
• "man" pages for Rational TestMate command-line interface commands.

---

Upgrading from Previous Releases

Once installation is complete, you may need to upgrade existing views containing TestMate artifacts, such as .tc files. This section describes how to upgrade views created by earlier releases of TestMate.

Upgrading From All Previous Versions

New API Views

There are new versions of the TestMate API view:

$APEX_BASE/ada/testmate.ss/<arch>.ada83.4.2.0.rel
$APEX_BASE/ada/testmate.ss/<arch>.ada95.4.2.0.rel
$APEX_BASE/c++/testmate.ss/<arch>.4.2.0.rel

User views which are upgraded to Apex 4.2.0 must import the corresponding TestMate 4.2.0 API. This can be done by using the apex import command or (preferably) by using the TestMate > Setup dialog box.

The new API views (which are Summit views) are used for software development in both Rational subsystems (new in Apex 4.0.0) and Summit views.

New Customization Keys

There are new versions of the TestMate customization keys. Typically, all views are setup to reference the new keys indirectly via the TestMate.sw file switch TM_CUSTOMIZATION_KEY. In the event that a site specific key is used or this switch is hardcoded to a value other than the default value, the appropriate modifications must be made to the TestMate.sw file to either use the new keys directory provided in this release, or to update the site specific keys referenced by the TestMate.sw file to reflect any modifications made to the new keys.

The new keys directories are:

$APEX_BASE/{c++,ada}/keys/tm_standard.4.2.0

For TestMate Ada Cross for Rational Exec or Tornado, the keys directory is:

$APEX_BASE/{c++,ada}/keys/tm_cross.4.2.0

For TestMate Ada Cross for LynxOS, the keys directory is:

$APEX_BASE/{c++,ada}/keys/tm_unix.4.2.0

For TestMate Ada Cross via Apex RCI, the keys directory is:

$APEX_BASE/{c++,ada}/keys/tm_rci.4.2.0

Testing View Imports

Existing TestMate-created testing views will not automatically be updated to import the new testmate.ss API view. Users should delete the testing views.

Note: Testing views are created for execution of certain types of data-driven test lists and for instrumented coverage. TestMate creates these transient views as needed, and they may be deleted by the user without affecting original source code or tests themselves. These views are normally located in the directory <subsystem name>.ss/TestMate.

Upgrading from TestMate 2.4.1 or 2.4.6

Upgrading TestMate Artifacts

TestMate 4.2.0 is upwardly compatible with all TestMate versions 2.4.6 and later. TestMate artifacts created using the earlier versions can be directly used by TestMate 4.0.0 without being upgraded. However, in general, test cases and test lists which have been created or modified using TestMate 3.x or 4.x may not be used again by TestMate 2.4.6.

There is no requirement to explicitly upgrade any of the TestMate artifacts (test cases, test lists, etc.) for 4.2.0.

With TestMate 3.0.0, a summary field was added to test cases and test lists. The tmupgrade command (described below) can be used to initialize this field to an initial value based upon the first line of the description field. In addition, the switches FT_HIDDEN, FTGEN, STGEN, LIVE_UNITS, and TEMPLATE_PLACEHOLDER_SUB are no longer used by TestMate and will be deleted from any TestMate.sw files by the tmupgrade command. It is recommended that tmupgrade only be used if it is desirable to initialize the summary field or these switches have been defined in the TestMate switch file (warnings will be issued by TestMate commands if these switches are encountered; leaving them defined will have no other affect besides issuing these warnings).

The tmupgrade command can be run either directly from the command line in an Apex xterm, or indirectly via a dialog interface accessible from the TestMate > Misc > Upgrade to new release... menu.

Dialog Option	tmupgrade switch	Description
Delete Obsolete files	+/-delete	Deletes TestMate files which become obsolete in this release and cannot be upgraded. Default: TRUE
Display changes only...	+/- effort_only	Only indicate what would be done; does not change/delete anything. Default: FALSE
Update switch model	+/- update_switches	Updates the TestMate.sw file to be compatible with this release. Default: TRUE
Initialize summary field...	+/-update_summary	Sets the summary field of the test case/list to contain the first line of the description field. Default: TRUE

The objects operated on can be views, directories, or any TestMate specific object. In the case of a directory or view, all objects contained within it will be processed, recursively processing any subdirectories.

Objects which are under Apex CMVC control will only be modified if the proper TestMate CMVC switches are set in the view in which the objects reside, or if the global session switch is set, or if they are currently checked out. See the TestMate switch information for AUTO_CHECKIN and AUTO_CHECKOUT.

Only test cases, test lists, and TestMate.sw files will be examined, and, if necessary, modified. Other files will be ignored.

---

What's New in TestMate 4.2.0

Numerous improvements have been made in instrumented coverage to handle source release variants, though some of these changes will benefit all users. TestMate's instrumented coverage feature now handles units that are imported by the testmate.ss subsystem itself, and units with library unit pragmas such as pragma Pure. Also, the user can now create subprogram tests for units arbitrarily deep in the Ada closure of the test list view.

The mechanism for manipulating instrumented library contexts has been completely reworked to maintain the integrity of instrumented code. The instrumented "state" is now saved in TestMate-created libraries (both views and Rational subsystems) so that TestMate can better determine when it can re-use existing instrumented code.

Many changes have been made to eliminate or reduce unneeded processing in TestMate. Processing of the Ada closure has been minimized to cases where it is really needed. In addition, ASIS library opens are now conditionally cached when possible. The results of some internal lookups are also cached. In some scenarios, these changes dramatically decrease the time it takes to run tests, especially for nested test lists of Ada test cases.

---

What's New in TestMate 4.0.0

The most obvious change in TestMate 4.0.0 is support for Apex's integration with ClearCase. Refer to subsequent sections for changes introduced in TestMate 3.0 and 3.2.0.

ClearCase Support

TestMate 4.0.0 offers full support for both Summit views and ClearCase style Rational subsystems.

Note that Apex 4.0.0 introduces the term "library context". A library context can take either of two forms:

• Summit view (.wrk or .rel)
• Rational subsystem (.rss)

TestMate adopts the same terminology. In general, any TestMate operation that applies to a summit view will apply equally to a Rational subsystem. TestMate also supports a new keyword, <library_context> (alias name <lc>) which resolves to the library context. For compatibility, existing keyword <view> will resolve to the library context, even for a Rational subsystem.

Instrumentation Improvements

• Code instrumentation is more robust, for both code de-encapsulation (used in data-driven tests and ada subprogram stubs) and instrumented coverage.
• Improved handling of complex import closures, especially mutual imports.
• Problems with decision coverage have been fixed.
• Instrumented coverage now handles Ada95 protected types
• Fewer dependencies on the runtime in instrumented coverage harness.

Smart Output Comparison

A "smart" output comparison script, tmeval_partial_compare, was introduced in TestMate 3.2. This script has been completely rewritten to provide diagnostics to locate differences between the files. In addition, a new test case prototype is provided to use this feature.

Improved TestMate Setup

TestMate setup has been made simpler to use and less error-prone, especially for use with Apex Embedded:

• TestMate > Setup dialog no longer requires separate TestMate models.
• Apex Embedded users no longer need to use the tmbuild command.

---

What's New in TestMate 3.2.0B

TestMate 3.2 introduced a few usability improvements and some modest new features.

Sample Evaluations

TestMate 3.2 provides more samples for evaluation actions. These can be used directly, or used as a starting point for writing your own evaluation actions.

For program test cases, new scripts are provided:

tmeval_output_compare	Evaluation based on simple file difference
tmeval_partial_compare	Provides "smart diff" feature

For Ada test cases, new procedures are provided:

tms_evals.output_compare	Supports test case evaluation based on the program's standard output
tms_evals.text_output_compare	Same evaluation when standard output is known to be text (provides line-based diagnostics)
tms_evals.gang_results	Mechanism to handle multiple result registrations

Detailed Coverage Reporting

The coverage object type has been added to TestMate Shell to support very detailed output of coverage information.

In addition, a sample report script tmreport_coverage provides an example that extract data from coverage objects (i.e., .cov files).

Enhanced Stubbing Capability

TestMate 3.2.0B provides additional capability for stubbing Ada code.

• Now supports stubbing subprograms declared "separate"
• Now supports stubbing library-level subprograms (i.e., subprograms that are not part of a package)

---

Defects Resolved in Release 4.2.0

The following defect and feature enhancements have been resolved in TestMate 4.2.0.

Defect #	Description
2810, 4034, 7455, 7459, 7474	Several fixes for the TestMate>Setup dialog
3959	TestMate Shell now distinguishes test list group type
3989,7860	Generate legal placeholders call when value determined at runtime for C++ test
4673	MCDC coverage now handles "not" operations in LHS of short-circuit operations
5620	Create unique name for a failsafe check instead of using Program_Error
5995	Handle case of constrained array subtypes as subprogram arguments
6444	MCDC coverage now handles protected types
6793	Running test list under debugger does not work for LynxOS target
6921, 7611	Enhancements in tmreport_coverage tool
6949	Handle embedded new-line characters in stub param specifications
7218	TestMate now treats deferred constant as a literal
7231	Detect attempt to run a shell script file under the debugger
7334	tm_examples.ss fixed for Tornado targets
7429	Fix problem in trap-based coverage with tasking programs
7444	Incorrect harness was shipped for LynxOS targets
7472	Request for awk script compatibility with Gnu Awk 3.0
7473	LynxOS customization key should not assume NFS
7536	TestMate>Setup is now no longer sensitive to case of testmate switch
7586	Request to print test case requirements field
7587	Testing towers could be spoofed by subsystem names
7612	Improvement in log messages
7618	Unexplained pause when analyzing MCDC decision pairs
7624	Coverage kind message is now a note, not a "positive" message
7625	Scripting problem in a utility shell script
7693	Misleading "missing test list element" message removed
7783	Coverage data is now retained in the case of an unhandled exception
7789	Coverage pool overflow message is no longer lost (rx, vw only)
7801	TestMate now correctly uses specified storage for instrumented coverage
7828	TestMate>Setup now correctly recognizes C++ testmate.ss import
7842	Clean up temporary files used to manage instrumented towers
7851	tmlink now recognizes successful link of C++ mains
7855	TestMate>Setup now uses correct path to search for lx harness
7896	TestMate>Setup now uses correct path to search for rx harness
7904	Instrumentation now creates correct tower for Rational subsystems
7905	Editor now correctly defaults the values for missing numeric fields
7909	Deencapsulation with multiple mutual imports for data-driven tests
7914	Handle anonymous subtypes as conditions of case statements and loops
7924	Remove extra layer of parenthesis when generating parameter qualification
7934	Restore test-list-specific switches between lists in a test list group
7966, 8216	Use MCDC algorithm for decision coverage following if statement
8111	Fixed MCDC coverage of a return statement following case statement
8196	Missing required key for LynxOS targets
8225	Invalid key setting for LynxOS target
8330	Control Tests subdirectories when setting up for ClearCase

---

Defects Resolved in Release 4.0.0

The following defect and feature enhancements have been resolved in TestMate 4.0.0. Some of these are available in TestMate 3.2.0B in the form of patches.

Defect #	Description
4170	Testing tower update attempts to update imports of predefined views
5428	Support defining an image function in Ada data-driven test cases
5452	"Ada Data Driven Tests, pattern generation wrong for subtypes"
5454	Visit Body operation is missing in Testmate data driven list
5469	"TestMate insists on de-encapsulating private types, even if no body"
5516	Stubbing subprograms both in private part and body fails
5534	Column widths should persist
5587	Add support for stubbing library-level subprograms
5621	Break up generated code for Tms.Assert in Ada data-driven tests when very large data values are involved.
5626	Handle named associations for de-encapsulated record components.
5641	Base views not frozen at post_install
5654	Handle mutual imports in Ada closure when de-encapsulating types
5684	Remove Korn dependencies from TestMate > Kill Test function
5713	Cannot enter ""abort foo"" in the epilog field for an Ada TC"
5719	Fix a potential re-entrancy problem in running tests
5722	extraneous coverage debug messages
5737	Getting previous TC result from 1st TC fails
5744	Handle declaration-only pragmas when instrumenting coverage
5755	Cannot set test case OUT parameter values from the command line
5789	TM_CUSTOMIZATION_KEY is wrong in delivered cross models
5795	Add output of "actual" for small record types in Ada data-driven tests.
5829	Failed assertion: usage error in tms assert command
5882	Assign TMS memory pool to BSS (affects cross only)
5922	Eliminate gratuitous creation of testing tower if spec & body are in different subsystems
5963	Failing to create a usable TestMate view in 3.2.0b
5979	Testmate harness and model for VADS RCI
6034	Restore two files required to build trap-based coverage analyzer for Apex cross
6064	Eliminate unnecessary warnings for TestMate > Setup
6134	Test Coverage for imported RCI view is zero even though it should be 100%
6225	tmlink does not work in batch mode
6279	TestMate: Decision Coverage reports invalid coverage results.
6312	pragma.1.ada needs to get into testing tower
6321	TestMate fails to instrument choices with \QFirst and \QLast
6423	Upgrading Test Lists from 3.0.0b to 3.2.0b - deletes part of the test
6448	MCDC coverage does not handle package renaming declarations
6697	Request to support fixed target directory (not nfs mount) for LynxOs
6774	Cannot comment the final \Qwith' clause of a stubbed package
6802	Testmate: unable to run a test case in the debugger with a config file
6822	Subunit cannot be stubbed

---

Known Limitations

Program Test Cases and RCI Environments

Program test cases are used to drive executables. In the case of RCI, these executables may be text or binary files that need to be downloaded to a target processor.

TestMate may not locate these executables properly in some RCI environments. In order to make program test cases execute properly for RCI targets, make sure you do the following:

For each test driver in the RCI directory, create a symbolic link from the target test driver to one of the locations on your test driver path (as specified in your TEST_DRIVER_LOC switch). The name of the symbolic link _must_ match the name of the file containing the Ada main procedure, minus the ".2.ada" suffix.

For example, if you have an Apex RCI to VADScross environment, and have a test driver named foo.vox, whose main subprogram is located in <view>/Tests/drivers/foo.2.ada, you must create a symbolic link from the file foo.vox to <view>/Tests/drivers/foo:

% ln -s <target_rci_directory>/foo.vox <view>/Tests/drivers/foo

(when executing the "ln" commands, you should substitute the correct values for the placeholders shown above).

When locating the Ada source for a test driver, TestMate first adds ".2.ada" to the end of the test driver and starts traversing the Ada library from that point (in other words, it assumes that the test driver executable file is located in the same location as the corresponding .2.ada file, and that they have the same simple name).

By creating a symbolic link from the executable to the place that TestMate expects to find the file (and with the correct name), you essentially work around a limitation that TestMate would otherwise have with running program test cases for RCI targets.

Test Result Group

Coverage Summary of Test Result Group Coverage

The test result group display does not summarize the coverage totals across all test results in the display. To obtain a summary, select all the test results in the image and use TestMate > Test Evaluation > Coverage Summary to get a coverage summary.

Graphical User Interface

Match/Filter

The CONTROLS operators only work for textual comparisons. Numeric comparisons such as filters for all "coverage < 80%" will not work.

Skeleton test cases

The test case signature and pattern information for a test list is kept in a special test case associated with every test list. This special test case has the string "_skeleton.tc" as part of its file name. You should never delete this test case nor attempt to manually insert this test case into a test list.

Missing test list elements

If elements (test cases or test lists) referenced by the test list (test list group) have been removed, relocated, or had their permissions changed, you may not be able to visit the test list. Depending on which element is missing, one of the following situations may arise:

• The test editor appears, but some of the elements are missing. In this case, an error message in the message output window indicates which element(s) were not found. If the test list is subsequently modified and saved, the reference to the missing items will be removed from the test list. If you want to keep the reference (because you may be able to remedy the situation by restoring the elements), exit the Test Editor without making any changes to the test list. The test list will still have the references to the troublesome items. Once the problem has been fixed, you should be able to visit the test list once again.
• Error messages indicate that the skeleton test case is missing and the test list open failed. In this case, you must copy one of the other test cases in the list to the name of the skeleton in the error message. Usually, this name is the base name of the test list (without the ".tl") with "_skeleton.tc" appended. You can dump the test list using /bin/cat or more to determine the other test cases in the list. Once you've restored the missing skeleton test case, you should be able to visit the test list.

Test List Editor and Test Case Cell Values

Prior to TestMate 3.0 the test case driver, evaluation action, epilog, precondition and prolog fields had three fields each. These were the Result field, Procedure field, and Parameters field. These have been collapsed into a single field which is parsed internally to derive the result, procedure and parameters fields. This section describes the language constraints associated with these cells.

There are three varieties of syntax which can appear in one of these test case fields. The syntax is described below using quasi-regular expressions. In the expression to follow a set of bracket enclosing an element means the element is optional. A \Q*' symbol following an element means the element can appear 0 or more times.

Program Test Cases

The format for program test cases is:

[<result> =] [<procedure> [<arg> [<arg>]*]]

<result> is optional, arguments can only appear if a valid <procedure> value is present, and the null string is accepted.

Ada Test Cases

The format for an ada test case is:

[<result> :=] [<procedure> [ ( <arg> [,<arg>]* ) ]]

<result> is optional, arguments can appear only if a procedure appears and must be enclosed in parens, and the null string is accepted. Further the values for <result> and <procedure> simple names.The values for <result> and <procedure> may be simple identifiers, or identifiers with selectors, for example, r = p.f(x).

C and C++ Test Cases

The format for a C or C++ test case is:

[<result> =] [<procedure> ( [ <arg>, [<arg>]* ] ) ]]

Parameters can only appear if a procedure appears and must be enclosed in parenthesis. The parenthesis must appear even in the absence of arguments.

This differs from the ada syntax above. The values for <result> and <procedure> may be simple identifiers, or identifiers with selectors, for example. r := p.f(x). The scope resolution operator is also available for <procedure> values, for example, r := c::p.f(x).

Data-Driven Testing - C/C++

TestMate does not currently support data-driven test lists for C or C++ units. Unit testing may still be performed using C and C++ test cases by writing your own driver.

Data-Driven Testing - Ada Test Cases

This section describes limitations of the data-driven testing features of TestMate. If these limitations apply to an Ada unit that you wish to test, consider using a non-data-driven Ada test case instead, and write an Ada driver manually. For more information see the chapter on "Ada Test Cases" in Using TestMate.

Note: Some limitations relate to TestMate's ability to de-encapsulate private types in data-driven test cases. Normally, TestMate allows you to enter literal values of any parameter in the test editor, regardless of whether or not the parameter's type is directly visible (any literal value that is entered is interpreted as if the literal were located immediately following the full type declaration). However, in some instances TestMate cannot generate code to de-encapsulate literals. In these instances you can still enter values in the cell that follow normal Ada visibility rules, for example, you can enter a functional call that returns a value of the private type.

Task Types

TestMate will either fail to generate a compilable script, or it will fail to generate the script for subprograms which have task type. (including function return types).

This restriction also applies to stubs with task type parameters or function return types.

Unconstrained Output Parameters

Unconstrained output parameters are not supported due to a current limitation in creating an appropriate constrained output target object. An example would be an out parameter of string type.

Anonymous Arrays

Global variables of an anonymous type are not yet supported (for example Xarr : array(1..10) of integer;). The user is encouraged to create a declared type and use that instead of the anonymous type:

type Xarr_Type is array(1..10) of integer;
Xarr:Xarr_Type;

Tasks

This release does not support data-driven tests of tasks. All other test cases support testing of tasks and task types (for example, an Ada test case may call a custom driver that calls task entries).

This restriction also applies to stubs -- automated stubs cannot be tasks.

Hidden Variables & Subprograms in Generics or Subunits

Automatic pattern generation for hidden global variables and testing of hidden subprograms is not supported for generic instantiations or units under test that have subunits.

Stubs and Generics

Stub values for generic instantiation subprograms are not yet supported. The user can workaround this limitation by declaring the generic instantiation unit and generic unit as live.

De-Encapsulation in Generics

A private type in a generic specification cannot be de-encapsulated. Note that generic formals used in instantiations can be private types with proper de-encapsulation.

Composites with Surrogate Components

If a composite type (record or array) includes a component which would require a surrogate routine (e.g., for stubbing or de-encapsulation), then the composite type requires a de-encapsulation. This de-encapsulation is not yet supported. Note that de-encapsulation is also required if the user tests a floating point component using a "range" test.

Unconstrained Private Records

A private type with non-default initialized discriminants cannot yet be handled with a de-encapsulation. A warning will be issued and the user should use external values of this type as parameters (rather than having TestMate create a surrogate based on the user-entered literal value).

Data-Driven Ada Tests - Test Data Generation

Limitations on Parameter Types

Test data generation supports subprograms with parameters of most Ada types, or types derived from them. However, the following types are not yet supported:

• arrays (except strings)
• task types
• incomplete types

Limitations on Certain Program Units

TestMate cannot currently generate test data for the following program units:

• tasks
• generic program units (generic instantiations are supported)

Coverage Analysis

Trap Based Coverage Analyzer Decision Limitation with Tasking

A window exists in which decision coverage can be missed due to two tasks executing the same object code decision at the "same time". If each task takes a different decision target, it is possible that one of the targets will not be recorded, resulting in no coverage for that decision when in fact there was coverage.

C/C++ Statement and Decision Coverage

This is currently not supported.

Ada Source Line Coverage

This is currently not supported. Statement and decision coverage provide more information than line information, and should be used instead.

In-lined Subprograms

Trap based coverage analysis is not supported for inlined subprograms for both Ada and C/C++.

C++ Templates

Coverage for templates is currently not supported.

Ada Generics

In order to collect trap based coverage on a compilation unit which contains, or is itself, a replicated generic, one must specify the following as units under test:

• The generic unit
• At least one instantiation of that unit

Optimized Code

Under some conditions of optimized code, source level decision coverage using the trap based coverage analyzer may indicate that a decision has been optimized away if the explicit target of the decision has been optimized away.

Trap Based Incorrect "Optimized Away" Message

Elaboration segments/decisions for generics are incorrectly annotated as having no object code. The source span for these elaboration segments covers the entire spec or body. If one displays the Ada source coverage based upon the summary totals for a unit which has 0% coverage for the elaboration code, the entire unit will be highlighted as uncovered. The coverage totals for each unique subprogram are correct, and as a workaround, one can display the Ada source coverage on a subprogram by subprogram basis (just not the entire unit at once).

Under some conditions, a return statement which starts a segment will be highlighted as optimized away, when in fact it has not. Under this condition, there is no way of knowing whether or not it was executed. There are two simple conditions in which this is known to occur - returning the value passed and returning a literal. For example:

function Foo (A_Value : Boolean) return Boolean is
begin            
  return A_Value;
end Foo;
function Foo return Boolean is
begin 
  return True;
end Foo;

A work around is to preceed the return statement with another statement which causes code to be generated.

Multiple UNIX Processes

Coverage analysis is not supported outside the normal Ada tasking implementation; i.e. if a user program (Ada, C, or C++) performs a UNIX fork thereby creating another UNIX process, coverage analysis on the new process is not supported and may result in erroneous coverage results for the originating process.

C/C++ Threaded programs

Coverage for C/C++ programs which use threading is not currently supported.

C/C++ Test Cases

CM/Timestamp for C/C++ Units

CM information is not collected for C/C++ units specified as units under test.

Test Case #Includes

In the C/C++ test case - Includes section, use of the notation "#include <myfile.h>" will result in a warning message during script generation:

!!! Unknown Placeholder <myfile.h> ignored

This warning can be ignored.

MCDC Coverage Analysis

MCDC_Analysis Primitive and Test Case Generation

Use of the "Test Case Generation" option with the MCDC Analyzer primitive may cause premature termination of this operation.

Coverage File Report Format Error

When generating a coverage file report that includes a CASE statement with a Boolean expression the display for this decision points will include an error message. Subsequent metrics information will be off by the condition totals for these expressions.

Pair Table Display for large Boolean Expressions

The maximum number of conditions supported for any single boolean expression is 24 for this release. Expressions with more then 24 subconditions are rejected with an appropriate error message.

Strong MCDC

The default algorithm utilized to generate boolean pair tables does not recognize coupled subconditions. Coupled subconditions are subconditions that are directly dependent upon each other (such as two subconditions that are the same boolean variable). The result of this is that when coupled subconditions are present, impossible to achieve pairs will be displayed and complete coverage will not be attainable without manual evaluation.

One solution is to restructure the code to remove (or test for) coupled conditions.

Overloading Package Standard

Overloading package standard may hinder the instrumentation of source level constructs that require type conversion. For instance, instrumentation of an expression of a type derived from boolean will attempt to convert the expression to type "standard.boolean" which may fail at compilation time if the overloaded standard package does not properly remap type boolean to the expected predefined type.

Nested Generic Instantiations

Visible objects in nested generic instantiations (more then one level deep) may type is based on a generic formal whose actual type is based on standard.boolean fail to instrument properly. The type used in type conversions is the formal type name instead of the actual (based on boolean) type name.

Floating Point Type Attributes used as CASE Choices

Floating point attributes used as CASE statement choices are not currently statically evaluated by MCDC processing. For instance:

case My_Obj is       
 when My_Float'Mantissa | My_Float'emax =>                
  null;        
 when others =>                
  null;
end case;

In the above example the \QMantissa and \QEmax attributes will be rejected with an appropriate error message and processing will continue.

Categorization Pragmas

MCDC instrumentation will fail to produce a compilable program if a unit with a pragma pure declaration is not instrumented and it "withs" a unit with the declaration that is instrumented. The same limitation applies to pragma shared-passive.

C/C++

Exceptions

Drivers which terminate with an Unhandled exception (C++) are not correctly processed by the Test Execution Server. A workaround is to change the default value of the FAILED_WITH_EXCEPTION status mapping. This should be set to the value 32774. See "User-defined Results" in the TestMate Customization and Configuration Guide for more information.