Component test runs

After you generate a test, supply it with data, and possibly customize the test, you want to know the test results. You get the results by running the test.

A component test run can include a test suite, a test case, or a combination of multiple test suites and test cases. You use run configurations to select the tests to run. You can also use run configurations to engage debugging or profiling on a test run.

During the component test run, a temporary project is created in your workspace for internal purposes. By default, this project is deleted after the test run. You can change this behavior in the Component Test panes in the Preferences window.

After a component test run, you can see the results in the Run folder of the Test Navigator view. If the results do not meet your expectations, you can modify the component-under-test or the test itself and rerun it as many times as needed until you get the expected results.

On the first component test run, each test case runs all of its data sets. After this however, you can rerun any level of the test, even just a particular data set.

Test data and executions

A test runs with the data you supply in a test data table. If you include sets, ranges, or datapools as input values in the test data table, then running a single test can result in the execution of multiple individual tests. You must take this into account when planning for the time required for running the test.

Test data is partitioned into one or more data sets. These are the columns in the test data table. Each data set contains input values and expected output values. When you run the test, each data set is executed separately which also increases the total time needed to run the test.

Evaluating the number of executions

When a data set involves sets, ranges or datapools producing multiple executions, the test data table displays an execution count icon in the data set header indicating the expected number of executions, as shown in the following table:
Icon Status Description
Number of executions Number The icon indicates the number of executions that can be calculated for the data set.
Multiple executions icon Multiple The test data table was not able to calculate precisely the number of executions. The actual number of executions will be a multiple of the displayed number.
Unknown executions icon Unknown The test data table was not able to calculate the number of executions.
Infinite executions icon Infinite The icon indicates an infinite number of executions. The test will enter an infinite loop.

The execution count icon allows you to evaluate how long it will take to run the test. If running the test will take too long, or will loop infinitely, you might want to consider modifying the number of values in the data set.

Data combinations

Clicking the execution count icon toggles between executing all combinations of input and expected values and executing one to one combinations. To explain this, consider the following example:Screenshot of
Test Data Table with sample values
  • All combinations: All input values are compared to all expected values. This mode produces the highest number of executions. For example, all combinations of the variable values a and b are compared against their respective expected values.
    Running the test with this data set will result in the nine following executions:
    Execution number In a Expected a In b Expected b
    1 1 {0, 1} 0 [0..20]
    2 1 {0, 1} 2 [0..20]
    3 1 {0, 1} 4 [0..20]
    4 13 {0, 1} 0 [0..20]
    5 13 {0, 1} 2 [0..20]
    6 13 {0, 1} 4 [0..20]
    7 0 {0, 1} 0 [0..20]
    8 0 {0, 1} 2 [0..20]
    9 0 {0, 1} 4 [0..20]

    Note the execution icon in the data set column header: Execution icon showing '9'

  • One to one combinations: In this mode, the variable values are initialized in parallel. The first initialization of a is tested in combination with the first value of b, the second value of a is tested with the second value of b, and so on. This dramatically reduces the number of test executions.

    Additionally, when the expected values are sets or datapools (but not ranges), they are also used in a one to one combination. For example, on the first initialization of a the result is compared to the first expected value for a, and so on.

    If the number of values of a set, range or datapool mismatch, the last value of the smaller set is retained. In our example, the expected value for a contains only two values, so the last value is repeated.

    Running this data set with one to one combinations will produce the three following executions:
    Execution number In a Expected a In b Expected b
    1 1 0 0 [0..20]
    2 13 1 2 [0..20]
    3 0 1 4 [0..20]

    One to one combination is materialized with double arrows affixed to the execution count icon in the data set header: Execution icon showing '3' and one-to-one combination arrows

Related tasks
Running component tests
Rerunning part of a component test
Running a component test with profiling engaged
Running a component test in debug mode
Stopping a component test
Feedback
(C) Copyright IBM Corporation 2000, 2005. All Rights Reserved.