Running a test suite

A test suite is a collection of test cases that are grouped to be run together. You can run a test suite by opening the test suite or by opening the Browse Test Suites window.

About this task

You can run the test cases in your test suite in a sequence or in parallel. If you run a test suite in a sequence, a test case runs only after the previous test case in the sequence finishes running. If you run a test suite in parallel, you can run all of the test cases at the same time.

After testing commences, you can pause test suite runs from the Test suite execution console. You might pause test suite runs when the system that you are testing needs minor adjustments before testing can proceed.

When you pause a test suite run, any manual test runs that are part of the suite are also paused. If you pause a test while an automated test is running, that test is completed; however, automated tests that are not started do not run. In either case, the test suite is paused. You can resume paused test suites from the Test suite execution console, Test suite execution record list view, or Test suite execution record editor.

You can stop test suites from the Test suite execution console. Unfinished test cases in that suite are marked incomplete. Stopped test suites cannot be resumed.

Procedure

  1. Open a test suite.
    Note: You cannot run a test suite until it is saved.
  2. In the upper-right corner of the test suite window, click the Run Test Suite icon (Play).
  3. In the Test Suite Execution Record section, either select a test suite execution record or create an execution record by accepting the default option, New Test Suite Execution Record. If you create an execution record, select the test plan, test iteration, and the test environment.
    Note: If you already have an execution record that uses the same test plan, test iteration, and test environment that you specified, that execution record is used and a new one is not created.
  4. Optional: Select a channel. During the run of the test suite, the default script for the channel is chosen by using the keywords in the script. The available channels are based on the selected test environment.
  5. Optional: Select a test cell. If the test suite includes automated test cases, you can specify the test cell to run the test suite. All of the automated test cases are distributed among the test automation adapters that are in the specified test cell.
  6. Optional: If the test suite contains test cases that have automated test scripts, you can select multiple test cases and assign the same adapter to all of them. Select the check boxes that are next to the test cases, and then click the Select a machine to run this test (Select a machine to run this test) icon.
    Note: Most automated test adapters can run only one script at a time. If you select the same adapter for multiple test cases, the scripts that are on that adapter run in sequence. To run scripts in parallel, you must select a different adapter for each script.
  7. Optional: Specify which test cases to exclude from the test suite run. If you do not exclude test cases, all of the test cases in the suite are included in the test suite run. To skip test cases, do one or both of the following actions:
    • If you selected a test suite execution record, you can skip any test cases that passed when the test suite execution record last ran. Click the Exclude Passed Test Cases (Exclude Passed Test Cases) icon.
    • To skip specific test cases, select the test cases in the table and click the Exclude from Execution (Exclude Execution) icon.
    All of the test cases that you selected are marked with an icon in the Exclude from Execution column and are not included in the test suite run. If you decide to include all of the test cases in the test suite run, select the test cases again and click the Exclude from Execution (Exclude Execution) icon.
  8. To associate the results of this test suite run with a build record, next to the Build Record option, click the Change Associated Build icon (Change Associated Build) and select a build record. The execution result from this test run is automatically linked to the build record that you select.
  9. To schedule the test suite to run at a future time, select Schedule Execution and select a date and time.
  10. Select whether to run the test cases that are in the suite in parallel.
  11. If you selected the option to run the tests in a sequence, you might select more options:
    • If your test scripts use execution variables and you want to pass the values between scripts during the test suite run, select Pass execution variables between scripts.
    • To specify that the test suite must be stopped if a single test case does not pass, select Stop suite execution if any test does not pass. This option is useful if you are running a test case in a test suite that depends on the success of previous test cases.
  12. To record a result for this test without running the scripts, select Create Result without Execution and then select a result.
  13. To temporarily modify the values of the execution variables that are defined in your test suite, click Modify Execution Variable Value(s).
  14. Open the Test suite execution console by clicking Run.
    Test suite execution console
    If you are running tests in a sequence, the first test case in the sequence is shown in the console. If you run tests in parallel, the automated test cases start to run and the manual test cases or test cases without scripts are shown in the console.
  15. If the first test case in the suite is a manual test, click Start Manual Test. If the first test case in the suite does not have an associated test script, start the test by clicking Run Test Case in the Test Case Progress column.

    If the first test case in the suite is an automated test, the test runs automatically.

  16. For manual tests, follow the steps in the manual test execution window, and when you are finished, click Close. If a test case does not have a script, record a result for the test case and then click Save and Close. If you have manual scripts or test cases without scripts and you are running the test in parallel, you can return to the Test suite execution console at any time and start to run another test case in the suite.
  17. When a test case is completed, you can view the execution result for the test case by clicking the icon in the Result column. The execution result includes the editable Actual Result field and more details about the testing artifacts and testing environment that is associated with the test case.
  18. Update the execution results as needed, and be sure to save your updates. For example, you can provide a detailed description of the test results in the Result Details section.
  19. To submit a defect, in the Change Requests section, click the Create New Defect (Create new defect) or Link to Existing Defect (Link to existing defect) icons. In the defect form that opens, type information about the problem, and then save the page.
    Tip: If you configured Rational® ClearQuest® as your defect provider and you use Internet Explorer, when you try to create or link to a defect, a Security Warning window opens. To continue to the defect form,, click Yes in Internet Explorer 7; in Internet Explorer 8, click No.
  20. Run the remaining test cases in the suite. If you want to finish running the test suite later, or if another tester is responsible for the next test case, close the page. You can return to the test suite execution later.
    If you close the test suite execution console during a test suite run, you can return to it in any of these ways:
    • Click Execution > Browse Test Case Execution Records and then click the Last Result icon.
    • Click Execution > Execution Console and find the test case that must be run as part of the test suite. In the Test Suite Progress column, click the link.
    • Open a test plan and in the Test Suite Execution Records section, click the Last Result icon.

Results

To view the results later, from the main menu, click Execution > Browse > Test Suite Results.


Have a problem or a question? Post it in the Jazz.net forum.