A test suite is a collection of test cases that are grouped
for test execution purposes. Test suites can include both manual and
automated tests, but they are most beneficial when you have a series
of automated tests that you can run without supervision. When running
a suite, you can specify whether to run the test cases in the suite
sequentially or in parallel. The following scenario presents one
possible workflow for test suite execution.
Figure 1. Workflow diagram for
test suite execution
- Create test cases.
- Optional: Create test scripts, and associate them with the test
cases.
Note: Test cases are required for test suite execution, but
each test case does not need to have an associated test script.
- Optional: Add the test cases to a test plan.
A test plan helps
to organize test execution activities, but it is not a requirement
for test execution. By including both the test cases and the test
suite in a plan, you can plan for the execution of the suite as part
of the testing activities for the plan. Test suite execution records
run in the context of the test plan and test plan iterations. Also,
all test suite execution results (including test case execution results)
are associated with the test plan. All reports that you run against
these results will include the associated test plan information.
- Analyze the existing test cases and test scripts in your environment
to determine which ones will be part of the new test suite.
Ideally,
each test case in a suite represents a piece of a scenario, such as
the elements that simulate the user completing a transaction. For
example, a test suite could contain four test cases, each with a separate
associated test script:
- Test case 1: Log in
- Test case 2: Select an organization
- Test case 3: Allocate dividends
- Test case 4: Log out
- Create a new test suite.
- Optional: Add the test suite to a test plan.
As mentioned in
step 3, a test plan is not a requirement for test suite execution,
but it can help you organize testing activities. If you add the test
suite to a test plan, ensure that the test cases that are included
in the test suite are also part of the same plan.
- Add existing test cases to the test suite.
Note that the order
of the test cases in the Test Case section of a test suite determines
the order in which they are executed in a sequential test suite execution.
When
you add test cases to a test suite, you can optionally assign owners
to each test case in the suite. When you run a test suite that has
owners assigned to test cases, the test suite execution page shows
the owner for each test case in the suite. Testers can filter on the
owner field to see the test cases in the test suite that they should
execute.
- Create a single test suite execution record for the test suite
or generate multiple test suite execution records for the test suite
automatically.
- Optional: Specify an environment for the suite to be executed.
If you only want the suite to test one environment, specify the test
environment at the test suite execution record level. If you want
to test a mixed environment, you must specify the test environments
in the individual test cases and not in the test suite execution record.
Essentially, the environment on the test suite execution record is
the default specification unless it is overriden at the test case
level.
- Run the test suite. You can run a test suite in the following
ways:
- From the Test Suite Executions Records section of a test suite,
select an existing test suite execution record, and click the Run icon.
- Open a test suite, and click the Run icon
at the top of the test suite. Select an existing test suite execution
record, or specify that a new one be created.
- In the Run Test Suite dialog box, specify the execution settings:
- Run this suite in parallel: When you run
your test suite, you can run all test cases at the same time (parallel
mode), or run them one after another (sequential mode). In this workflow,
our test cases represent a sequence of activities (you must log in
before you can select an organization, for example). To run this test
correctly, clear the Run this suite in parallel check
box to select sequential mode. Sometimes you might want to run all
of your test cases at the same time. For example, if the scripts do
not depend on each other and you do not need to execute them in any
particular order. In that case, you can choose parallel mode.
- Stop suite execution if any test does not pass:
With sequential test suite execution, you can stop the test suite
if one of the test cases does not pass. For example, if the test to
log in to the application fails or is blocked, the test suite execution
stops before continuing with the test to select an organization.
- Create Result without Execution: Use this
option to record results for the test suite without executing the
test scripts.
Optionally, you can choose test cases to exclude from the
test suite execution:
- View the test suite execution results, and submit defects as needed.
- Update the test suite execution results as needed and save the
results.