Testing Plan

Purpose

A testing plan sets up the framework to determine if the project deliverables are performing in ways in which they were expected to perform. It establishes both the objectives and the game plan for evaluating project deliverables in part or in whole. It also identifies the participants in the process, their roles, and the environment in which the tests will be conducted.

Application

Initially, the testing plan is used to gain the concurrence of a given audience to the testing approach. It is reviewed by the affected parties to determine if the proper elements are being tested and if the approach to testing truly assesses the efficacy of the element in question. It may be used as a defense of a particular test methodology or tool (or suite of tools) and as a means to validate (or invalidate) assumptions about anticipated test and project performance.

Content

The test plan includes a description of the test objectives, the environment, and the approach and is generally appended with the outcomes of the tests when they are complete.

1.0 Test Objectives

This is a description of what the outputs of the test will be and the project objective those outputs are intended to serve. It may include a description of the level of granularity of the outputs, but generally does not include the anticipated outcome of the test.

2.0 Test Approach

This is a description of what approach will generate the outputs described in Section 1.0 and (as available) any historic reference as to why that approach is acceptable, valid, and/or appropriate. This may also include a list of any tools to be applied or methodologies to be pursued, as well as any deviations anticipated from standard testing procedures (if they exist).

3.0 Environment and Assumptions

Because the test environment will often determine outcomes, a detailed explanation of the project test environment and any assumptions used to establish that environment should be incorporated in the test plan. This may include details on the duration of the testing, geography or physical location, physical environmental considerations, and cultural or organizational assumptions. This will also include any rationale as to why and how the test environment and these assumptions best emulate the real-world application of the project’s deliverables (or the test subject’s outputs).

4.0 Testing Responsibilities

Either by function or name, the parties responsible for various aspects of the testing are identified, including their respective responsibilities.

5.0 Anticipated Outcomes

In some organizations, this element is omitted by standard practice. In others, it is common. Anticipated outcomes identify results the testing is expected to produce. Some organizations omit this component over concerns about possibly tainting the outcomes. Others believe it is a crucial element in assessing organizational capability to determine feasible approaches.

6.0 Outcomes

Not completed until after the testing, this element of the test plan is built in to ensure that an historic record of the project testing is maintained. In organizations where anticipated outcomes are documented, this information may be presented in a comparative matrix with the anticipated outcomes.

7.0 Conclusions

Based on the outcomes, some conclusions about project or deliverable performance can be drawn. Those conclusions should be rooted in the methodology and data and in their relationship to the original test objectives. This often includes a brief summary of the data that led to the conclusion.

8.0 References

Because testing is normally based on scientific approaches or historical information, any references used in developing the test plan, approach, or data should be captured for future reference.

Approaches

The test plan is intended to create an objective environment in which to evaluate project or deliverable performance. As such, it is important to maintain objective criteria throughout. Metrics are an important component of such criteria, and the test plan should work to ensure that the testing methodology adequately tests for the metrics and that the metrics reflect the desired information.

Considerations

Test plans often create an ideal environment for the project element being tested. As such, any differences between the test environment and the real world should be clearly called out in the test plan. Also, any external considerations (e.g., weather, resource constraints, scalability) that do not fit within the testing model should be identified as specifically not having been tested. The considerations that were not taken into account may be as critical as those that were.