The testing process divided into a well-defined sequence of steps is termed as software testing life cycle (STLC). The major contribution of STLC is to involve the testers at early stages of development. This has a significant benefit in the project schedule and cost. The STLC also helps the management in measuring specific milestones.
STLC consists of the following phases (see Fig. 1).
The goal of test planning is to take into account the important issues of testing strategy, namely, resources, schedules, responsibilities, risks, and priorities, as a roadmap. Test planning issues are in tune with the overall project planning. Broadly, following are the activities during test planning:
- Defining the test strategy
- Estimating the number of test cases, their duration, and cost
- Planning the resources such as the manpower to test, tools required, documents required
- Identifying areas of risks
- Defining the test completion criteria
- Identifying methodologies, techniques, and tools for various test cases
- Identifying reporting procedures, bug classification, databases for testing, bug severity levels, and project metrics.
Based on the planning issues, as just discussed, analysis is done for various testing activities. The major output of test planning is the test plan document. Test plans are developed for each level of testing. After analysing the issues, the following activities are performed:
- Developing a test case format
- Developing test case plans according to every phase of SDLC
- Identifying test cases to be automated (if applicable)
- Prioritizing the test cases according to their importance and criticality,
- Defining areas of stress and performance testing
- Planning the test cycles required for regression testing
One of the major activities in testing is the design of test cases. However, this activity is not an intuitional process; rather it is a well-planned process.
The test design is an important phase after test planning. It includes the following critical activities.
Determining test objectives and their prioritization: This activity decides the broad categories of things to test. The test objectives reflect the fundamental elements that need to be tested to satisfy an objective. For this purpose, you need to gather reference materials such as software requirements specification and design documentation. Then, on the basis of reference materials, a team of expert compile a list of test objectives. This list should also be prioritized depending upon the scope and risk.
Preparing list of items to be tested: The objectives thus obtained are now converted into lists of items that are to be tested under an objective.
Mapping items to test cases: After making a list of items to be tested, there is a need to identify the test cases. A matrix can be created for this purpose, identifying which test case will be covered by which item. The existing test cases can also be used for this mapping.
This matrix will help in the following:
(a) Identifying the major test scenarios
(b) Identifying and reducing the redundant test cases
(c) Identifying the absence of a test case for a particular objective and, as a result, creating them.
Designing the test cases demands a prior analysis of the program at functional or structural level. Thus, the tester who is designing the test cases must understand the cause-and-effect connections within the system intricacies. However, according to Tsuneo Yamaura- There is only one rule in designing test cases:
Some attributes of a good test case are given below:
(a) A good test case is one that has been designed keeping in view the criticality and high-risk requirements in order to place a greater priority upon, and provide added depth for testing the most important functions.
(b) A good test case should be designed such that there is a high probability of finding an error.
(c) Test cases should not overlap or be redundant. Each test case should address a unique functionality, thereby not wasting time and resources.
(d) Although it is sometimes possible to combine a series of tests into one test case, a good test case should be designed with a modular approach so that there is no complexity and it can be reused and recombined to execute various functional paths. It should also avoid masking of errors and duplication of test-creation efforts.
(e) A successful test case is one that has the highest probability of detecting an as-yet-undiscovered error.
Selection of test case design techniques: While designing test cases, there are two broad categories, namely black-box testing and white-box testing. Black-box test case design techniques generate test cases without knowing the internal working of a system. This will be discussed 'later in this chapter. The techniques to design test cases are selected such that there is more coverage and the system detects more bugs.
Creating test cases and test data: The next step is to create test cased on the testing objectives identified. The test cases mention the objective under which a test case is being designed, the inputs required, and the expected outputs. While giving input specifications, test. data must also be chosen and specified with care, as this may lead to incorrect execution of test cast cases.
Setting up test environment and supporting tools: The test created above needs some environment settings and tools, if applicable. So details such as hardware configurations, testers, interfaces, operating systems, and manuals must be specified during this phase.
Creating test procedure specification: This is a description of how the test case will be run. It is in the form of sequenced steps. This procedure is actually used by the tester at the time of execution of test cases.
Thus, the hierarchy for test design phase includes: developing test objectives, identifying test cases and creating their specifications, and then developing test case procedure. specifications as shown in Fig. 2. All the details specified in the test design phase are documented in the test design specification needs, and other procedural requirements for the test case.
In this phase, all test cases are executed including verification and validation. Verification test cases start at the end of each phase of SDLC. Validation test cases start after the completion of a module.
It is the decision of the test team to opt for automation or manual execution. Test results are documented in the test incident reports, test logs, testing status, and test summary reports, as shown in Fig.3. Test incident is the report about any unexpected event during testing, which needs further investigation to resolve the bug. Test log is a record of the testing events that take place during the test. Test summary report is an evaluation report describing summary of all the tests and is prepared when the testing is over.