Start With a Strong Testing Foundation
To End Up With a Quality Product, Your Team Must Begin With a Good Set of Testing Practices
By Kiran Vankatesh
Software Test and Performance Magazine
Volume 5, Issue 3, March 2008
Quality is a key differentiator for business growth. Good software testing practices help ensure a quality process, which ultimately leads to quality software. These good practices include understanding requirements, a well-prepared test strategy and continuous enhancement of the test suite. It's also important to know when testing is complete - something usually based on specific indicators of product readiness.
This article addresses the fundamentals of the testing process, explores the good practices of product testing and offers recommendations for a phased approach to component-, feature- and system-level testing.
Testing effectiveness can be improved by testing with all possible combinations and techniques. However, it's difficult to do this and sometimes impossible to ensure that the product has zero defects.
There are various approaches to improving the effectiveness of testing an application. Can it be installed fro a CD? What other factors or inputs are going unseen or unchecked? It's easy to overlook such factors. But as software testers, we need to be aware of and test them, because they may not always function as expected.
Inputs should be selected uniformly among the different input domains. If the testing effort is limited due to cost or schedule constraints, I suggest that you base your testing on highly used input factors only. These include product specification, design documents, product reviews, competitive information, schedules, feedback from previous versions, customer surveys, look-and-feel specifications, software architecture, test plans, usability data and software code. By focusing on these factors, testers can adequately determine operational reliability.
Processes used by testers are verified by the software quality auditor, who also ensures that the testing processes are followed correctly. This involves an internal audit of processes from the preparation of test case documents to the release of the product. Central to the process is to measure the quality of [the] developed application and to authenticate its correctness, completeness, security, capability, reliability, efficiency, portability, maintainability, compatibility and usability.
In terms of technical investigation, the process can be performed on behalf of the stakeholders and may also include reviews, walkthroughs or inspections.
The testing team should receive new builds at regular intervals, each new build containing defect fixes identified in the previous release. For every release, the testing team must follow the same practices to help minimize testing time and increase efficiency.
End products of good quality blend user satisfaction, compliant product, good quality and delivery within budget/schedule. An end product should be reliable, modifiable, understandable, efficient, usable and portable. Software is developed to fulfill the specific needs of a person or a group; i.e., the customer. Part of the testing team's job is to understand the customer's business model, wants and needs. This can be done by using surveys, direct feedback and face-to-face meetings.
Once this information is understood, the exact approach for testing can be determined without assumptions. The only assumption the testing team should ever make, in fact, is that of the customer's perspective, such that it reveals any defects or diversions from the application's intended purpose.
DEVELOP TEST CASES EARLY
One good practice is to start developing test cases during requirement analysis. Once the requirements are available, the testing team can write high-level test cases to demonstrate an understanding of customer requirements and business scenarios. As the project progresses, test cases should adhere to the test strategy, performance of the tests and enhancement of the test suite, and adherence to quality and value additions. Test case documents can be provided to the customer for continual review and approval.
Once software begins to be sent to the testing team, test cases are executed and the test case documents are updated for features that weren't clearly defined in the requirements document. The team confirms that the test case documents comply with the requirements. Documents should be maintained in a CMS tool to ensure that document versions and changes are tracked.
DESIGNATE AN AUTOMATION TEAM
Setting aside resources for an automation team saves manual testing time and resources. The team should be dedicated not just to development of the automated testing, but also to maintaining incremental advancements of the product. Regression testing also can be assigned to this team.
When frequent builds or executable files are to be tested, test automation in an organization plays a vital role in reducing manual testing time. Specialized automation testers are required to maintain a structured testing setup. This would require proficient and certified testers for advanced testing methodologies and comprehensive usage of popular testing tools.
UNCONVENTIONAL TEST MATRICES
An unconventional test matrix (decision table) is used for testing complex functions and frequently needs regression testing. The unconventional test matrix provides insight into the important concept of the product testing life cycle. Monitoring the specifications in the requirements document and verifying that all requirements have been met by the time of release can be cumbersome and laborious.
If this aspect of the testing life cycle isn't considered high priority, it can result in confusion and arguments between the QA team and stakeholders. This problem can be solved by using a traceability matrix.
Created before test cases are written, the traceability matrix is a complete listing of all that has to be tested. Sometimes that means one test case for each requirement, or several requirements that can be validated by only one test scenario. This depends on the kind of application that is available for testing. Irregular test matrices or decision tables for testing complex functions can be created whenever it's necessary during the testing activity using matrix elements such as requirement, function specification, design specification, source code files and test cases.
A traceability matrix provides a cross-reference between a test case document and the functional/design specification document. This can show if the test case document contains tests for all the identified unit functions from the design specification. From this matrix, the percentage of test coverage can be collected, taking into account the percentage of functionalities in the ratio of total-tested to total-untested.
"If you don't identify defects, the customer will." This well-known axiom is likely to pop up whenever a software product isn't appropriately tested. Releasing a product with defects can occur for many reasons. But common to all scenarios is speculation about tested and untested parts of the software. The traceability matrix is an important tool for helping avoid this scenario.
DEFINE ROLES AND RESPONSIBILITIES
A vital practice for a high-quality product is to divide the product into modules and assign them to individuals. Define roles and responsibilities that allow the testers to perform their functions with minimal overlap and without uncertainty as to which team member should perform which duties. One way to divide testing resources is by specialization in particular application areas and nonfunctional areas.
The testing team also may be able to take on more roles than those of testing the product. And it's usually a good idea to assign maintenance and automation responsibilities of modules to the module leads.
MAINTAIN TEST CASE DESIGN DOCUMENTS
Testing is a continuous process involving frequent software builds and multiple product releases and updates. It's therefore important to create test case documents and keep them up-to-date along with change requests and release notes.
The consistent use of good testing practices will help improve the quality of the tested software, help teams overcome the challenges and effectively deal with software defects. If you're open to learning from others who have come before, you're much less likely to repeat their errors.
