to the NEW Bug Report!  We've spruced up our image a bit with a new website, a new referral program, our new and popular Targeted Quality Mentoring Program, and of course a revamped newsletter.  We hope you enjoy the new "us"!  We always look forward to feedback from you so please feel free to reply to this email and share your thoughts.

 

Written by: Neil Price-Jones, CSTE - NVP Software Testing

With the current economic climate, people are concerned about the potential cost of a project more than they have been in the past.  Testing is one of the major components of any software project and can consume anywhere from 50 to 80% of the budget.  This causes issues for some of the following reasons:
The budget is often estimated at the beginning of the project when the scope may not be fully realized.

Development often 'borrows' from the project budget when the scope changes in order to satisfy the new user requirements.  Unfortunately the total budget is not usually expanded as a result of this 'borrowing' occurring.
Despite testing's attempts at early involvement, up-front estimation, document review, and testcase creation, active testing on developed code cannot occur until code has been delivered.  This pushes a large proportion of the testing to the end of the project when the project is under both time and money pressure.
Scope changes then put further pressure on the testing time and resources at a point when they are already under stress.
 
We try to address this issue through a phased approach that allows each phase to be estimated based on the previous one.  We put a large emphasis on the Master Test Plan which is Phase 1 (see below).  In the Master Test Plan we try to estimate the time for testcase and test data preparation.  This allows a second review of the project and an attempt to quantify the testing cost based on the number of testcases that will be created and their priority.  At the end of the Preparation Phase, we have a good estimate of what it will take to execute each testcase once. If the number of testcases is out of line with the risk of the project, then this can be identified, quantified and addressed at this stage before we get into active execution.  However, the execution phase of testing is the one that frequently gets out of control.  Seemingly endless iterations of the code and repeated executions of the same testcases cause an escalation in costs with no appreciable benefit.  Clients need to be informed very quickly of this situation and allowed to make a decision that will best address their needs and risk before the project runs out of money and time.  The final completion report form can be built ahead of the testing effort and then just filled in at the end.  This is not a large effort and does not impact the final cost much.
 
Do you have any comments to make on this issue?  If so, become part of our Linked In Software Testing and QA group by CLICKING HERE and join the discussion today!

For further information about NVP's 4 Phase Methodology please CLICK HERE and be sure to include your inquiry under "comments".

It has been well documented that development and testing are two separate domains, each with certain unique characteristics. In a traditional software life cycle, code is being developed first and turned over to test to ensure that a quality product is being delivered.

With so many contrasts between development and test, it is easy to overlook the similarities that exist between these two fields.
 
In many of the companies I have worked in, testing is considered a luxury. If it happens, it happens. Otherwise, no big deal. There's a direct correspondence between these two fields and I argue that testing should be given equal importance to development. Of particular focus is automated testing, in which testers write scripts that can be as lengthy as the code under test itself.
 
I have been doing automated testing for years, and have been to a good many companies during that time. It's interesting to learn people's views of automation, which range from manual testing with tools to simple record and playback. I will explain some of key characteristics of automation and their relationship to development. Hopefully, it will open the eyes of people who have been pretending to know everything about automation but in reality know close to nothing.
 
In a nutshell, the development process consists of these distinct steps:
1. Software Requirement Analysis
2. Code Generation
3. Unit and Integration Testing
4. Maintenance
 
Depending on the company and project requirements, there may be other steps added on or in between, but automated testing has a one-to-one relationship with each and every one of these steps.
 
1) Requirement Analysis
Requirement analysis is always the first step in a software development process. The process is simple. Get the requirements from the requirements team and use them to design and develop your code. Just as correct code can't be written without clear requirements, a correct quality approach cannot be taken without correct requirements.
 
As much as requirement analysis applies to almost every domain, it is even more significant when it comes to automation. I have been on more than one occasion been in the evening and told to start writing automation scripts next day. Sometimes, it has even been for projects for which I had no prior information; I was just told that "everything needs to be automated" within few weeks.
 
As a tester, it's your responsibility to fully understand the expectations of automation for a particular project, otherwise it's nothing more than a wild goose chase. Unless a tester has clear view of functionalities to be automated, creating the automation scripts does not serve their intended purpose. It's important to understand what needs to be automated and use that information to structure out the automated tests.
 
2) Code Generation
Code generation refers to creating reusable automation scripts. Based on what business process needs to be automated, scripts are created that can be replayed again and again. As is the case with development, automated scripts need to follow coding guidelines and adhere to standard practices.
 
It is imperative for people to understand that automation is just not record and playback. You write code. The difference between writing code for automated testing and coding for development is that in testing you write not to develop something but to break what is developed. All the considerations involved in development apply to writing automated code as well.
 
3) Testing
The actual testing phase consists of two parts: Unit Testing and Integration Testing. Let's look at these two separately.
 
3a) Unit Testing
Unit testing involves testing a single unit of functionality, the smallest possible component of code. The purpose of this testing is to make sure the core components work fine in isolation. So, you pick a small independent piece of code and make sure it works as per specifications. For example: in development it may refer to a class, method, function, or a Web service call. You develop, test drive and run the associated unit with different parameters, both positive and negative; to make sure it works as expected.
 
While creating test automation scripts, you should structure your code in manageable units. I divide my code into small units called "Actions," which are synonymous with classes in object oriented programming. These classes can be further subdivided into smaller units, such as functions and methods, just like development methods and functions. After the test script is ready, you test it with different parameters and sets of data to ensure it works as expected.
 
Let's take an example to illustrate unit testing correspondence. Let's say you have an application with a login page.
 
Development
A developer will write code to create the required components in the login page and ensure valid inputs are accepted and correct results returned. He might put this code in a function named "Login," just as it might be called in many places. Then he will plan to test the "Login" unit with various sets of data.
 
Testing
A tester will write code with different verification checks to make sure that the components in login page accepts valid inputs and rejects invalid ones. The tester might put this in a function so that he can make it reusable across various other automation scripts he might write, and ease the effort in maintaining the code. Then, he might run this script with various sets of positive and negative data.
 
3b) Integration Testing
This testing involves testing various units together. As a part of unit testing, various small standalone components are tested. As a part of integration testing, these units are combined to make sure the components continue to work when put together. It is not rare to see a piece of code working fine in isolation but failing when used in conjunction with other units.
 
Let's take an example to illustrate unit testing correspondence. Let's say you have an application with a login page. When clicked on "Search" button in login page, it takes you to home page of the application.
 
Development
You should have already performed unit testing for login page. Now, you want to make sure things are as expected when you combine the "Login" function with "Search" function. You might call "Search" function from Login function or vice versa and make sure the home page shows up as per the requirement.
 
Testing
You should have tested the Login page with different sets of data. Now you can include clicking on the "Search" button as a part of your test script. You can also add checks that ensure that clicking on the "Search" button brings up the home page (or search page, as appropriate). You run this script with multiple sets of data.
 
4) Final Check And Maintenance
There is always some maintenance involved in anything you do. It's even more significant when your code is used repeatedly and by many people.
 
Development
Maintenance required in the development code is well known. Often in projects, much of the total software life cycle involves some sort of maintenance. Activities include making sure any changes in code are documented, the code is being updated regularly as the application changes, and unintentional modifications by people who are not authorized to do so are prevented. Various strategies are formulated to make sure code is maintained as best as possible.
 
Testing
It may seem that testing doesn't require much maintenance. But actually, the opposite is true. Keeping automated scripts usable requires involves a good deal of maintenance. If you fail to keep your scripts up to date with changes in the application, your scripts might just become obsolete. The longer you wait, the harder they are to bring up to date. It is also important to make sure that scripts are modified only by authorized people. If someone is not aware of what the script does, he might unintentionally modify something which might break the script. Various procedures are put in place to ensure that maintainability of scripts is a major consideration in the project.
 
The best and most successful projects I have worked on have involved testers from the beginning and throughout every phase of development. Even better are developers who do their coding with testing in mind. If everyone on the project is kept aware of the needs of everyone else, both sides of the coin can work together to bring about the right kind of change.