Bug Report Logo
NVP Software Testing Newsletter
 
August 2008
Greetings!

We hope you are enjoying your summer!
 
With vacations and camping and backyard BBQs underway, it's hard to focus on those Quality Assurance and Software Testing goals that you had in mind going into the summer.  Fall is fast approaching and we can help you regain your motivation to make any changes necessary
 
 
Stay tuned for our new Targeted Quality Mentoring Program -- coming soon.
 
Regards,
 
Your QA Advisors
In this issue...
Tip of the Day
Lessons from a 0-7-5 Season
The Importance of Being Visible
New Website Announcement!
Tip of the Day
From Seminar: A Squirrel's World - Requirements Gathering from a Testing Perspective 
 
A Methodology for Dealing with Requirements as a Tester:
 
1. Get sign off on the document.
2. Write one-line test objectives reflecting the document.  Do this immediately after the document is approved.
3. Add new objectives based on what you remember.  Label them as not in the document but coming from your notes.
4. Add new objectives based on experience. Label them as such.
5. Add new objectives based on other types of testing.  Label them as such.
6. Internal review by a tester in your organization.
7. Issue a copy to the user(s) for review with the understanding that this constitutes the coverage for the particular document.
8. Be prepared to re-open the Requirements or Design document based on your suddenly new test objectives.
9. Revise the test objectives based on the feedback and possible updates to the document.
10. Issue for review.
11. Get sign off.
Lessons from a 0-7-5 Season

 
Written by: Neil Price-Jones, CSTE - NVP Software Testing
 
It's amazing how much you can learn about software testing from watching and being involved in sports teams!  I have two daughters who have been heavily involved in soccer and I have had the opportunity to watch many games and also coach some of them.  Suffice to say from the above title, the last season did not go well. The following applies to teams of both genders but of course having daughters, I have mostly witnessed the Girls teams and their playing strategies.  These are recounts of previous games over the years and how I felt they related to the software testing industry.
 
The first game involved a mixed Boys and Girls team of under 5's, the second was an under 14 Girls game that one of my daughters plays on, and the other was an under 16 Girls match where one of my daughters was the referee. 
 
The analogy to testing software was almost too close to be a coincidence.
 
In the under 5 mixed teams, there is a knot of kids moving around the micro sized field.  Somewhere in the middle of it is the soccer ball and a number of legs kicking at it.  Direction is not a concern nor is 'playing your position'.  Some of the kids were looking toward their parents at the side of the field and not playing attention to anything.  Once in a while someone would get a breakaway (50% of the time in the wrong direction) and progress up the field with everyone yelling "Stop - you're going in the wrong direction" or "Go for it!"  Every time the ball went out of bounds everyone stopped and looked at it - all play came to a halt.  Those that make a mistake and realize it, stop and admit their mistake (even if no one is refereeing or checking).  At this level we do not keep score - everyone gets a trophy or medal at the end of the season and everyone is a winner.
 
As Junior Testers or those without a plan, we sometimes operate in the same way.  We all follow the same tests (especially those that work).  Once someone gets a test to work, everyone else rushes away to try the same test or close variations.  We try a lot of the same things over and over again -- that worked last time -- hopefully it will work this time.  The overall plan or direction is completely lost to sight as long as we are doing something that is useful or appears to be useful and fills our time.  When the test goes 'out of bounds' we stop working -- we cannot think outside the box.  Tests that belong to different realms or are orthogonal to what we are used to are not tried.  When the software fails, we frequently expend a lot of effort trying to figure out what we did wrong.  Lastly, we do not realize that we should be recording what we have done (keeping score) and trying to work to some completion.  We test on and on until someone says stop.  I am not sure if everyone is a winner but we all feel that we did our best at the end of the testing.
 
In the next layer up (Under 10 Girls in the case I witnessed), the players know their positions but do not always want to play them.  There is still a knot of players following the ball (pursued by the referee) but a few of the players realize that they need to be outside the area to be ready to receive the ball.  The field has grown to a medium size.  For the most part, the players know which way they are going and are starting to realize what the game is about.  Some players are still totally lost to the game and some are fully focused.  Scores are kept (blowouts are common at this level) and the team knows if they won or lost.  There is some familiarity with the rules about what to do when the ball goes out of bounds although the referee usually has to explain the calls in detail to the players.
 
As Intermediate Testers, we know our positions and what we are supposed to do but the test that someone else is doing looks very interesting.  If only we could be over there doing that test, it would be a lot more interesting.  The interesting and fun / exciting tests attract the most attention.  The dull and uninteresting tests are left to the end.  We mostly know where we are going and what the end result is supposed to be, although it is easy to lose sight of it in the rush or if the project is very large.  We know to record our results and issues and even though we may have little control over the result, we at least know how close we got to it.  Some things are still mysteries and need explanation but they are far less than before.  We can start to imagine other tests.
 
The Under 14 Girls are a completely different team than the others.  Two items are warring here.  One the one hand, everyone knows their position; they can play it and probably one other position as well (since they are moved around).  However, as teenagers and preteens listening to the coach and referee is completely optional!  Balls that roll out-of-bounds are immediately dealt with and some of the players hardly wait for the referee to indicate what is happening.  Calls by the referee are explained but more for the parents than the players and the explanation is simply enough for understanding.  Most of the players have played for a number of years and they all know each other.  Scores tend to be relatively even.
 
As Senior Testers, we are versatile in a number of situations and can fill in for more than one person if necessary.  We will get the job done and we know better than anyone else how to do it.  Project Managers and Test Managers are just there to run interference with other groups and get us what we need to do our job.  Every project (almost) will be brought to a reasonable conclusion with some assurance that the software is good and has been tested.  We know what to do with most situations and do not need more than a brief explanation to move on.  If the situation is new, we have the backing of experience and tools to know how to tackle it.
 
The Under 16 Girls are different yet again.  We have a combination of in-house teams playing people from neighboring towns.  The in-house teams all know each other very well -- somewhere along the line they have played with almost everyone.  They are out there to have fun and comments on the referee (usually a young guy) and their teammates and the opposing team are continuous.  Playing other towns seems to be fun from our end although some of the other towns seem to take it too seriously.  The coach does not tell any of the players what to do; they seem to mainly concentrate on switching players and keeping track of items.  The players themselves coach each other (sometimes seriously and sometimes not) from on the field and on the bench.  Periodically the parents wish that the player(s) on the bench would be sent out to play so that we would not have to hear them so much!
 
So, to what level do these players belong in the testing pantheon?  Are they professional testers, coaching other people and helping themselves?  They might be Project Managers or Test Managers.  They coach themselves; know where their strengths lie and what has to be done to reach project completion.  This is work and we have to have a good time at it.  It is not there to fill the time!
 
 
This article has been derived from an entertaining and informative presentation that Neil will be presenting during the QAI QUEST Conference Expo September 24/25, 2008 in Toronto, Ontario at the Hilton.  For more information regarding the conference please visit:
http://www.qaiquest.org/toronto/
The Importance Of Being Visible
Written by David Kapfhammer 
 
Software Test and Performance Magazine
Volume 5, Issue 6, June 2008
 
Grapefruit
One of the worst things a testing organization can do is to operate in general obscurity from the rest of IT.  If a testing organization doesn't treat the development organization, business analysts and users like "customers," it will lose credibility as the "team that operates behind the curtain," and ultimately become ineffective.  Testing should be conducted visibly.
 
The importance of testing visibly can't be understated.  Providing line of sight into the operations of a testing organization is crucial for the effective management of not only the testing team, but the entire IT project.
 
Quite often, testing teams do just the opposite, intentionally conducting their test management and execution in obscurity.  The project team doesn't know, nor do they understand what's going on in the testing phases of the SDLC.  Testing teams feel compelled, for some reason, to keep their activities secret, only to surface when there are bugs to be reported.
 
Why is this?  Perhaps they feel that to provide and unbiased perspective, they must be completely disconnected from the rest of the project team.  This condition is an unfortunate reality that often ends in frustration.
 
Creating visibility into the testing team and all of its operations can provide comfort to the project team, develop expectations and cultivate predictability.  Allowing the project team insight into test operations doesn't discount the credibility of the tests, nor does it disintegrate the importance of being unbiased.
 
How to Add Visibility
The testing team should create a series of "defining documents" that outline specific ways to create visibility for the team.  Guides for defect management, test project management and user engagement (user guide) are the primary mechanisms for communication.
 
The user guide outlines for all consumers of testing services exactly how to engage the testing team, defines expectations and gives an understanding of the type of testing services available to the project team.
 
The user guide should outline a set of procedures for the project team to employ in its interaction with the testing team, and provide detail on several categories of activity, which include:
  • The systems that testing will support
  • The organization of the testing team
  • How users of testing activities request and receive services
  • Principles that drive governance, decision making and validation
  • An orchestration document that connects all of the testing team's defining documents for the user
  • Service-level agreements (SLA)

Project teams often don't know what's happening during the testing phases of a project.  As a result, when budgets need to be trimmed, the testing program is often on the chopping block.  However, clearly codified procedures and deliverables help a testing organization illustrate their worth to the overall effort and provide a set of expectations for the project team.

 The testing organization can establish the user guide as the vehicle for assigning expectations to the rest of the project team.  For example, turnaround time for bug fixes can be defined there, and subsequent impact to the project can be conveyed if those expectations aren't met.  These are often referred to as service-level agreements (SLA).  Furthermore, the user guide can make a reference to the defect management guide at this stage, inviting all readers to familiarize themselves with the test team's processes.
 
Business demands for shorter project schedules, combined with a greater degree of impact to an organization when faulty code is deployed, have generated more demand for repeatable, predictable testing.
 
This surge in the marketplace has manifested itself in a variety of ways.  One of the more visible means has been a push by organizations to have a clearly defined blueprint for their testing and quality assurance needs.  Testing and QA are starting to make an appearance at the executive levels via enterprise-wide testing strategies, enabled through a clearly defined future-state road map.  Road map and strategy creation requires foundational principles; testing visibility is one of them.
 
In addition to the positioning of testing and QA at the executive levels of organizations, the testing and QA industry has also been responding with its own mechanisms for the future.  The Quality Assurance Institute (QAI), the world's leading institute for providing effective solutions for testing in the information services profession, is finalizing its development of a quantitative rating system.  These ratings will allow organizations to quantify their ability to perform testing and quality assurance.  This type of measurement of organizational capability is an indicator of the future expectations for testing and quality assurance.
 
At the onset of a project, the testing team should issue a user manual that details how to engage testing services, including a specific operational workflow breakdown, to maintain the testing organization's open-door policy.  All members of an IT project team should know exactly what the test team is doing and where they stand with respect to schedule, cost and scope.  By testing visibly and providing clear insight into testing operations, project teams can more effectively communicate.  More often than not, this results in more efficient operations.
New Website Coming Soon!Iced Tea
NVP Software Testing is proud to announce that it will be launching a new website September 1st, 2008.  We are excited to provide with you with a more stimulating and welcoming design paired with a more user-friendly experience.
 
If you are looking to have your website redesigned please let us know and we can both benefit from our new web partner's referral system.
 
We hope you enjoy the new look.  This is YOUR site so any feedback is greatly appreciated and welcomed. 
 

The Bug's Last Words

Instead of offering a joke this month, the NVP Bug wanted to tell you this important news:
 
NVP Software Testing is undergoing many exciting changes to serve you better.  We will be launching a new Targeted Quality Mentoring Program this fall.  The goal of this program is to provide you with support and mentoring that can help you increase your Software Testing and Quality Assurance skills and tackle challenging problems in your department. 
 
With over 15 years of experience in many different industries we felt that we could help our fellow testing folks by providing guidance in a partner-like relationship rather than in a business-to-business service sense.  We want to help you improve and we want to learn from your unique challenges and situations.
 
Stay tuned for the TQMP and look for this logo - appearing soon!! 
 
TQMP