I hope the software works

Do you tell this to yourself every time your team releases a version of your software to your customers?

Are your customers spending more time filing bug complains than actually using your software?

In case you haven’t heard, there is what we call quality-assurance (QA for short) activities that make sure you release a software that actually works.

There are 3 things that could happen when your customer receives a defective software product.

1. Customer never buys an upgrade or never renews the license for the product or any of your other offerings. Even though the other product is of high quality, customer’s experience with the defective product will affect his buying decision.

2. Customer takes business somewhere else. Many companies believe ‘it is better to have sold and lost than never to have sold at all’. While there is financial benefit to this, it is a short-term gain compared to having lifetime customers.

3. Customer tells others about the low-quality product. Spreading word about their satisfaction or dissatisfaction of your product is just an email or blog post away.

In light of these long-term consequences, I still don’t understand why you don’t have a dedicated QA team.

Defects can creep into every stage of software development. A defect in the requirements can produce one or more defects in design, which can produce many faults in code, which can delay testing, and eventually forcing the development team to repeat another design-code-test cycle again for the nth time.

Software will always have bugs. It is a reality that every software-producing company, whether as their primary product or a complimentary one, should accept. This is why a QA team is essential to ensure high-quality software products are delivered to customers.

The QA team is not just a group of testers or low-class programmers waiting for their chance to write production code. In fact, the QA team should be independent, knowledgeable, and powerful – if possible, powerful enough to stop the release of a product if it does not meet the quality objectives or if it has not gone through QA activities.

The QA team’s role is not limited to testing. The software development team should also understand the value of QA activities and it is the QA team’s job to make sure that developers do. This will include involving developers with QA activities and evangelizing the value of quality in software development.

So why is it that some companies don’t have a QA team?

QA activities will lengthen schedule

Many project managers shorten their schedule by reducing time spent on QA activities like code reviews. Others, just deliberately take these activities out. There are also managers who thinks QA is simply testing, which is also vulnerable to reduction since it is at the end of the schedule.

QA definitely affects schedule but in a positive way. In fact, projects that achieve the lowest defect rates also achieve the shortest schedules. What Capers Jones found out after surveying 4000 projects was that poor quality was one of the most common reasons for schedule overruns. Poor quality is also the reason for almost 50% of cancelled projects.

Defects that are discovered early are much cheaper to fix than those found later on. However, developers cut corners because of tremendous pressure to deliver on schedule (where deadline is based on ballpark estimates and overly optimism). Eventually, a product gets harder to modify, and bugs more difficult to fix resulting in schedule overruns.

Reworking defective requirements, design, and code typically consumes 40 to 50 percent of the total cost of software development. Employing QA activities such as inspections can reduce this rework. As industry data suggests, short-cutting a day of QA activity early in the project is likely to add 3 to 10 days of unnecessary activity downstream. Below are some statistics supporting the value of QA activities:

1. Inspections can improve quality 10 times, and increase productivity by 14 percent.

2. Each 1 hour of inspection saved 20 hours of testing and 82 hours of rework effort had the defects found by inspection remained in the released product.

3. The maintenance cost of inspected programs is 1/10 the cost per line of code of uninspected programs.
Design and code inspections typically find 50 to 70 percent of the defects.

Great developers are enough to produce high-quality software

Bugs leak out because programmers didn’t see them. Oftentimes, a second-eye is the only thing needed to discover the bug. It is the job of the QA team to act as second eyes of the developers. Remember Linus’ Law &mdash “given enough eyeballs, all bugs are shallow.”

QA is costly

A QA team is additional personnel cost (some companies have more testers than developers) but still cheaper than developers on a per-person basis. If you don’t have a QA team and assuming you really want to do QA work, you will force programmers to move away from what they do best, that i.e. coding, and do what they 2nd least like, i.e. testing (the least like is documentation). Depending on how much you value a developer’s time, there is a loss incurred if developers don’t spend their time on coding. Also, developers may quit their job because they don’t like doing QA in which case the loss is even greater.

Now that you have convinced yourself and your boss to form a QA team, prepare for the challenges ahead of you.

Good QA staff are not easy to find

The best programmers are an order of magnitude better than average ones. The same is true for QA people. A QA person has the skill of a programmer so she could write test codes. Unfortunately, like very good programmers, a very good QA is hard to find. Also, people who knows how to write code would definitely stay in writing code.

There are a lot of repetitions in QA especially during testing phase. Smart people tend to get bored with routine tasks, thus a good QA staff staying for only a few months is not surprising.

QA has no career path

There is a definite career path for programmers, support staffs, but not yet on QA. Aside from the usual junior-to-senior promotions, a QA career should be nurtured in the company. One way to do that is to promote support people to become full-time QA staff. QA is definitely much better than answering emails from irate customers everyday. Also, support staff understands the customer experience with buggy software and may take this chance to make a contribution to the software.

The company should support the QA staff in improving their programming skills (for writing test scripts) and communication skills (for reporting bugs and interacting with developers)

QA staff are second-class citizens

QA teams do not report to the development team. Even if they do, programmers may see them as second-class citizens trying to intrude in the work of smart people. Suggestions or even requirements by the QA team may be discarded not just by the programmers but even by the whole development team, management included.

QA as an optional activity

Since the company’s inception, QA is most likely seen as an optional activity. The success of QA activities depends a lot on how both management and developers see QA work. If QA activities will always be sidelined, there is no way we can see and measure the benefits of integrating QA in software development.

Putting too much faith on QA

While there are documented and concrete benefits in doing QA activities, it is just one variable in delivering high quality software. Developers should also employ good coding practices and inject high-quality codes in the software products. Software failures can be identified by the QA team but it is still up to developers to fix it and make sure this won’t happen again.

So next time your manager asks why your customers are spending more time filing bug complains than using your software, ask for a budget so you could go to Medical City and have a surgeon implant a new pair of eyes on your forehead.


5 thoughts on “I hope the software works

  1. I always tend to say that.. simply because I never had a good QA team. I must say, I loved the QA team that we had (in a project I belonged to before).. and yes, they were indeed powerful enough to stop a release.

    I hope that more companies wouldn’t ignore the essence of a good QA team. This will aid not only in egoless programming but also for keeping maintenance programming to a low.


  2. I am one of those customers that spends more time filing defect reports than using the software — and I complain about tools made for testers. 😉

    Good testers (they are not always the same) are there to help protect revenue and reduce liability. The risk of releasing software without adequate testing is just too great to bypass testing by an external (as in separate from development) group of testers.


  3. I have never met a programmer who prides himself in producing bad software. Building software that will help others is simply our nature. We have the open-source success to prove that. The quality of software is a management issue, not a technical one. QA activities being discarded, testing time being reduced, or programmers doing what they are not good at are all management decisions.

    Steve McConnell once said that the success of software depends on strategic decisions, not on tactical ones. QA, no doubt, is one strategic decision that management should make.

  4. @Greg

    I think you nailed it. Bad software is rarely the fault of individual programmers. Bad software is usually due to management issues and/or requirements that do not match the users’ (and the developing company’s) needs.


  5. I am a software tester and looking for a software tester certification, and ofcors the training that goes with it, that i can take here in the Philippines which will be recognized internationally….

    If anybody knows anything about any certification that i can get please email me at ynahhany@gmail.com. I would really appreciate it..Thank you

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s