Friday, February 02, 2007

Regression Testing

I am a student on James Bach's Rapid Software Testing Online course. There in that course we have a class forum where we can discuss on testing topics, challenges. It is one of the great source of information for me now. In the forum I have been discussing on Regression Testing topic. The forum topic was started to share the Regression Testing Strategies post by Bj Rollison. This post is mostly based upon those discussions and particularly the contributions made by Erwin van Trier in the post. I must thank Erwin for his views which helped me in thinking deeply and clarifying some concepts on Regression Testing.

Now here in this post I will share some of the insights from the forum discussion. As far as definition of Regression Testing goes Erwin likes to say it is an activity of "retesting (of) earlier tested features/functionality". James Bach defines it as "any testing motivated by the risk that a change to the product could have harmed it in some way". These two definitions are very different from the commonly held view that it is "repetition of previously executed tests". This later definition does not have scope of designing new tests targeting the regressions that may have been introduced in the system by the change/additions. I think it is not a definition at all because it does not say "what" is regression testing; it tells "how" to perform regression testing i.e., by repeating previously executed tests. So it is kind of leaving us with no choice but to execute the previous tests to find any regressions. But what if the tests cases were not complete? We will miss any regressions if those test cases do not touch upon possibly regressed function/path. So we need to design new test cases also as a part of regression test planning. So, regression test strategy includes (I don't think it is the exhaustive strategy) ... adopting some points from Bj Rollison's post:


* Functional impact analysis
* Selecting previously written test cases (we may use our discretion depending on schedules, etc.)
* Selecting previously fixed functional defects
* And, writing additional test cases, too.

Now this whole set of test cases would be called Regression Test Suite. After executing this suite the tests may find some bugs. For some all these bugs are regression bugs as they are found during regression testing phase. For some this differentiation of regression bugs from non-regression bugs is not needed. (The non-regression bugs means the bugs found during regression testing which have nothing to do with the changes/additions made in this build.) I was able to think of one benefit for this classification i.e., we can say whether our regression suite was successful in identifying regression and hence whether the strategy adopted for designing regression test suite was effective enough. But I think I am not convincing enough in stating that as benefit. May be I know what benefit is but not able to put it in words. Or may be there is no benefit at all?

1 comment:

Debasis Pradhan said...

Nice post on Regression Testing Vinayak. You seem to me as a passionate tester. And I consider myself as a passionate tester too. So I hope that we can learn many things from each other. I have posted an article on regression testing too. Hope you would like it as well! URL:http://software-testing-zone.blogspot.com/2007/03/how-important-is-regression-testing.html .

Thanks and Regards,
- Debasis Pradhan