Test cases are here for a long time. Used equally willingly by these who work with heavy-weight processes and those who choose the agile way. They can be stored in an appendix to documentation in BDUF (Big Design Up Front) project or be written on yellow cards along with user stories. Where’s their main value then?
Think about the comparison: test cases for application functionality are the same as unit tests for application code. Except of one thing: it is way harder to write good and complete list of test cases. You may safely assume that’s impossible. This means the main value of test cases isn’t in covering full application functionality because you simply won’t achieve that, no matter how hard you try.
Well, maybe it is a valuable tool for QA engineers? You know, they have some kind of walkthrough for testing sessions. Hm… show me quality engineers who prefer just to follow test cases instead of explore the application by themselves and I’ll tell who should be fired. Testing is such a creative task that there should be no place for follow-the-manual people in QA teams.
OK, test cases don’t ensure completeness of tests and don’t help QA people much. So? Show me the money, as one of my customers used to say. Where’s value?
The main benefit of test cases lies in the activity of writing them down. Or, to be more precise, in the process of thinking about them before they’re written down.
Test case is not much more than short scenario of application usage. Sometimes pretty weird scenario but still. Unfortunately we often forget about this perspective. We tend think in terms of code or requirements and forget about the very basic thing – no class in the code and no requirement alone makes any sense for the end-user.
Looking at application while being in end-user’s shoes benefits in two ways:
– You find black holes of your design which aren’t covered with user stories, or requirements
– You find usability issues since you see whole usage flow, not just a bunch of atomic functions
The main value of writing unit tests is actually within the writing part.
Subscribe RSS feed
Follow on Twitter
Subscribe by email
11 comments… add one
Pawel,
Good post, I agree. QA Test plans derive most of their value from the initial writing, the thought processes that go into that writing, and the first iterations through the code while writing the test cases.
Blindly running through older test cases is one of the main causes I have seen for missing obvious product bugs as people who blindly follow the test plans often put aside the creative aspects of testing.
John
Writing Test cases has some benefits.
1. Delivery to offshore testers, particularly if development is onsite
2. Test Design, it helps you design how you will test the product, much like a developer who is designing how he or she will program the software
3. Repeatible process, particularly useful if bringing in a new tester
Test cases are a prerequisite for regression testing.
Let’s take an example.
You test the signup page in your web application and notice that even if you don’t enable the checkbox “I agree with the terms and conditions” you still get your account created.
You mark it as a bug and write down how to reproduce it – that’s a test case. On the next build, you go through the test cases you wrote and check if they pass.
I see no other way of being able to do this regression testing, unless you write down these test cases.
Steve,
For me the second poing you bring is especially important. That’s exactly looking at the application via user eyes. And you look at the whole flow, not just one particular function.
Paul,
Yes, but… Actually one of problems with test cases is that you can hardly keep them up-to-date. If you’d like to generat test case from each bug you find you’ll be soon overwhelmed with a number of test cases.
To Paul and Pawel,
Those tests case should be automated.
Then, it doesn't matter how many of them they are.
So, you will still be confident that the new build doesn't break a previously fixed bug. This also gives increase the developer's confidence, since he knows that if he make a mistake, there are good chances it will be catch by those regression tests.
Hi All,
can any one tell me “what are the mandatory documents for the requirements phase”
I believe there is no such thing as mandatory documents for requirements phase in general. It depends heavily on environment you work in. If you think about agile projects you can do good with just a set of epic stories which describe on high level what the application should do.
On the other hand if you work with one of highly formalized environments you may have a very detailed list of artifacts you have to produce to comply with methodology which is in use.
So unfortunately there isn’t single answer to such a general question.
Hi all,
can any one tell me “What is importance of RTM (Requirement Traceability Matrix)”
Hi everyone,
Really sorry to jump in on this but I do not agree with you. Nobody doubts the fact that we should not put aside the creativity of our exploratory tests, we should be doing this anyway regardless of our formal test sessions.
Test cases benefits from my perspective
– Helps to identify scenarios that you cannot identify while doing just an exploratory test. This also offers QA some time to analyze the impact of the requirements and identify risks early (not during the testing phase, which is a bit late)
– Cover the business scenarios and the main workflows, this gives QA confidence that they’re really validating the right thing.
– Generate proper reports based on actual metrics and share them with the team (transparency)
– Re-use any of the test cases during a full regression session or just by using some of them in specific areas that might be affected.
– If we have proper test cases we can also consider them as part of our documentation (in scrum this works really well).
If we do not document anything and just do exploratory sessions we leave nothing behind..the knowledge is lost if we leave our team.
These are just a few benefits, I don’t want to enlist more..
Sadly I just want to highlight that people who complain about it are actually lazy. You hear them..
– There’s no time to create test cases, it’s too time consuming
– There’s no time to execute them formally, it’s too time consuming
– Our customers do not want this thing because it’s time consuming
I think Alex sums up the opposite side of the argument nicely. As he says, it’s one tool in a QA professional’s bag. In addition to all of the benefits he lists, it’s a fantastic training tool for new members. Being able to give the new member a set of test cases to teach them the software has a ton of value. Teams often fail to keep them up to date, although that is a failure on the part of that team and not truly seeing the high value.
It’s also very valuable for quality engineers. You imply the argument is a binary choice, either use test cases or explore on your own, but why not both? You’ve now widened the scope of your test by including what another person outlined as important and also what you deemed important. Or potentially their test case is a springboard for your thoughts.
Lastly, it’s very valuable in getting buy in from the business/client. When QA is able to transparently present what they are always testing, there is shared responsibility if an issue is missed. This fosters a team atmosphere instead of everyone doing their own thing.