They will also have noticed the blatant and shameful lack of authority with which one contributor (me) replies to questions and issues around testing.
What you may not be aware of is that I frequently also share my confident but dangerously simplistic views on testing when running courses, talking in the pub and anywhere else. So hopefully those better educated in the mysterious ways of the tester will continue to correct my wayward advice before anyone follows it.
So, with that opportunity, here is an email request I recently received:
Dear TWG (Testing-wannabe-guru):
On our last day of our BA course, you mentioned several things a tester should test for in system testing. Would you be able to send me the list?
Yours – BA who believes in quality
So here is my response- subject to correction from the pros:
Dear BA
In system testing we should test for things that might be wrong with the system. You would think this means the process, the IT applications, the training material and other things that a business person would think are the “system” we are delivering. But this would be hard and might result in a lot of work when we realise there are issues.
So we normally only test the IT applications and even then we only test three things:
- Test things the developer should have already thought of when writing the code.
- Since the developer should have thought of these things, we should not need to test for them at all. So you can assure yourself of the system’s integrity by asking the developer “does the system do what we asked you to make it do?” They will generally say “yes” and look a bit shifty.
- We then ask the developer “how do you know” and they will respond “I don’t know – its the testers job to tell you if the system does what you asked me to make it do”
- Test the developers patience. You should now ask the developer if they are a proud member of the professional elite of great developers. They will tend to say “yes”. Then you say – “Excellent, then clearly you would have not only made the system do what we asked you to make it do .. but even suggested innovative new things that nobody else could have made it do … Surely that is the difference between the adequate developer and the great one. So, how can I prove to others that you deserve their respect and even their awe?”
- They will generally look perplexed or annoyed – so ask them “OK, assuming we had great developers then we should only need adequate testers – what would an adequate tester test for to make sure the system works”
- Now ignore what the developer says, because this is the stuff they have thought of and that will generally already be working. This is therefore not what we spend time testing in the system testing phase.
- Testing the developer’s patience should always be included in your testing. Not only is it fun, but you will soon learn who the good developers are (they actually do know the system does the obvious things they were asked to make it do) and this is a good measure of the quality of the overall system.
- Test for bizarre things the developer would not think of
- Begin with scenario tests – create some examples of how people might use the system. Don’t just look at one requirement but think up and example where the user would want the system to do several things. For example, to test a banking system, don’t test just whether you can log in. Do an example where someone logs in, looks at an account balance, performs a transaction and then goes back and looks at the account balance.
- Think of some more examples (scenarios) and spend about 70% of however much time you have available running these though these examples to see if they work.
- Now do some “bizarro testing”. This is often called “exploratory testing“. Think of random things that someone might do if they were dumb/they were distracted/they were intelligent but did not understand the system/they were older than normal/they were younger than normal/ they were in any way different to how a developer would imagine them to be. Spend 30% of your time just doing some of these random seeming things and see what errors and unpleasant outcomes you can uncover.
Its pretty easy really. The only problem is that you will only have a few days available and you will need about 6 months to test the system properly. So, oddly enough what we pay testers for is not “finding bugs” but
“Spending very limited time to
- Help the good developers discover the issues and problems they would not have thought of and
- Let us know if it is likely that the system will suck in production and embarrass the team, or work as designed and make us look good.
I am not sure how they do this with so little time available but apparently this is their job.
Does anyone know how we can do better in system testing that the above approach? It seems quite straight forward and yet appears to be a real challenge on projects.