News

Exploratory testing is a style of testing that emphasises quality by treating testing activities as a chance to continually learn and optimise work throughout a project (Kaner, 2006). Testers are allowed the freedom to be creative; executing more test cases and covering different testing scenarios to find bugs while keeping the perspective of the end user in mind.

The problem is that exploratory testing can be limited by our own perspectives; when we explore, we tend to perform instinctive, well-worn, intuitive tests, ones that are ingrained in our minds. Exploratory testing relies on mental models, graphic models, mnemonics and checklists which are practised and used all the time. These become patterns that help us think and to make decisions.

The trick is to become aware of how you model a product and learn different ways of modelling. The model allows you and others to see the problem at hand – this visualisation can also help us extract information about the system that we can’t see otherwise.

One common approach used in exploratory testing is the Scientific Method. Typically, the steps are Observation (Observe) – Hypothesis (Decide) – Test (Act) and then iterate. The aim of this approach is to enhance critical thinking; gaining new information, creating an informed assumption and considering, re-evaluating the results and doing this to continually improve.

However, there is a missing link in the Scientific Method that can be found in the OODA loop which includes an orientation phase.

Beat your biases with the OODA Loop

The OODA loop concept is a cycle often used as an approach to understanding commercial operations and learning processes, often in the business agility space. It lends itself well to the disciplines of Lean, Lean Start-Up and Design Thinking.

The OODA loop builds on the Scientific Method by adding the orientation phase: Observe – Orient - Decide – Act (Boyd, 1976).

The orientation phase is the main focal point in the OODA loop. It involves analysing your own culture, traditions, heritage and past experiences which shape your world view and the way you process new information; putting all these elements together into a new orientation (Boyd, 1976).

This means you need to challenge all that you are – your history, your upbringing, your education, your environment, your culture – challenge your confirmation bias and the many cognitive biases that simplify your worldview so that you can look at systems from new perspectives.

There are a lot of different techniques that can help you orient yourself away from your perspective and tackle software, design, system and process problems from completely different angles:

• Personas: personas can help you understand the needs, behaviours and limitations of users so you can perform test cases as if you are a customer trying to achieve a specific goal. For example, Mary, a shopper addicted to loyalty offers may have several traits including having too many things to juggle and being rushed and flustered. Mary wants a particular outcome; to scan shopping items - some with special discounts, some without. Keeping the persona in mind while testing makes sure your solutions are fit for purpose.
• Workflow diagrams: you can map out Mary’s journey to achieving her goal in a workflow diagram. Visualising this process can help you to simplify the steps and identify any potential issues.
• Exploratory Testing Tours: there are several different test tours that you can use to ensure wide test coverage. In each scenario, you assume the role of a tourist that arrives somewhere they have never been. Each tour is a plan to test from this point of view. For example, a rained-out tour assumes the worst. For Mary, this could mean operations stopping, time-consuming steps etc.

So, the story goes something like this – Mary gets a range of items. She gets flustered by the “three items for one dollar” special as she only has two items. When she gets the third item it turns out to be the wrong brand and therefore not part of the special offer. Ultimately, she will be wanting to scan the items and leave the shop satisfied but we are trying her patience and testing the system.

Now we are learning how robust the system is, we are taking a customer’s viewpoint and trying to empathise with them, we may stumble across a limitation of the system, we may see flaws in the design or workflow. There is a lot we can learn by putting ourselves in other peoples’ shoes. Thinking about other perspectives helps us to reveal capabilities, limitations and risks that might otherwise go unnoticed.

Exploring different mental models and trying to get a good understanding of your users when building systems, software, processes and products can ensure that you finding more defects creating better quality products.

Post by Toby Thompson.