Software Quality: The Missing X-Factor
Shane Hastie, June 2002According to Peter Denning (in an article for the 2000 edition of the Encyclopaedia of Computer Science) "Public dissatisfaction [with the IT industry] is at an all time high". He states that this manifests itself in a number of ways, including the high level of complaints about technical support for home and office systems and the widespread fears about failure in safety-critical systems.
How happy are your users/customers with the software you have delivered recently? Looking at the statistics and figures across our industry the results are horrifying - the 1998 Standish Group CHAOS Survey found that 28% of projects are failures, 46% are challenged and only 26% are considered successful, and this is an improvement - in 1994 the failure figure was 40%! This type of return on investment in other industries would probably result in malpractice claims!
What creates customer satisfaction? Let's call it Software Quality, something that I believe is sadly lacking in many of the systems developed today.
Let me nail my colours to the mast here - I am a practising software developer with over 20 years in the industry and have built a lot of systems over the years, and I honestly believe that we developers work exceptionally hard attempting to deliver systems that meet the customers needs, and are satisfying for us to build. We are not irresponsible or unconcerned with our customers needs, and will often go the extra mile to deliver what is asked for under extreme time pressure; the stereotypical picture of the programmer working a 36 hour stretch fuelled by V and cold pizza really does happen.
In researching this article, I looked for an authoritative definition for software quality - and discovered that there isn't one. Software Quality is such a broad concept with so many components that it defies a simple soundbite definition. Perhaps this is symptomatic of the problem - not enough attention has been paid to defining what is required and specifying it in an unambiguous clear manner.
Both the ISO and IEEE have definitions that refer to various "quality attributes". One of the best (in my opinion) descriptions for Software Quality that I found was given by Robert Glass, an authority in the field, who said that software quality is made up of a number of "ities", including:
- Functionality
- Reliability
- Understandability
- Usability (human engineering)
- Modifiability
- Testability
- Portability
- Efficiency
These "ities" cover many of the quality attributes from the ISO/IEEE definitions.
Other authorities have extended this list to include additional characteristics such as flexibility and modularity. My intention with these articles is to examine ways of addressing some of these individual "ities" and thus provide some tips and techniques for improving the overall quality of software "in the wild".
Let's look at functionality - by this I mean does the software do what is required of it. This could also be termed fitness to task, or meeting the Functional Requirements.
The CHAOS survey of 8000+ projects found that of the eight main reasons given for project failures, five are requirements related. Getting the requirements right is probably the single most important thing we can do to achieve customer satisfaction.
I recall a number of conversations with customers where they have said things like - "that's what I asked for, but now that I see it - it's not what I want". The levels of frustration for both practitioner and customer that this results in can probably be measured on the Richter scale.
It is my belief that the level of detail and rigour in requirements gathering can be considered a scale with two extremes - on the one hand you have systems that have high criticality and exposure (medical monitoring equipment, rail service scheduling etc) and on the other there are ad-hoc generally small systems with minimal risk and exposure (it will probably not cause serious financial or health concerns if the address format is slightly wrong in the mailing list for your local amateur dramatic society). The requirements gathering and documenting process applicable to a specific project will fall somewhere on the scale between these two extremes.
If you are building a mission critical, safety critical system you must have a very tightly defined and documented requirements gathering process, where every element of functionality is explained with a level of rigour that will provide certainty and assurance (possibly even a legal backing) that what gets built is what is actually needed. Organisations like NASA use this level of rigour in defining systems for the space program - these applications cannot afford any margin for interpretation or missed requirements, and they follow a strict set of processes to ensure the requirements are fully gathered and implemented. Even with high levels of process and rigour, they have produced some monumental blunders - there is a difference between inches and centimetres!
However, a less critical system can have the requirements documented in a more flexible manner, and often development could start from somewhat loosely defined requirements. But development must not stop with these loose requirements - if you choose to follow a looser requirements gathering process, the customer or subject matter expert who is providing the requirements will need to make themselves available throughout the requirements gathering, design and development phases of the project. The subject matter expert becomes an integral part of the project team and provides immediate (or nearly so) documented answers to questions that arise from the loose or ambiguous requirements statements. If the requirements are not detailed in full up front, they need to be built up as the project continues. Prototyping is a great tool for this sort of development process - the customer sits with the designer/developer and they nut-out the details of the system together.
Where this process has the potential to fail is when it is not documented clearly enough during the prototyping/build phase. If a requirement is not documented (even as a simple bullet point) then it cannot be tested and verified when the system is delivered, and becomes a potential point of contention. Design and build for testing and verification! (I will address the hairy topic of testing requirements in a future article). Even in "small" systems there is a strong demand for verifiable requirements statements that can be used as a measure of the suitability of the product - have we built what was required of us.
One of the chief tenets of XP and Agile Programming is bringing the customer and programmer closer together, but my intention here is not to examine these methodologies in depth - I'll leave that for Bryan Dollery's excellent columns. I will say that I believe very strongly that the closer the programmer can get to the customer the better - any project will be improved by effective communication between the customers and the developers.
When discussing requirements specifications, Don Mills (our resident quality and testing Guru at Software Education) has stated that a requirement that is not specific cannot be considered a specification, it is an "ification" - open for interpretation and ambiguity. If the requirement statement is not clearly and precisely stated then the designer and/or developer must decide what is wanted - these people are skilled IT professionals, not business subject matter experts and the probability is fairly high that they will not interpret an ambiguous or unclear requirement (even worse, there are often contradictory requirements that must be fitted together) in the same way as the subject matter expert, potentially leading to conflict and eventually to an unhappy customer.
If you have a definitive set of requirements statements that have been built up as the project continues and the customer has been closely involved in deriving those requirements the chances are far higher that you will achieve agreement with that customer that this is the product they want and need - provided of course that you deliver a system that actually meets the requirements!
In the next article in this series I will present some techniques for gathering and keeping requirements up to date, and presenting them in a manner that makes clear, logical sense, helping to ensure that what is defined (at the beginning and as the project progresses) is what the customer really does want and need - thus improving the quality of the functional requirements.
