Prevention is the highest form of testing: The value of zero bugs

The software testing and software engineering disciplines frequently use the number of bugs and their severity as the main way to evaluate not only an application's state but also the performance of both software and test professionals. But, this is not the only way. We don't need to rely solely on bug reports. There are multiple opportunities in the software development lifecycle where testers bring huge value without raising a single bug. In this article, we'll cover those activities. 

Here are some examples: 

• Requirements gathering, 
• Backlog refinement, 
• 3 amigos sessions, 
• Code reviews, 
• Early feature testing. 

During all these phases, testers are stopping issues from even getting into the code, stopping them through thoughtful questions, comparisons and early feedback. 

The problem with bug‑count metrics 

For most people involved at any stage in the software development lifecycle, testing is almost thought of as a synonym for bugs. Therefore, it is easy and straightforward to make the association that having good testers means that more bugs are found. This idea, on its own and at a very high level, might make some sense. Managers are happy to see that bugs are being found before features are released. Testers feel valued and important by stopping that error from reaching customers. Developers (not all of them) feel protected that someone is helping them deliver code that meets the required standards.

Then we can discuss the fact that a bug can only be seen by the entire team if it is raised as a ticket. Exists in the current workload board, or gets discussed during different meetings, such as backlog grooming, refinement, and stand-up meetings. When this happens, we can clearly tell that showing off and counting issues have become more important than the simple fact that our main goal is to help deliver a piece of functionality that meets customers' needs. We don’t realize that attachment to our bug tickets is becoming more valuable than providing prompt feedback, fixing issues as early as possible, and ideally stopping them from happening in the first place.

One of the reasons that this happens is that the work of bug prevention is often simple or invisible, making it much harder to account for. A tester could raise an edge case that was missed during refinement of a new user story or an acceptance criteria that might go unnoticed by most of the team. In reality, that is the same thing as raising and fixing a bug, but through a simple conversation, entirely based on knowledge sharing. 

The (in)visible work testers do 

Think of it like this. A firefighter is seen as a hero when they put out fires, as everyone sees the flames and the effort that goes into putting the fire out. But no one really notices or pays that much attention to them when they are inspecting buildings to help prevent fires from starting in the first place.

As an engaged tester, there are multiple stages in the software development lifecycle where you can continuously “test” the functionality being built. Here, the word “Test” is used in a very broad way, and we could define it as any activity that we perform in order to clarify, question, validate or even monitor the behaviour of a particular element currently within our test scope.

Dan Ashby’s blog post enphasizes how testing is part of every single stage of the development process. From Planning to Releasing, there should be no limitation on where we can test our software, from its initial ideas and designs, evolving to small bits of implementation and infrastructure, to full end-to-end features. 

A similar line of thought is described by Lisa Crispin when talking about shifting left and right, where shift left covers all the activities that we can carry out earlier in the SDLC and is almost entirely focused on preventing errors from being built. This can be achieved through helping build a common understanding of what needs to be built, removing ambiguities and working towards a clear design. Shift Right, on the other hand, focuses on everything that happens after code is deployed, and through monitoring and learning from the application’s usage in the real world, we can identify improvements and actions we need to take in order to ensure the next set of features do not repeat the mistakes that have been found.

When we think of it in terms of how it fits within our day-to-day activities, one way of approaching it is through the different ceremonies that usually happen when a project is conducted using The Scrum Framework(Scrum.org). Let’s talk through some of its events and how potential bugs can be prevented.

Backlog refinement

The Scrum organisation defines this activity as “ the act of breaking down and further defining Product Backlog items into smaller, more precise items. This is an ongoing activity to add details, such as a description, order, and size.” At this phase, the team has the opportunity to raise questions, concerns, challenges and risks related to a given user story. Under this context, testers are usually the ones with a wider view of how a specific functionality fits within the bigger picture, understanding the domain, linking it to our perception of the actual user needs and therefore in a position to help the team describe the user stories in a clear manner, eliminating ambiguity and identifying gaps. 

Example: 

Scenario:

The following user story is being refined:

Given an existing user fails to log in

When clicking " Recover password”

Then I should be able to log in successfully

It describes multiple acceptance criteria and technical considerations, but it does not clearly describe the contents of the recover password email.

Potential issue: 

Email content exposes undesired information in plain text, or the implementation does not match user needs.

Bug prevention: 

The tester questions what type of password recovery should be performed. Reset Link? Memorable Word? One acceptance criteria should clearly state security restrictions, such as not displaying passwords in plain text. All assumptions should be clarified and recorded in the user story

Sprint planning 

The main focus of the sprint planning is to determine which user stories will be part of the next iteration of work. At this stage, we can focus on aspects such as identifying the relationships between items, checking if what we are planning to build meets users’ expectations and delivers value, and confirming that the scope we are committing to is achievable.

Example: 

Scenario: 

The team overcommits on the scope to be delivered for the current sprint.

Potential issue: 

Shortcuts are taken in order to ensure all items are delivered on time, compromising future maintenance and the product’s quality.

Bug prevention: 

Clearly highlight any concerns regarding scope during sprint planning, and use daily standups as an opportunity to closely monitor progress and identify risks. Ensure everyone has visibility if corners are cut to achieve delivery dates, so that the scope can be split into other tickets and any technical debt is properly registered for future work.

Mid-sprint checkpoints  

From day one to the final day of the sprint, there are multiple opportunities where testers can help prevent issues not only with developers but also with designers, business analysts, and DevOps professionals. Your task here is to be fully present and take advantage of any interactions available throughout the sprint. Daily stand-up meetings, 3 amigos sessions, early feature walkthroughs, design document reviews, backlog grooming, etc. 

Example: 

Scenario: 

The developer raises a pull request for a user story that is getting ready for QA, but only other developers are assigned to review.

Potential issue: 

A developer’s code review tends to focus more on code structure and usually cannot identify clear gaps against the user story’s specification or how the new functionality impacts other existing functionality.

Bug prevention: 

Get testers involved in pull request reviews. If not on every pull request, at least on the ones that are identified as more critical to the current scope of work. The nature of how testers work allows them to have a better understanding of the big picture and how the new functionality fits into the existing ecosystem. This allows them to easily spot potential issues and also take the opportunity to ensure the user story is properly covered, testability has been considered, and no early problems/blockers are in place. 

Sprint review

At the end of each sprint, the development team demonstrates the work that has been completed to key stakeholders. During this session, the stakeholders have one more opportunity to provide feedback on what is being built and the overall project progress. This feedback is extremely valuable and, when actively monitored and managed, can play a key role in ensuring issues and improvements are fixed, but more importantly, are prevented from happening again in future user stories.

Example: 

Scenario: 

The stakeholder provides feedback stating that form validation needs to be displayed whilst editing and not on form submission.

Potential issue: 

New user stories are refined and added to the sprint without clear acceptance criteria to ensure validation is done on form filling.

Bug prevention: 

Sprint review feedback is captured, and the missing AC is raised if missing on new user stories

Alternative ways to measure software quality 

In terms of evaluating software’s quality, monitoring the bugs raised by QA is not enough. It won’t represent the level of quality of what has reached customers, and it won’t provide a full picture of how effectively the team is working. Therefore, we need to incorporate a diverse set of measurements. 

Here are some examples:

Through reviewing these indicators, we are able to continuously analyse the effectiveness of our test strategy. Identifying what is working well and what needs improvement so we can enhance our bug prevention strategy.

A new mindset for engineering leaders 

We are often quick at providing feedback when things don’t go well, but on the other hand, we don’t do the same when it comes to positive behaviour. A bug raised in the backlog demands time investment from other people in the team, from triage to fixing and release. If we want to emphasize the importance of prevention, we need to start shining a light on them whenever they happens. 

Everyone on the development team should have an understanding of the multiple benefits of preventing issues. Here are some examples:

• Reduced cycle time
• Less bugs raised
• Less back and forth between developers and testers

So the biggest question becomes, how do we encourage this behaviour? How do we provide more visibility on what needs to be done? Janet Gregory and Lisa Crispin's book Holistic Testing describes a healthy culture as “an environment where people feel safe to experiment, ask questions and point out problems as a prerequisite to an organization’s ability to deliver a high-quality product”. This is a key requirement to allow for free-flowing communication and gives everyone space to take responsibility for the product’s quality from day one.

Reward prevention, not just detection 

Active listening and being fully present on every team interaction are probably the most relevant changes you can implement. That will lead to a considerable increase in your capacity to identify gaps and propose changes to help the team build the right feature or product. Stop multi-tasking on team calls, be present and focus on bringing your skills and experience to every interaction.

Here are some ideas to increase visibility of prevented issues:

• Document Project’s Standards: Keep a record of architectural decisions, design standards, definition of done, etc. Refer to them where needed, highlighting the areas that impact each user story.
• Keep track of lessons learned: Common issues, best practices and any kind of learning from previous interactions should be easily accessible and referred to in the backlog items.
• Bug prevention log: Keep a simple prevention log list that helps keep track of actions, questions and practices that have helped the team build the right product.
• Bug of the week/sprint: Frequently shine some light on a “bug” that has been addressed by the team and never reached our end customers. Make sure prevention items are also included.

Can AI help with bug prevention?

Artificial intelligence is everywhere in the software development lifecycle, and there is no way to avoid it. As we shift towards a different development process, we should work towards using AI to help us prevent and identify these issues before they are coded into our apps. When considering our previous examples, most can be enhanced by incorporating AI agents or prompts, if done in the right way. 

Closing thoughts 

As testers, we cannot deny that there is a degree of pleasure in identifying an important issue that could have cost the company a lot of money. As a discipline, we want our work to be seen and valued, but we should not get our priorities wrong: Raising bugs is part of it, but the objective is to deliver the right product.

In this article, we talked about the problem of letting your approach focus entirely on raised bugs, remembering that a big part of our role is also to help our teams to be more effective, and that prevention is our best way of contributing to this. Reducing cycle time, lowering costs, and fewer escaped bugs are signals of a highly performing team, and we should set that as a target. 

Remember. Prevention is the highest form of testing.

References:

• Continuous testing in devops: Dan Ashby
• Shifting left & right in our continuous world - Holistic Testing with Lisa Crispin 
• What is Scrum? 
• 2020 Scrum Guide 
• Cycle Time: Martin Fowler 
• Holistic Testing: Weave Quality into Your Product: Janet Gregory and Lisa Crispin 
• What is Defect Escape: Aayush Saxena 
• Cost of Defects in Software Testing: The Real Numbers: Deep Barot