Spring Cleaning Your Test Cases – Make Time for What’s Important

There isn’t really a time of year when it is better to look at your test cases and to undertake a spring cleaning exercise than now. Projects, processes, and organizations all have different periods of highs and lows in their workloads and, only occasionally, do these line up against a regular schedule. These facts do not negate the reality that, over time, the best-intended habits around test case maintenance will fail and some extra effort will be needed to clean things up. Test case debt is a reality that must be faced.

It is an eternal fact of software development that things change over time; new features are added, bugs are fixed, environments change. When a test case is written, it is designed based upon the current workflow of a system. You can develop your test cases, and you should do this, to be data driven and have minimal amounts of revisions necessary through anticipated change. This cannot take into account, however, the fact that base flows, landmarks, operations can and will change as the requirements of the application change. This evolution that renders test cases out-of-date can be termed “test case debt”.

The first and best way to keep your test cases, clean, targeted, and relevant is to create a culture in your testing team to update, fix, and maintain test cases as issues are found. The best and most reliable time to find issues and to know what the appropriate changes are is when you’re testing. At this stage, you can find a discrepancy and have the immediate context to know how the case should be changed. In the real world, however, this isn’t always possible. Sometimes other factors get in the way and no matter how rigorous you have promised to be in the future; issues will be passed over for the sake of time, efficiency, or need. Over time, these issues will build up.

It is advisable that time is set aside on a regular basis to tackle some spring cleaning on your test cases (regardless of the season). As with anything that needs regular maintenance, if things are left unmaintained for too long, the amount of work to get them back to being tidy becomes overwhelming and the workload itself becomes the barrier to getting the test cases back in line. Inaccurate, old test cases become unusable for new employees, contractors, or outsourcing efforts. Often, when you onboard people or go out to external resourcing, time is of great concern and having difficulty using test cases will be a problem.

One thing that I’ve done on some of my teams is to keep a collaboratively accessible list of test cases that need work. This means day-to-day, when someone has some time, they can go back and fix a test case. Or, when you decide to dedicate some time as a team, you have that resource available. Another fun option is starting something called “Test Case Friday” where once a month, or so, you block off a Friday and make it into a fun afternoon with treats, maybe adult beverages, and competition for the most test cases fixed.

Prioritizing your test cases for review and updating can be challenging, but as with all tasks that may not be entirely completed in the allocated time, you should look at tackling the most important ones first. Although intuition might indicate that you should look first to the cases that get the most use, we recommend that during spring cleaning you should take care of the less used, more obscure cases. We consider this a risk-based approach to spring cleaning. Commonly used test cases have a higher chance of being corrected during regular work and steps that do not stray far from the actual are more likely to be caught and have work-arounds. On the other hand, rarely used tests cases will likely be in parts of the application that are tested less often and, as such, testers will be less familiar with those areas. This will lead to, at best, time wasted when false fails are investigated and, at worst, mistakes that simply aren’t caught.

No doubt about it, test cases need to be maintained to be useful. The more that they drift from useful over time, the more work it will take to bring them back on-point when they are needed. No matter how dedicated you make your team to in-line maintenance, there will always be test case debt to be recovered. By dedicating some time on a regular basis you can keep this under control. You don’t have to do it in the spring, of course, but what better time to do it than when the flowers and birds are out and cheerful?