Release Criteria – What is your ‘Quality Bar’?

In a previous article, I discussed managing risk with quality gates (“None Shall Pass…unless? Managing Risk with Quality Gates” http://thinktesting.wordpress.com/2012/06/26/). Such gates and their expectations facilitate tracing issue root cause and examining which preventative measures failed or need to be enhanced so as to continue to lower costs associated with poor quality.

But that last gate is there to enforce a level of standard, or ‘quality bar’, for what is ultimately to be published, deployed, or otherwise seen outside of the project team itself.  To guard that gate, we need “release criteria”.

Assuming that you won’t be checking against these criteria unless you have passed all prior gates, release criteria are the last few things that need to be ‘just so’ to make your release a success.  Or more bluntly, meeting these last criteria is required to allow you to make a release at all.

Go/No-Go – A Rare Binary Decision

What is important at the point of making a release?  As a team, we need to be sure that both the technical and business risks associated with the value we intend to provide have been mitigated to acceptable levels given the purpose or intended usage of the release.

How do we evaluate whether something has been ‘mitigated’?  As a team, we need to describe a set of criteria in a quantifiable way such that they can be confirmed or not by a simple ‘yes’ or ‘no’.

Black, White, and a dab of Grey

To keep the focus clear and create the best chance for a release to make it out the door without compromising our standards, we can separate release criteria into two categories; what MUST be achieved before a release can happen and what is EXPECTED to be achieved.

The following are the minimum criteria that MUST be met prior to a given build being released. If these criteria are not met, the build will not be approved for release and corrective action must be undertaken and given the highest priority.

Every testable defect fix has been confirmed by testing to be implemented successfully.

All acceptance tests must pass.

The release notes will document if a particular change request or new feature is included in the build but has known open issues against it at the time of release.

The following are the additional criteria that are EXPECTED, but not absolutely required to be met for a given build prior to it being approved for release.  If any of these criteria are not met, corrective action will be undertaken following a risk/impact assessment of the unmet criteria (eg: non-conformances will be logged as issues of varying priority).

This provides the project team the ability to react to other situations should they be determined to have a higher priority at the time.  However, it would be expected that corrective action would be undertaken in time for the following release.

Going Back for More

If we return to the idea of quality gates throughout the project lifecycle, we can add another dimension to release criteria.

Suppose we consider the promotion of a build from one environment to another a ‘release’ and, as such, establish criteria for this maturing of the system or code-base towards the final gate.

The following diagram provides an example of what this might look like at the highest level:

Conclusion

If the project team has a ‘quality bar’ based on criteria that are practical and quantifiable, you will benefit from having a repeatable standard for quality in your releases.  Then challenge yourself to be even better.  Your customers will love it.