Augmenting Testing in your Agile Team: A Success Story

One of the facts of life about Agile is that remote resources, when you have a mostly collocated team, generally end up feeling a little left out in the cold. Yet, with appropriately leveraged tools, sufficient facilitation, management support and strong team buy-in, it can end up being a very successful arrangement.

There is an implementation model that lends itself more naturally to adding testing resources, or a testing team, to your delivery life cycle.  Rather than embedding your resources, you can find ways to work with the teams in parallel, augmenting their capabilities and efforts in order to achieve greater success.   In this article, we’ll look at a particular case where PQA Testing implemented an augmenting strategy to tackle regression and System Integration Testing (SIT).

Recently we were working with a company that delivers a complex product in retail management to assorted third party vendors.  Features were created, tested and marked ready for release by functionally targeted Agile teams.  Coming out of a sprint wasn’t the last step before a feature was released, however.  Due to the complexity of the product, environments, other systems controlled directly by the third party vendors and other systems controlled indirectly through their third party vendors, System Integration Test (SIT) cycles and User Acceptance Test (UAT) cycles were necessary.

The original intent, when our client went Agile, was to be able to continue to support these relationships through the Agile teams.  What soon became evident was that the amount of regression testing in the SIT environments required for the new features was overwhelming to the testing resources dedicated to a feature team.

Additionally, as multiple environments and numerous stakeholders from various vendors with their own environments were introduced, simple communication, coordination of environments and testing became much more complex and time consuming.  Defects that were found in SIT testing needed to be triaged and coordinated with the other issues created from other vendors, and then tracked as they moved their way through the different teams and vendors to their solution.

As the testing resources on each team focused more on their functional area, their knowledge became more and more specialized and they were no longer the “go-to” resource for questions that might span the entire domain. With this specialization, testers were no longer collecting as much domain knowledge. Additionally, while automation was an integrated part of the company’s solution, test automators were also embedded in the Agile teams.  This changed the focus of automation; it slowly drifted away from providing benefits at the end-to-end integration testing level.

When we began the engagement with this client, they were succeeding from release-to-release, but not at optimum levels of quality, or to vendor satisfaction.   They were borrowing resources from multiple Agile teams and sometimes breaking sprints to ensure that the release could get through the SIT cycle within the specified time frame.  As we do on every PQA Testing engagement, we began by learning the existing process, how the software worked, and about the entire domain.  Before long, we took over regression testing for the releases.  Our focus then became to make sure that the existing functionality remained stable and clean, and that the new features integrated into the system well. 

The testing team is now a separate team that is semi-integrated with the existing teams.  We transition knowledge back and forth, but there is a distinction in responsibilities between new features and regression and SIT testing.   As we began taking over these testing responsibilities, we also began to take over communication and facilitation between the core vendor and our client for release and testing.  An automation resource is also able to work through the tests from the big-picture integration perspective, and is reducing the amount of manual testing that is necessary.  Increasing our documented domain knowledge is making it easier to scale the team as necessary during busy times and releases.

Taking over these requirements with a dedicated team has greatly improved the feedback coming from the vendors.  The Agile teams have more focus on their core deliverables.  Integrating remotely with the client’s teams has worked well because we don’t have to constantly interact face-to-face to show value in our work.  We are simply another team trying to move the ball forward for the company, just like everyone else.

Remote testing teams dedicated to ownership of specific testing functions can remove many of the obstacles of testing remotely in an Agile environment and, in this case, better ensure quality for the end user.