Performance testing survival tips for your e-commerce website this holiday season

A few weeks ago, during their Thanksgiving Day sale, Costco’s website crashed for nearly 16 hours. It is estimated that the company lost $11M as a result.

This story, along with others about large retailers like Nordstrom Rack, demonstrates that no matter how big the company, when it comes to an e-commerce website, there is always a need for more performance and load testing.

Costco certainly didn’t expect that sudden increase in their traffic. It not only cost them money, but caused customer dissatisfaction and reputational damage as well, and these are much bigger prices to pay over the long haul. This unforeseen event could have been avoided by conducting performance testing on their website to test various loads and conditions.

Performance testing for websites and mobile apps is essential to prevent bottlenecks, and to identify potential performance hiccups that may surface under extreme load conditions during big holiday shopping traffic times. We have some specific steps your team can take to ensure that your holiday season sales run smoothly.

Obtain a baseline number, then benchmark and repeat

• • Identify the critical business flow and peak production user load over a period of time, also known as the historic peak
  • Develop a performance test suite to simulate your historic peak, and obtain a baseline performance matrix for your application
  • Execute these tests before every release (or go live), or whenever changes are made to the underlying system, and compare your results with that of your baseline
  • Use those metrics to understand how your system reacts to load, and determine what affects the performance of the application

Identify stress points with historic peak load + 30-40% increase load

• • Use your baseline performance suite to execute increasing load test (with historic peak + 30-40% increase in load) against your system in order to determine the point at which:
    • Functional errors appear; and then, monitor the application logs
    • Performance degrades (system and application) to unacceptable levels

System endurance: soak testing

• • Determine the behavior of your system during prolonged use under peak load condition. Memory leak, Garbage collection, SQL cursors, storage space etc. have a tendency to only illustrate potential problems over time.

Brace for impact: risk mitigation (performance tuning)

• • Determine the stress points of the system under test and identify the points of failure. Mitigate risk around those areas by:
    • Increasing the hardware capacity of your system
    • Altering your system configuration and load balancing strategies to optimize performance
    • Enhancing solution and architectural design

Verify your solution

• • Rerun your performance tests to the point where they failed previously and confirm that your solutions have been effective

Extrapolation to production environment

• • Extrapolation is a tool for determining the theoretical performance capability of a production environment by scaling the results obtained from the QA environment

Be mobile ready

According to Salesforce, in 2018, 42% of Black Friday orders on various Canadian e-commerce portals were placed on a smartphone, and only 49% on a desktop or laptop computer. This marked the first year that computers generated less than half of all online orders.

Your mobile app should be able to sustain beyond anticipated peak load. This can be achieved by running mobile app performance tests on the full range of new device/OS combinations, alongside the web application performance test.

Black Friday, Cyber Monday and Boxing Day are sources of massive potential for online retailers. To make sure you don’t miss out, perform due diligence around your system. If you’re not sure where to start reach out to us and we will help you make sure that your holiday season sales run smoothly.