Two of the most exciting things in the world - load testing and the US census – recently came together to provide us with an interesting case study in launch preparation. I’m talking of course about the new (and free) website on the 1940 census that crumbled under huge traffic last week.
The story is interesting on a number of fronts (particularly, how it was paid for), but for the sake of this blog, I want to stay focused on the load requirements put in place prior to the site’s launch. It’s something we see here at uTest quite frequently: companies simulate what they consider to be an absurd amount of traffic, only to have that figure exceeded after launching. Not the worst problem to have, but a problem nevertheless. This can be caused by a huge media pickup or, as was the case with Inflection, becoming a trending topic on Twitter.
So how well prepared were the operators of the site? Here were the contractual requirements in terms of load testing, according to msn.com:
- “When browsing from one image to another, each image should be presented to the user in 3 seconds or less.”
- “When moving from the standard rendered image to each zoom level (e.g. zoom 1x, 2x, 3x), the reformatted image should be rendered in 2 seconds or less.”
- “Support up to 10 million hits per day while providing response times of less than three seconds for keyword searches of the descriptive metadata.”
- “Support up to 25,000 concurrent users.”
And how far off were they? Inflection’s general manager was quoted as saying, “We were expecting a flood, but we got a tsunami.” Here were the hard numbers: