This is a summary of the talk we did today at Velocity EU 2012.
In 1746, the abbé Jean-Antoine Nollet ran an electric current through 200 monks. The monks were connected to each other with 25ft iron rods creating a line a mile long.
The purpose of this bizarre experiment was to find out how far electricity could travel and how fast it could get there. The lack of suitable measuring equipment at the time led to hacks like Nollet’s, and as far as he could tell from the convulsions along the line of monks, electricity did in fact travel the entire mile instantaneously. Coincidentally, around 2006, lacking suitable equipment to measure network latency from within a web page, a certain Mr. Souders started us down a path of hacks that led to boomerang.
Experiments by Nollet and his contemporaries started us on a path to near instantaneous messaging. A message that, on horseback, might have taken weeks to get to its destination could now make it across in minutes. Of course, this initial boost in latency came with a significant reduction in bandwidth… to paraphrase ast, “Never underestimate the bandwidth of a mail horse as compared to the telegraph”.
Bandwidth v/s Latency
Bandwidth has gotten better over time, but it doesn’t look like light’s getting faster… and therein lies the problem. Users expect a faster web experience when they pay for more bandwidth, but we all know that for the web, more bandwidth doesn't matter much.
Unfortunately, our users don’t know that. As the price chart below from a popular ISP shows, users are trained to associate advertised bandwidth with speed.
So what are a user’s expectations and how do they change over time? It’s actually easier to tackle the latter part of that question, and even build a model to predict how things will progress in the future.
As an example, consider that the Bronze medal winner in the 100m dash at this year's Olympic games was faster than the Gold medal winner in 2004.
Expectations at the Olympics have changed so fast and so drastically, that the Gold medalist from 8 years ago wouldn’t even be in the top three today.
Let’s take a different example, one that doesn’t deal with speed at all. About 10 years ago, I was looking to book a room in a hotel at Picadilly Circus. One of the “features” of the pricier rooms was that they had an attached toilet and shower. Imagine a hotel today trying to sell rooms without an attached toilet and shower. FWIW, the hotel is no longer in business.
What delighted users a few years ago is now an expected baseline, the absence of which will frustrate them.
Now consider rate of change of expectations on the Internet v/s Athletics or the Hotel Industry.
To delight or frustrate… that is not the question
As long as you’re somewhere in between delighting and frustrating your users, you’re meeting expectations. If you drop below that band, you’ll lose customers, and if you go beyond, you’ll steal them away from your competitors… you’ll also change the baseline that you need to meet. To predict where you need to be, take stock of what delights your users today, and that’s your baseline in 2 years.
So where’s the web at today?
To easily quantify and quantify user expectations, we’ve defined a term called
LD50. The term derives from the biological sciences and refers to the Dosage of a drug that’s Lethal to 50% of the tested sample. We define the
LD50 value for a website as the load time at which 50% of your customers bounce.
For this study, we looked to see if there were cultural differences between different types of users that led to different levels of patience.
We first looked at the
LD50 value for users across different browsers, and found that IE users tend to be the least patient.
50% of IE users left a site after 3.6seconds, while the numbers for Firefox and Chrome were 5.6 and 6.5 seconds respectively.
Also notice that after a certain point, your page is so slow that you’re annoying all users uniformly. When things start to suck, culture becomes irrelevant. I’d love to run this test between
vim users and
emacs users and see how things shape up.
Since we’re looking at culture, and few things define culture more than the nation that one chooses to live in (well, perhaps ones text editor or email client), we decided to look at the
LD50 values for users from different countries.
In this test, Australians and Germans tend to be outliers (either falling entirely below or above the 50% mark), and we’re not entirely sure of why, though we have our theories. Of the other three countries we looked at (and this is based on the quantity of data we had for each country), Americans are the least patient, and Canadians are the most with Britons close behind.
We enjoyed doing this talk. Our slides are on slideshare if you’re interested in looking at them.
We’ll run these tests periodically to see how things change, and we’d love to run others as well. If you have ideas of what we should look at, please let us know, and if you’d like us to measure and analyse your site’s data, sign up for a 30 day free trial and we’ll get started right away.