Four key performance measurement terms explained (so that normal people can understand them)



(Today’s guest post comes from Exclusive Concepts’ partner, Radware. Exclusive Concepts partners with Radware to offer its enterprise-level site speed optimization solution as a hosted service. Read more here.).

Joshua Bixby of Radware

I’m Joshua Bixby. By day, I’m VP Application Acceleration at Radware, where I evangelize web performance both internally and externally. By night, I mine data about performance and its impact on users.

In my travels, I meet a lot of site owners who tell me their sites are blazing fast, with response times of less than a second. Sometimes they get this number from their analytics, sometimes from a performance vendor — but wherever they get it from, it’s clear that the message this source is sending is that response time is a great metric for understanding how users interact with your site. It’s not.

There are two common – and understandable — mistakes that people make when discussing web performance:
• Using “response time” and “load time” interchangeably.
• Not realizing that “response time” can mean any number of completely different things.

There isn’t a lot of effort to educate the lay public about what these and other measurement terms mean. To address this problem, here’s a simple guide to understanding four fundamental performance measurement terms, and when and why you should care about each.

Response time

What it means: Response time is incredibly tricky, and it causes a lot of confusion. It can refer to any number of things, depending on whom you ask: server-side response time, end-user response time, HTML response time, time to last byte with no bandwidth/latency, and on and on. Long story short: There’s no single definition.

Caveats: If someone starts talking to you about response time, ask them to clarify which response time they mean. Be wary of anyone who tries to sell you on the idea that there’s only one definition. If user experience matters to you, ask how whatever type of response time you’re looking at relates to what the end user actually sees.

When it’s useful: Different types of response time measurements tell you different things, from the health of your back end to when content starts to populate the browser. You need to know what you’re measuring and why.

Time to first byte

What it means: Time to first byte is measured from the time the request is made to the host server to the time the first byte of the response is received by the browser.

Caveats: Time to first byte doesn’t really mean anything when it comes to understanding the user experience, because the user still isn’t seeing anything in the browser.

When it’s useful: For detecting back-end problems. If your website’s time to first byte is more than 100 milliseconds or so, it means you have back-end issues that need to be examined. (Web performance consultant Andrew King has written a good post about this, as has Google performance expert Pat Meenan.)

Start render

What it means: As its name suggests, “start render” indicates when content begins to display in the user’s browser. This term seems to have evolved as an alternative to “end-user response time”, but it’s not yet widely used outside of hardcore performance circles.

Caveats: Doesn’t indicate whether the first content to populate the browser is useful and important, or simply ads and widgets.

When it’s useful: When measuring large batches of pages, or the performance of the same page over time, it’s good to keep an eye on this number. Ideally, visitors should start seeing usable content within 2 seconds. If your start render times are higher than this, you need to take a closer look.

Load time

What it means: The time it takes for all page resources to render in the browser — from those you can see, such as text and images, to those you can’t, such as third-party analytics scripts. (Geek version: “Load time” is also known as “document complete time” or “onLoad time”. It’s measured when the browser fires something called an “onLoad event” after all the page resources have fully loaded. No matter what you call it, it’s used as a primary measuring stick for site performance.)

Caveats: Needs to be taken with a grain of salt, because it isn’t an indicator of when a site begins to be interactive. A site with a load time of 10 seconds can be almost fully interactive in the first 5 seconds. That’s because load time can be inflated by third-party scripts, such as analytics, which users can’t even see.

When it’s useful: Load time is handy when measuring and analyzing large batches of websites, because it can give you a sense of larger performance trends.
Three things to remember:

1. There’s no single “right” way to measure performance. Each measurement tells you something meaningful about how your site performs.

2. You need to understand the different performance measurement terms so that you can interpret your own data. If you don’t, sad to say some people will take advantage of your ignorance to mislead you for their own benefit. (For example, some performance vendors have convinced site owners to tie bonuses for key employees to backbone test results, which do not measure real-world performance.)

3. As a matter of due course, you always need to gather large batches of data about your site’s performance and rely on median numbers. But you also need to periodically get under the hood – using tools such as WebPagetest* — and take a real-world look at how your pages behave for real users.

Case Study

site speed optimization case study

See how Exclusive Concepts and Radware helped an IR500 merchant cut load-time up to 73%, while instantly increasing revenue by 28%.



*WebPagetest is a third-party tool that simulates how fast a site loads for real-world users using a variety of browsers.