Get in touch with our team

18.02.2021

5 min read

Where to start with a testing hypothesis

This article was updated on: 07.02.2022

Starting an A/B testing campaign or CRO as a whole strategy can be a daunting prospect especially if you’re already working on traffic acquisition (SEO and PPC etc) as well. You have to choose your software, possibly appoint a CRO agency, or choose to learn as you go along. It’s a lot to think about, so here is some help on where to begin with creating a/b testing hypothesis.

Why hypothesise?

It’s really tempting to dive straight into a/b testing and start testing all of those little things that you have wanted to change on your website but have not had the dev resource to make it happen. Yes, you can get some results from the low hanging fruit and obvious changes that need to happen but, the real results come from building informed hypotheses created from data and user information. 

By using a plethora of information sources, we can build a clear picture of the areas of your website that can be improved and most importantly a steer on how that might look. This process removes guesswork, bias, HIPPOs (highest paid person opinion) and ensures a/b tests are user-based.

Quantitative Data

Before analysing data that you will base hypotheses on to change your website, the first step is to make sure that your data is clean and of high quality. This way, you can be confident in the decisions you are making. If you have not checked your Analytics data for a while I would recommend having an analytics audit first, before deep diving into analysis. The last thing anyone wants is to end up forming hypotheses based on bad data because a/b tests will fail and time will be wasted.

What data and metrics are the best to analyse?

To find areas on your site that can be improved, page and user metrics are the best places to start. Look at landing pages and site entry points to see if you are losing traffic (that you probably paid to acquire) immediately. Analyse the following:

  • Bounce Rate
    • Any pages 60% or higher bounce rate with a good session volume
  • Avg time on page
    • Check how long users spend before clicking through or leaving, too low and this could suggest that content is not engaging enough, too high may mean that the page could be confusing for users
  • Conversion rate
    • Do users convert after landing on this page? If not they could be bouncing immediately and if they aren’t bouncing,  they may get lost in the following pages of their journey

If your landing pages are performing well then user issues could be further into the user journey. To find those problem pages where users leave the site before converting analyse pages with:

  • High exit rate
    • Any page with a high amount of pageviews and an exit rate above 35%
  • Average time on page
    • Again how long are users spending before clicking through or leaving, if you have a high exit rate and low average time, this can signal problems on the page.

Running this type of analysis will sign-post you to pages and areas of your site that can have issues. The next step is to analyse those pages with qualitative user data. 

Qualitative user data

From the quantitative analysis, you have a list of areas and pages that need attention. The next step is to analyse the user behaviour occurring on those pages and try to discover at which point users are getting frustrated and leaving your website.

Qualitative user data can show us the different types of behaviour of users on your website, this comes in forms such as heatmaps, scrollmaps and session recordings.

Heatmaps

A basic heatmap will show us where users are clicking and importantly, where they are not clicking. We can find the business driving content and page elements being missed by users and hypothesise how to reposition them in an a/b test to make gains.

Scrollmaps

Scrollmaps will also show us important content and elements being missed by users but from a page length perspective. You can find what content is scrolled past (ignored) or positioned so far down the page that it’s missed.

Scrollmaps can inform your hypothesis on moving content high or lower down the page so that as many users as possible view the most important elements are viewed by as many users as possible. 

Session Recordings

Session recordings are a great way to back up your thoughts. Finding examples of users tripping up on the problem areas you have previously identified can be a very satisfying end to your analysis, and to confirm your hypothesis.

Session recordings can be easy to get lost in, most software can record thousands of sessions so time can be wasted trying to find site issues just by viewing recorded sessions. Only use session recordings to find specifics when you are quite confident from previous analysis and you can pinpoint what you are looking for.

Conclusion

The information from this activity allows us to form testing hypotheses founded on qualitative and quantitative data. This helps eliminate guesswork and testing based on hunches. Grounding test hypothesis in user data gives you a much higher chance of running successful a/b tests. Increase your test win rate and increase your conversion rate faster.

To find out more on optimising your website to get the most of your existing website traffic, head to our CRO agency service area or get in touch with our team today.