How to Start A/B Testing Websites


“It is a capital mistake to theorize, before one has data” – Sherlock Holmes.

What is A/B testing?

A/B testing is a design practice that tests two different versions of a piece of content with your audience, and then measures the performance. In an A/B test, both versions are run simultaneously to equally sized audiences, and the page that performs better (i.e. gets more conversions, has higher engagement rates) is declared the winner!
Don’t wait any longer — use our how-to video for beginners above and the following guide to start now!  

What should I test?

Main Landing Pages

The landing page is the first page people hit on your site. Remember — this isn’t necessarily the homepage of your site. In the same way that stores often put up “SALE!” signs on their display windows, you want your front doors to be as pretty as possible. The only way to find out what works best is through testing your landing pages!

Conversion (SEM) pages

These are pages that advertising or campaigns send traffic to, with the goal of ‘converting’ people to take action. Test out the color of your donate button. Experiment with more convincing copy on your newsletter signup page. Never make assumptions about your audience will do, when you can instead be using data!

Page Templates

Optimizing page templates is a quick way to optimize multiple pages at a time. What happens when I make a menu change, or add something to the header? A/B testing will help you find an answer to all of those questions.

What is the most important component of A/B testing?

Content! Say it with me – content is king. If there is no text on your site, people won’t do what you want them to do. Here are some things to remember with regards to copy when website testing:

1. One call to action is better than multiple calls to action.

2. Be concise, use bullets.

3. Copy should influence decision.

How should it look?

Design is crucial. You should definitely have a designer on your team (we have one here at Whole Whale – Ann, she’s awesome). Here are some guidelines when thinking about what your site should look like and how to test it:

1. Design for one action – we only want people from our audience to do one thing.

2. Simplicity/K.I.S.S – Keep It Simple Stupid. Remove all unnecessary distractions.

3. Color consistency and contrast – these are all your friends. For example, make sure that your “action” button is a high contrast color so it’s easy to see and people can pick it out quickly.

4. Mobile friendly – if your page is not mobile friendly…then…I don’t even know. People on mobile devices, are people too. They can convert…I promise.

5. Use hero images – The Good: babies, dogs, active happy kids, eye contact. The Bad: feet, busy photos, unrelated stock photos.

Where to start A/B testing?

Just do it! The only way you can mess this up is by not doing anything. Don’t wait 6 months to start an A/B test. Do it now. In the video above, we provided you with the most crucial steps to get you going. Now that you’ve watched it, you have no excuses. We A/Believe in you! Let us know how you did, and send us any comments or questions on Twitter @WholeWhale.

Thank you for reading and watching our videos – we create these for free in the hopes that they find their way to the marketing and content creators at nonprofit organizations. Anything you can do to share these with the sector is deeply appreciated!