The Game Episode 2: Double conversions with Sitecore split testing
How long is the ideal piece of content? According to Buffer Social, content that takes around 7 minutes to read captures the most attention. For context, 7 minutes equates to around 1600 words. Of course, each site and audience is different. The only way to find out what length of content your audience responds to best, whether it’s text or video, is to split test it.
As part of our Self-awareness is a Superpower campaign we tested two versions of the same text against each other to find out which one generated the largest amount of conversions. The first version of the text was more detailed, with a total of 711 words, whilst the other version briefly explained the subject in around a third of that amount - just 250 words.
By running a basic content test in Sitecore, we were able to test the two versions against each other. Learn more about the campaign in the video blow.
Our split test showed that the short version generated twice as many conversions compared to the long version. Visitors also spent twice as long on the short version of the page. That lead us to believe that, at least in the case of engaging with this campaign, people’s time is precious and their attention spans are short. The sooner we could explain the message of the whitepaper, the greater the chance we had of somebody downloading it.
To gain a truly accurate result, you’ll want to repeat the test with multiple pieces of content. By playing around with different content lengths you will eventually find that “sweet spot” - the ideal length for you and your audience.
Interested in split testing your content? To set up one of these tests in Sitecore yourself, follow the instructions below.
Setting up a split test in Sitecore 8.1
Step 1: Create multiple versions
After you’ve uploaded the content in its original state, in this case a blog post, create a new version of that page and make edits to it. This way, you’re creating two different versions of the same page.
Step 2: Create test in Experience Editor
After having created and saved the two versions of the page, go to the Experience editor to create the content test.
Step 3: Optimization
In the Experience Editor, click the Optimization tab. In this bar, you are able to create and monitor tests. To create a new test, click Page test.
Step 4: Setting up a page test
After having clicked Pages to test against, you will be shown different test options. In this case, we are testing the existing version with a previous version of the page.
Step 5: Review and start test
After choosing which version you wish to test your page against, go to Review and start. Here, you will be offered a variety of different options for your test:
• My expected effect of changes. Predict how the test will effect the visitor engagement. You can select from:
1. I expect a negative change in engagement value (There are situations where this option is useful. For example, the content author disagrees with a suggestion about a change in the content, and uses this option to start a test to show that they are right about the suggestion.
2. I expect no significant change in engagement value
3. I expect a positive change in engagement value
Sitecore calculates a score that shows how good you are at guessing. You can see this score once the test is over.
• Percentage of visitors exposed to test. Set the percentage of visitors you wish should be exposed to the test when running.
• Statistics. Set the minimum confidence level the test needs before it declares the result statistically significant. A high level requires longer test duration.
• Test objectives. Set the objectives for the test as well as how the winner will be selected.
• Duration. Set the duration of the test.
When you have chosen your preferred settings, start the test.
Step 6: Monitor or cancel test
The red circle next to the Optimization tab shows that there is an active test on the page. To monitor the test results, or to cancel the test, simply click Test results and the tracking of the test will show as well as an option to cancel the test and manually pick a test winner.
- The only way to find out what length of content your audience responds to best, whether it’s text or video, is to split test it.