LimeSpot offers an A/B testing feature that allows merchants to determine which to combinations of recommendation boxes and/or personalized content lead to the best performance. Using the A/B Testing feature gives you, the ability to create and run multiple parallel experiences and monitor the results using the LimeSpot analytics.
🔒 This feature is included in Premium and Enterprise plans and is available as an add-on for all other plans.
Preparing for an A/B test
To start an A/B test, you will need to create 2 different LimeSpot experiences. For example, you may want to test whether the Related Items or the Frequently Bought Together recommendation box is better for the product page of your store. In which case you would ned to create the following 2 experiences:
Experience A: Default with Related Items in the product page
Experience B: Frequently Bought Together in the product page
Next, you need to decide how much weight you want to allocate to each of these experiences during the experimentation phase. The weight is an number that corresponds to how much of the traffic should be sent to an experience compared to another.
For example, if both experiences have a weight of 10, each will receive 50% of the traffic, whereas if one has a weight of 5 and the other 15, the traffic distribution would be, respectively, 25% and 75%.
A weight of 0 will disable an experience.
Creating an A/B test
Go to Conversion > Website Personalization. You will see a button at the top right called A/B TEST.
Clicking on the A/B TEST button allows you to see your current default setup. Since the default is the only setup at the moment, its weight and live odds are both 100%. Click on CREATE to create a new experience or you can also DUPLICATE the current Default experience to copy all the customizations and settings that you have set up.
In the Experience Settings, change the title to give details for the testing experience, the weight, start date, and start time. Click on APPLY to save the changes.
Once you have created all the new experiences, you will see the experiences listed under A/B Testing Experiences. From now on, each experience has its own independent setup.
When you click on each experience, you will see what boxes are enabled on what page. Go ahead and customize the setup by clicking on CUSTOMIZE for each experience.
Experience B: Product Page: Related
Experience C: Product Page: Frequently Bought Together
Once you are ready to start the A/B test, PUBLISH the changes in each new A/B Test experience and this will schedule the experiences to start on the start date and time specified.
Go to the currently running experience (ie. Experience A: Default) and click on EDIT to put an end date and time for this experience. The end date and time should be the same as the start date and time of the A/B test experiences created above. This will make sure that this experience will end when the A/B test experiences start and all the A/B test experiences (ie. Experiences B and C) will start with fresh traffic.
Let the A/B test run for at least 14 days to get enough traffic and data to compare each experience.
To ensure the integrity of your tests, customizations will be disabled while A/B tests are running. If you must make changes, you will have to click on STOP NOW to stop the A/B test and then DUPLICATE it to re-run it for a new A/B test after you have made the changes.
Make sure that you create new experiences for the A/B test with the same start date and time so that the traffic on both experiences starts from scratch. Do not forget to put an end date and time (the same as the start date and time of the other experiences) on the currently running experience so that the current experience will stop at the same time the A/B test experiences will start running.
🤔 Questions? Need help? Contact us via the in-app chat or email.