1. Knowledge Base
  2. AB Testing
  3. Using A-B Testing in Skills

Using A-B Testing in Skills

What is A-B Testing?

A-B testing (also called split-run testing) is used to compare two versions of a webpage/app against one another. A-B testing is used to determine which version is performing better. Consider it an experiment where variants of a page are shown to your customers at random. Then statistical analysis is used to figure out which of the two variations is performing better, based on your goal. Fera uses A-B testing in all 7 Skills to determine whether or not showing your customers the urgency or social proof notification versus not showing them has an impact on conversion rates.

Why Should I Use A-B Testing?

A-B Testing can be used to make changes to your site and then compare them to leaving the site unchanged. It lets you collect data about the impact the changes had on your site (whether they’re good or bad) and helps you determine what changes to make. A-B testing can help you prove a hypothesis wrong or it can be used to continually improve a given experience. Fera uses A-B testing to show your customers a variation of your site with our social proof or urgency notifications and the control without them. This lets you determine if Fera is having a positive effect on your store and boosting your conversion rate!

How Does A-B Testing Work?

For an A-B Test you can create 2 versions of a webpage. The change can be something simple (font change, colour change, you moved a button or title) or be a complete redesign of the original page. Then when customers visit your visit half of the visitors are shown the original version of the page (the control) and the other half are shown the modified version of the page (the variation). Once the data is collected and analyzed you can determine whether changing the page’s design had a positive/negative effect, or if it had no effect.

How Can I Use A-B Testing?

Within the Fera app, click on the “Learn” tab and then you will see the “A/B Tests” section. By default, this will be disabled and Fera will show to 100% of your site’s visitors. If you have a lot of data (~>500 orders a month) you can use A-B testing to test the results. If you don’t have a lot of orders yet leave this off and we’ll use average numbers for stores your size to calculate results.



When you navigate to other tabs within any of the Skills you will get a warning. This warning is a reminder not to modify or edit the campaign in any way while you have an A-B test runningĀ live. This is because it can skew your data and make your results invalid or inconclusive.


The A-B Testing Process

  • Collect Data

    • Collecting data helps you to understand where you could make changes & what areas could be optimized
  • Create a Hypothesis

    • Once you begin you can think of A-B testing ideas and create hypotheses about why they’d be better than the current version
  • Create Variations

    • Using your A-B testing software you can make your changes to create a variation of the original or run a comparison
  • Run the Experiment

    • Launch your A-B test and wait for customers to visit your website. Data is collected from visitors. The data is based on whether they were randomly assigned the control or the variation
  • Analyze Results

    • Once the time is up on your A-B test you can check out the results. This will let you know if there’s a statistically significant difference between the control and variation. You’ll be able to decide whether or not making the change is worth it!


For more information you can check out this VWO article. Go ahead, prove yourself right…or wrong!

Was this article helpful?

Related Articles