Articles on: A/B Testing

A/B Test Product Recommendation Campaigns

Plans: All Plans Platforms: Shopify


AfterShip Personalization’s A/B testing feature allows you to split your website visitors into two groups and display each with a different product recommendation strategy. You can determine which strategy is performing better and which needs improvement by analyzing the key metrics, including conversions and average order value (AOV).

You can A/B test your product recommendation strategy for the home page, product, cart, checkout, thank you, and post-purchase page.

You can enable version A with product recommendations and disable version B without product recommendations to show the merits of product recommendations on various key checkpoint webpages

You can enable version A and version B to compare two different product recommendation strategies and determine which one is performing better.

What you’ll learn

In this article, we will show you:

Set up A/B testing for multiple scenarios
Additional considerations
Data accuracy

Set up A/B testing for multiple scenarios

A/B testing is available for all the supported scenarios in AfterShip Personalization: home page, product, cart, checkout, thank you, and post-purchase page.

For every scenario, the setup process is the same. Let’s take a look at the example of the home page.

Go to Smart upsell widgets in the AfterShip Personalization admin

Click Get started. You can edit an existing campaign (widget) or create a new one. To create a new one, select Create widget

Product recommendations strategy editor will open

Edit the product recommendations settings based on your requirements. These settings will be used for version A

Configure settings with ease with detailed guides on each here

Click Create A/B test on the upper-right hand side

On the popup dialog, select whether you want to create a copy of version A and edit or create version B from scratch and hit Continue

Configure the traffic volume for A/B testing. Drag the bar left or right to adjust the display percentage of each version.

By default, the traffic would be distributed equally amongst both versions.

Click Continue

The editor that will now open will be for version B. Edit the product recommendation settings as per your requirements and click Save

Switch between both versions by clicking the downward arrow in the upper-right-hand corner of the page

Additional considerations

Enable both versions if you want to show each version to the customers based on your traffic configuration.

You do not have to install the widget for both versions on the Shopify theme editor separately. The editor code for each will remain the same.

The preview window will always be open for the enabled version first. If version A is disabled, a preview for version B will display on the preview page.

After accumulating enough samples, you can check the progress of each version on the product recommendations dashboard and access which one is performing better.

Data accuracy

The traffic distribution division is based on each browsing session without a tracking cookie, meaning consumers can only see one version during one browsing session.

The accuracy of traffic distribution is significantly dependent on the traffic volume. Our research shows this accuracy is much higher when the traffic amount exceeds 1000.

Here’s the data indicating how data accuracy enriches with the traffic volume.

Traffic amountAccuracy (1:1)
Version AVersion B

For any further questions or help, please contact our chat support team.

Updated on: 22/11/2023

Was this article helpful?

Share your feedback


Thank you!