Annadiva is a lingerie store that specializes in larger cup sizes. In addition to their webshop they also have a number of physical stores in the Netherlands.
Annadiva has been working with Adchieve for a number of years now and has achieved excellent results during this period of collaboration. In this case study we’ll discuss the A/B test for Shopping that we designed together in more detail.
Challenge
In this industry, two image variants are generally used to showcase a product. A variant with a model and one without a model. Annadiva always advertised in Shopping using an image of the lingerie, without a model. The challenge that they faced was finding out what produces a higher CTR within Google Shopping: an image without a model or one with a model.
For Annadiva, Shopping is a major source of income. Minimal differences can therefore have a huge impact.
Approach
To provide insight into the differences in results, we started testing both image variants using an A/B test. The Shopping A/B test is a feature in the Adchieve software that can be used to test three components of the content: the title, the description or the image. In this case the image was chosen. The first variant only shows the product and the second variant is an image of the product worn by a model. To lower the risk, it was decided to limit the test to one brand. We chose the brand Marie Jo, which is one of the bigger brands and has a sufficiently high volume.
For the purpose of the A/B test, a hypothesis was formulated in advance to describe the expectations:
“In Shopping, an image with a model results in a significantly higher CTR than an image of the product alone”
The test was set up using the Adchieve software. The variable that was tested, in this case the image, was filled using the feed. Annadiva made sure that one column was filled with image 1 and one column with image 2. Then the complex technology of the Adchieve software was put to work.
Within the Annadiva campaigns, each groupid is an ad group. Within an ad group, each itemid normally is a product group. However, for the test a new production partition was created based on the itemid, inserting variant 1 and variant 2 as a product group via a custom label.
Let's go!
Using the Adchieve software, variant 1 or 2 was randomly forwarded to the Merchant Center. After 21 days, Adchieve again randomly sent variant 1 or variant 2 to the Merchant Center.
When a new image is sent to the Merchant Center the question always is when it will be updated within Google. Unfortunately there’s no way of telling that. Google indicates that it will happen after no more than 7 days. To overcome this problem there’s an “Everything else in…” product group that also receives data. When a new image is sent, it will be part of this group for the first 7 days. At that time, no variant 1 or 2 is attributed to the product in question in the custom label. After 7 days either variant 1 or variant 2 will be allocated to the product.
This ensures that the test is as fair as possible. Since this is done using a custom label, the results are visible in the Google interface.
‘‘The image with a model scored a significantly higher CTR in the test.’’
Results
The test ran for six months (from January 2019 through June 2019). Variant 2, which was the image with a model, scored a significantly higher CTR in the test. The significance was checked using a Welch’s t-test.
Based on the results, we can conclude that the chance of a click is greater when we use an image with a model.