The authoritative guide to product testing with FB Ads

The authoritative guide to product testing with FB Ads

You have researched your niche and imported a collection of products that you think can sell into your store. How do you confirm that they indeed have a demand?

In this guide, we will discuss how to test products effectively with FB Ads. It is a relatively technical guide and should take 50mins to read.

Contents show

What is product testing?

It is not about examining a product’s functionalities or quality. In the context of dropshipping, it means rapidly validating the market demand for a product.

Specifically, to carry out product testing, one runs a test marketing campaign with a product offer to see if anyone in a target audience takes it.

Why do you write this guide?

I have received many questions about product testing.

Dropshippers have troubles finding a winning product for many reasons. Firstly, before product testing, there’s a research process that people tend to omit.

Also, FB Ad Manager is highly technical to use, with many levers and controls. It takes a great deal of effort to set up and run a successful ad campaign on Facebook.

There are many videos and articles out there teaching various techniques of product testing. But not all of them are effective. Some are downright incorrect and can cause you to waste your money.

These must be confusing to a dropshipper who is starting to figure product testing out. So I want to write a comprehensive guide to cover all aspects of this process.

I want to cover how best to run a campaign to validate product demand. Scaling up the winner is out of the scope and is a separate topic for the future.

Pre-requisite for following this guide

  • You have a list of product candidates you want to test.
  • You have a decent-looking store with the products added to it.
  • You have set up Google Analytics on your store.
  • You know the basics of FB Ads and own a FB Business Manager and a FB Page for your business.
  • You have installed a FB pixel on your store and configured it correctly to track the different buyer events.

This guide is for more advanced readers, so I will not cover the basics. If you need help with the above topics, I suggest reading:

What does this jargon <X> mean?

I will make heavy use of FB Ads terminology and acronyms for convenience. In the text, I will expand each acronym only once on its first use.

You can find the definitions of the technical terms we use here.

How do I create a proper product page for product testing?

The product page is the landing page of your product testing campaign.

You should create a decent-looking product page before you send any traffic to it, but perfect or complete the page only after identifying an ad set that can send targeted traffic to it.

Your product page and the whole store, in general, should have a professional design that radiates vibes that match the niche.

You should do proper copywriting to highlight the product’s value and show how it solves the targeted customers’ problem, if any, or how it meets their needs, rather than passively listing the products’ features.

You can’t expect the visitors to read every line of your product specs and figure out what your product’s value is.

That said, you shouldn’t spend more than a few hours making a product page in the beginning.

Chances are you won’t be able to find a winning ad. If so, you would then have wasted the effort you put into perfecting the product page.

Instead, start optimizing a page’s conversion rate only after finding a way to land potential buyers on it, such as discovering an ad set with good CPC (cost per click).

That said, it is very challenging to create a product page from a blank slate. To help myself, I keep a list of successful dropshipping stores categorized by niches as models to study their product page design and copy.

The list is available to all our newsletter subscribers on request. If you are interested, sign up for our mailing list here and request us for the list.

A well-written copy, an irresistible offer, and quality images are what make a landing page highly-converting.

How do I set the product price for testing?

Do not fret over your product’s selling price initially, as you are almost certainly going to tweak it down the road.

For the starting price, price your product on the low side compared to your competitors’ prices (if you can find one).

It should be at least n times your COGS (cost of goods sold or supplier price). As a rule of thumb, n should be large enough to leave some room for profit when you factor in the operating cost.

Also, n should be inversely proportional to COGS (when COGS decreases, n increases). Here is a rough guideline:

n <= 2 if COGS >= $20,
n = 3 ~ 2 if $10 < COGS < $20, 
n >= 3 if COGS <= $10.

How many FB pixels do I need for my store?

If you are running a 1-product or niche store, 1.

If you are running a general store, 1 per major collection.

The reason is that once a pixel is optimized for a niche audience, it may not perform well if you try to market a very different product to the same audience using the pixel.

So, it’s better to deploy a new pixel to optimize for different types of products.

How much budget do I need for product testing?

There is no fixed figure. It depends on how big your wallet is.

The more money you put into product testing, the more time you allow FB to collect data and optimize the pixel, the more confident you are on whether a product is or is not a winner.

If you set a smaller budget, you won’t go broke quickly, but there is a higher chance you may miss a winner.

If you set a bigger budget, you burn your cash quickly, but there is less chance of missing something.

So it’s a trade-off.

Typically don’t set your ad set budget less than $10 per ad set per day per product. You can go as high as your wallet allows.

Should I use CBO (Campaign Budget Optimization) in product testing?

There are many reported success stories of using CBO in testing. However, there are also equally many reports that CBO takes a longer time to optimize.

There is also an inherent uncertainty on how CBO determines which ad set to spend money on.

In the cases where you have more than 2 winning ad sets, it’s possible that the CBO algorithm only focuses on the top 2 winners and neglects the rest.

To avoid these problems and simplify the testing process, I don’t advocate using CBO for testing. As far as product testing is concerned, use a daily budget on ad set level.

Show me an example of a campaign’s settings?

  • Campaign Objective = Conversions
  • Campaign Spending Limit = a desired limit, like $500
  • Campaign Budget Optimization = Off

Should I use Dynamic Creative / Automatic Placement?

No. What better way is there to burst your testing budget than letting Facebook try so many different combinations of the settings?

You can use these features at the scaling stage to find the most optimal ad automatically, provided that the traffic is already converting.

What Conversion Window should I set?

It depends on how strong a buying impulse the product can trigger.

For a unique, passionate product, set the conversion window to 1 day click or view.

For an item that takes a longer time to consider, set the conversion window to 7 days click or 1 day view.

If unsure, leave it at the default value.

How to set the ad set targeting?

You should have gotten some ideas about what countries and interests to target from your product research process. It follows that the product research process is where you learn about your target audience.

Do that well, and you can save much money in your product testing.

From your Ads Manager or Audience Insights tool, you can get suggestions on other related interests, e.g., a brand or an influencer in the product’s niche.

Generally, keep the audience size of your targeting in the range of 500k – 1mil so that you leave enough room for FB to find buyers but keep it small so that it doesn’t take too long for FB to optimize.

Here’s a list of targeting you can try:

  1. A big country + a broad interest.
  2. A big country + a narrow interest.
  3. A small country with no interest targeting (usually smaller EU or Middle-eastern & African country)
  4. If an audience size is too big, you can split it further into two, one with “Engaged Shoppers” behavior, one without.
  5. LAAs (Lookalike audiences) you have successfully used in the past.

It is crucial that you have minimal overlaps among the audience to prevent self-competition. You can use the Audiences tool to check for overlaps.

How many ad sets should I create?

You can create as many ad sets as your total capital allows, as long as they are related to the product. 5-10 is a typical number.

However, you should split your product testing into at least two separate rounds, with one starting after the other ends.

You can then analyze the results from the first round, apply what you learn and improve the ad set targeting and the ad you create in the second round.

Show me an example of an ad set’s settings?

  • Conversion Event Location = Website
  • Conversion Event = Purchase
  • Dynamic Creative = Off
  • Offer = Off
  • Daily Budget >= $10
  • Detailed targeting Expansion = Off
  • Placements = Manual Placements, Facebook and Instagram feeds.
  • Optimization for Ad Delivery = Conversions
  • Cost Control = None (lowest cost bid strategy)
  • Conversion Window = 1 day click or view

How do I create a decent ad for product testing?

If your product requires some demonstration for the audience to get it, create a video ad. Refer to the secrets behind highly converting product video ads.

You may want to engage a contractor to create and edit the video for you, or if you are skilled, you can do it yourself with software like filmora.

Otherwise, please keep it simple by creating an image ad. You can use canva or Photoshop to create an impactful image.

It is challenging to create an ad from scratch. You would need some references. Our product research process describes a way where you can find ads that promote a product.

After you’ve collected these ads, take some time to study how their ad messaging targets the audience’s relevant emotion. Emotion is what prompts the buyers to make a buying decision.

You would need to test at least 3 ad creatives to find one that clicks, so create 3 ads per each ad set.

Always shorten your landing page URL with the internal URL shortener Shopify provides. It looks neat and professional to the viewers.

Should I perform split testing (A/B testing) to determine the best ad creative?

Not during the product testing stage.

We should keep it simple by putting all ads under each ad set and monitoring them to kill the non-performing ones.

Because split testing generally requires a decent amount of purchases to tell which candidate performs better. Your A/B test could never reach the statistical significance level required because of the limited number of purchase events received throughout the testing campaign.

Should I run an engagement campaign to increase an ad’s engagement before testing it in a conversion campaign?

It is recommended if your budget allows it. Page post engagement does provide social proof and help with conversion.

Furthermore, your PPE campaign allows you to tell what kind of ad creative and landing page copy attracts your audience’s most attention.

How long should I run each ad set?

Facebook takes time to learn and optimize the performance of an ad. The ad metrics are likely to fluctuate during the learning phase.

A decision made based on fluctuating metrics could be a bad one. It follows that you should leave each ad set running if you do not have the confidence to rule it out as a potential winner.

That said, there is a trade-off between the confidence level you want and how long you can afford to allow the ad set to run.

Simply put, no one has the budget to run every ad set till it reaches every person in the audience, although that will give the marketer 100% confidence about how the ad performs.

So pick a threshold—a minimum duration to run an ad set.

We recommend the threshold to be 3 days at $10 per ad set per day.

In short, run your ad set for at least 3 days before you decide whether to kill it by any rules you set.

When you have more budget, you can adjust the threshold and let ad sets run longer to collect more data and learn more about the audience.

How do I read and interpret the results? If CPM is greater than <x>, what does that mean?

CPM: cost per 1000 impressions. It reflects the competition level in FB Ads.

As other advertisers also bid to show their ads to the same audience as yours. You pay more for ad placements if you target a popular audience, such as those with higher buying power.

So CPM depends on targeting, e.g., CPM is generally higher in developed countries than in developing countries.

High CPM could also be due to the seasonal effect, e.g., CPM is always higher during shopping seasons when the major brands are splurging on ads.

In general, CPM should not be the parameter you use to decide whether to kill an ad set because when your CPM increases, it shows FB is trying to show the ad to people with the strongest intent to buy.

CTR: click-through rate. It reflects the effectiveness of the ad messaging on the targeted audience.

The more effective an ad is, the more people click on it.

An optimized ad usually has a high CTR (>=3%), an unoptimized ad, low CTR (<3%). You can kill any ad with CTR < 1%.

How do I improve my ad?

If you have an unoptimized ad, you can check the following metrics to get some clues on what could be improved.

Quality Ranking, Engagement Rate Ranking, Conversion Rate Ranking: the metrics for diagnosing underperforming ads.

You can read this page: Ad Relevance Diagnostics to learn more about how to read and interpret their values.

CPC, CP-ATC, CP-PUR: cost per click, cost per add to cart, cost per purchase. These are the costs required to achieve each count of the stated result.

The key difference among these metrics is that CPC is solely about the ad’s performance, whereas CP-ATC and CP-PUR are about the performance of both the ad and the landing page since ATC and PUR are user actions on the landing page.

So it’s a mistake to use CP-ATC and CP-PUR to decide whether to kill an ad set while the landing page design is not yet optimized, such as during the early stage of a testing campaign.

In such a situation, we should use CPC as the leading indicator of whether the ad set performs well.

The last column of the table below shows the threshold you should apply to CPC while monitoring it. Kill an ad set if its CPC is greater than the threshold after 3 days of running.

selling pricemax CP-PUR (1/3 of selling price)max CPCCPC threshold to kill (max CPC + $0.50)
$20$6.67$0.33$0.80
$30$10$0.50$1
$40$13.33$0.67$1.20
$50$16.67$0.83$1.30
$60$20$1.00$1.50
table1: CPC thresholds for different selling prices

Explanation of how the table is computed

1. Our pricing formula sets the selling price to be about 2x to 3x of COGS.

It follows that to remain profitable, the maximum we can spend on FB Ads to land a sale (max CP-PUR) is approximately 1/3 of the selling price, taking into account that there may be other operating costs besides the ad cost.

2. CPC can be calculated using the formula

CPC = CP-PUR x conversion rate

where

conversion rate = number of buyers / total number of landing page visitors

We assume the landing page conversion eventually will reach 5% (the average conversion rate of a 1-product store). Based on that, we can work out max CPC from max CP-PUR.

3. The threshold is 50 cents above max CPC. That means if your ad is below the threshold and is still above max CPC, you need to put in extra effort at optimizing your ad creative and ad targeting.

You can kill any ad set with CPC above the threshold, as they may not be profitable in the long run.

How do I lower my ad’s CPC?

CPC is derived from CPM and CTR through this formula:

CPC = CPM / (1000 x CTR)

When CPC is too high, it implies either CTR is too low or CPM is too high.

If your CTR is too low, check the CTR section above for the corrective actions you can take.

If your CPM is too high, see below.

How do I lower my ad’s CPM?

You shouldn’t unless you are sure that high CPM is the sole reason behind your ad’s poor performance. For example, when you have an ad with a decent CTR, average ad relevance metrics but too high CPC, CP-ATC and CP-PUR.

To deal with high CPM, you have three options:

  • Monitor your campaign’s daily CPM to get a read on the competition level on Facebook, then schedule your ads to run on those times with lower CPM. However, this is often impractical.
  • Kill the ad set and test with a new audience in the next round.
  • Alternatively, you can increase your product price so that your CPC threshold increases and it’s less likely for you to make a loss.

I have an ad set with good CPC; what then?

If you have at least one surviving ad set with CPC within the threshold, look at CP-PUR next.

Check table 1, column 2. If

CP-PUR > max CP-PUR

you need to improve your landing page and the offer.

The actions you can take include

  • Improving the landing page speed, design, images, and copy
  • Tweaking the price
  • Sweetening the offer, such as giving out freebies.

While you do that, pay attention to two metrics from Google Analytics on your product page.

Avg. Time on Page: the average amount of time visitors spend on viewing your product page.

Bounce Rate: the % of product page sessions where there is no interaction with the page.

These two metrics indicate how well-received your product page is to its visitors. A healthy landing page has

Avg. Time on Page > 1min 30secs
Bounce Rate < 60%

You can use these two metrics to guide your landing page refinement.

I have tried everything, and CP-PUR is still above the threshold. What now? Should I kill the ad set?

No. CP-PUR alone does not give you the full picture. We need to look at another metric.

ROAS: the return on ad spend. The following formula defines it.

ROAS = AOV / CP-PUR

where AOV is the average order value.

If the buyer purchases only 1 piece of the product, then AOV = selling price. We have

ROAS = selling price / CP-PUR
min ROAS = selling price / max CP-PUR

From table 1,

selling price / max CP-PUR = 3

As a result,

min ROAS = 3

Or,

ROAS >= 3

We need to assume most buyers only purchase your main product during checkout, so always keep your ROAS to be at least 3 to remain profitable.

To declare a product is winning, you need an ad set with at least 10 sales and ROAS >= 3.

If you have done everything you can and your ROAS < 2.5, you can kill the ad set. When that’s the last ad set you have got, the product is a no-go.

If however your ROAS >= 2.5, you can try to improve your AOV.

You can do so by

  • offering volume pricing to incentivize the user to buy more quantities of the product,
  • providing cross-sell options or a value bundle to encourage the user to add other products into the purchase, or
  • showing an up-sell option to inspire the user to buy the higher-value product.

How do I increase my ad’s ROAS?

ROAS is a composite metric that reveals very little information about what went wrong under the hood.

You should look into specific metrics like CTR, CPM, CPC, and CP-PUR to better understand your ad. Please read up on the previous sections to learn more.

I have done everything right, but why no sales?

It is always one or more of these 5 factors that could go wrong:

  1. the ad
  2. the targeting
  3. the landing page
  4. the offer on the landing page
  5. the product choice

Please audit your sales funnel with the help of this guide. You may want a fresh pair of eyes to look into that.

How long does it take to find a winner?

It depends on how well you conducted your product research. A very experienced dropshipper can find a winner from testing 20 carefully selected products.

I have found a winner, but the ad died within <X> days.

It means the product does not have the demand as you initially thought, or it is a saturated winning product.

Facebook could find people who are the most likely buyers in the first few days of your campaign, giving you a false impression that you have landed a winner.

As the likely buyers dwindle out, your ad will turn unprofitable.

What do I do after I find a winner?

What’s next is called scaling.

In most cases, you want to duplicate the ad into a different CBO campaign for scaling.

Sometimes, when you have scored many sales with your product testing, you can create CAs (custom audiences) and LAAs right away for scaling while letting the winning ad set continues running in the testing campaign.

Rarely, you want to increase the winning ad set’s budget directly.

We will cover all these in a future article.

Jeff Wong

Jeff has spent 12 years architecting data analytics and offering consultation services for clients in the b2b, telco, healthcare and e-commerce industry. He likes to blog about the problems online businesses are facing nowadays and share creative and inexpensive solutions.

Leave a Reply