META MARKETING ANALYTICS PROFESSIONAL CERTIFICATE

Course 5: Marketing Analytics with Meta

Week 3: Running Facebook Experiments

Coursera Study Guide

Click to Enroll in Coursera Meta Marketing Analytics Professional Certificate

CONTENT

In week 3, you’ll learn how to run Conversion Lift and Brand Lift tests on Facebook Ads Manager. You’ll learn how to evaluate the results of these tests and how these results can inform the future of your campaign.

Learning Objectives

  • Evaluate the confidence for each test and recommend next steps
  • Interpret the results of Conversion Lift tests and Brand Lift tests
  • Choose the best approach to evaluating advertising effectiveness given a scenario
  • Evaluate advertising effectiveness with Brand Lift tests
  • Evaluate advertising effectiveness with Conversion Lift Tests

PRACTICE QUIZ: CONVERSION LIFT TESTS

1. A conversion lift test measures changes in conversion. What is the “lift” being measured?

  • The amount of conversions lost due to poor advertisements.
  • The amount of additional conversions in the treatment group vs. the control group. (CORRECT)
  • The projected sales if a specific ad campaign is used.

Correct: That’s right! The lift is the difference in conversions between the group that didn’t see your ad and the one that did.

2. True or false? If you want to run an experiment to test your advertising effectiveness, you have to plan it before you create and run your advertising campaign.

  • True (CORRECT)
  • False

Correct: Correct! Experiments are carefully planned ahead of time. That’s because they involve creating groups of subjects that receive different treatments.

3. When you use an experiment to test the effectiveness of your advertising campaign, what do we mean when we refer to the treatment or test group?

  • The group of people that did not see your ad
  • The total target audience for your ad
  • The group of people that definitely saw your ad
  • The group of people that may have been exposed to your ad (CORRECT)

Correct: Correct! These are the people who may have received “treatment”. Treatment here being exposure to your ad.

4. True or false? The ‘control group’ is the group in the experiment that will be exposed to the treatment.

  • True
  • False (CORRECT)

Correct: Correct! The ‘control group’ is the group in the experiment that is NOT exposed to the treatment.

5. You want to run an experiment for your company’s advertising campaign. When designing your experiment, you randomly assign people to a control group and a treatment group. Studies that use this type of experiment are referred to as:

  • Incremental Impact
  • Brand Survey Test
  • Holdout Test
  • Randomized Control Trials (RCTs) (CORRECT)

Correct: Correct! Randomization is the process of assigning people to the different groups by chance, or randomly.  By doing that, you distribute people with certain characteristics equally over the groups.

6. What does it mean when Facebook says it uses Intention to Treat (ITT) in advertising effectiveness tests?

  • It means that people are randomly assigned to the test or the control group
  • It means that people who are part of the test (or treatment) group in an experiment will remain in that group, even if they ended up not seeing the ad that was intended for them in the test (CORRECT)
  • It means that Facebook splits the audience evenly in a test and control group.
  • It means that Facebook will only count the results from people in the test group if they ended up seeing the ad that was intended for them. 

Correct: Correct! By using Intention to Treat, the effects of potential bias are minimized since all participants remain in the test group.

7. Imagine you ran a Holdout Test for your Facebook campaign and you find the following information in the report of the results:

  • Conversion lift percent: 15.2%
  • Conversion Lift:   2,155
  • Sales Lift: $88,100

Based on this information, which of the following is correct?

  • Your Facebook ads caused 2,155 additional conversions to occur that wouldn’t have happened otherwise (CORRECT)
  • Without your Facebook campaign, there would have been $88,100 in sales
  • There is 15.2% chance that your Facebook ads caused additional conversions
  • Sales among the people who did not have the opportunity to see your Facebook ads was $88,100

Correct: Correct! The conversion lift tells you how many additional conversions happened as a result of your ads. So in this case, the Facebook ads generated 2,155 additional conversions.

PRACTICE QUIZ: BRAND LIFT TESTS

1. What is brand lift?

  • An increase in the awareness of a brand (CORRECT)
  • A quick fix of your brand image
  • How far your brand is behind another

Correct: That’s right! Like conversion lift, the brand lift is the difference in awareness between a group of people who didn’t see your ad and those who did.

2. One of the most important parts of your brand lift test is:

  • The date range for your test
  • Specifying the name of your Facebook page
  • Creating your poll questions (CORRECT)
  • Specifying the region where your target audience lives

Correct: Correct! The first question is the standard ad recall poll. This question basically asks respondents whether they recall seeing your ad. Then, you can add up to two more questions. Once your questions are written, you can run your test.

3. How might a Brand Lift Test on the Facebook platform help an advertiser?

  • It shows the incremental lift in ad recall caused by a brand awareness focused campaign, thus helping to evaluate the effectiveness of that campaign (CORRECT)
  • It shows you what people think of your ad
  • It tells you how you could improve your advertising creative to make your campaign more effective
  • It helps evaluate the incremental lift in conversions resulting from your campaign.

Correct: Correct! The Brand Lift Test is the best way to measure how much brand awareness an ad campaign has generated.

4. True or false? Facebook suggests that in order to run a successful test, you should at least spend about $30,000 on advertising in the campaigns you are testing.

  • True (CORRECT)
  • False

Correct: Correct! In order to see a difference, you need to poll a large enough audience about whether or not they saw your ad. So, that means you have to have a certain budget to make the test work.

5. You run the marketing for a high end clothing retailer and recently run a brand lift test for your company’s Facebook ad campaigns. Your results consist of which of the following? (Select all that apply)

  • Cost per conversion
  • Brand lift (CORRECT)
  • Brand lift percentage (CORRECT)
  • Confidence level of results (CORRECT)

Correct: Correct! You also see the total brand lift or the estimated number of people that are likely to respond favorably to your brand as a result of your advertising.

Correct: Correct!The results summary shows you the increase in percentage of people who recalled your ad.

Correct: Correct! The test results also tell you what the confidence level is for your results. A confidence level of 90% or higher is good.

6. Imagine you ran a Brand Lift test for your campaign on Facebook and it is clear from the report that the confidence level for your test is low. In fact, it says in your report: “the chance that your Facebook ads caused brand lift of 63.2%”. You would like to see a higher confidence level. What could you do?

  • Keep running the campaign but reduce the daily budget so you can spread the test of more time.
  • Run a Holdout Test instead, which will most likely come back with a higher confidence level
  • Change the target audience for your campaign
  • Keep running the campaign and increase the test period and the ad budget (CORRECT)

Correct: Correct! If the confidence level for your test isn’t very high, you might want to try to run your test for a bit longer and increase your advertising budget to see a bigger effect.

7. What is an acceptable confidence level for your test?

  • Anything above 75%
  • Depends on the test
  • 90% and above (CORRECT)
  • Between 80-85%

Correct: Correct! A confidence level of 90% or higher is good. Ideally, you shouldn’t go below that. That means your test isn’t as strong as it could be.

GRADED QUIZ: RUNNING FACEBOOK EXPERIMENTS

1. The core of your Brand Lift Test is:

  • Specifying the name of your Facebook page
  • The date range for your test
  • Creating your poll questions (CORRECT)
  • Specifying the region where your target audience lives

Correct: Correct! The first question is the standard ad recall poll. This question basically asks respondents whether they recall seeing your ad. Then, you can add up to two more questions. Once your questions are written, you can run your test.

2. Imagine you run an online art supply store. You are looking to increase awareness of online purchases by 5 percentage points among the target audience within three months. Which test would best measure this goal?

  • Brand Lift Test (CORRECT)
  • Holdout Test

Correct: Correct! A Brand Lift Test measures the effect of your ads on ad recall and other brand related metrics measured through the use of a survey

3. Which of the following are elements of an experimental design?

  • Variable Group
  • Treatment Group (CORRECT)
  • Control Group (CORRECT)

Correct: Correct! We call the group that received the treatment in an experiment the treatment group.

Correct: Correct! The group that does not receive the treatment in an experiment is called the control group.

4. In a RCT, what does it mean to use intention to treat?

  • The test is repeated until treatment is completed on all participants.
  • All experiment participants are kept in the group they were originally assigned to, regardless of what they see or do. (CORRECT)
  • All participants are asked if they intend to purchase.

Correct: That’s correct! In an RCT, this minimizes the effects of potential bias.

5. True or false: to determine the lift ratio, you divide the incremental conversions by the number of conversions without ad exposure.

  • True (CORRECT)
  • False

Correct: That’s correct! Incremental conversions are an important part of determining a conversion campaign’s success.

6. True or false? If the confidence level for your test isn’t very high, stop running your test for a bit longer and decrease your advertising budget to see a bigger effect.

  • True
  • False (CORRECT)

Correct: Correct!. You should keep running the campaign and increase the test period and the ad budget.

7. Les would like to connect their external marketing data to Facebook. What is a tool that they could use to accomplish this?

  • The Facebook App
  • The Facebook Conversion API (CORRECT)
  • A Spreadsheet

Correct: That’s correct! The Facebook Conversion API is designed to create a direct and reliable connection between marketing data and Facebook.

8. Imagine you work in the marketing department of a large electronics company. You are planning to set up a new conversion campaign on Facebook focused on generating online sales for computer monitors. Your manager asks you to make sure you can report on the effectiveness of your campaign once the campaign is completed. How do you make sure you can fulfill her requirements?

  • By setting up a Brand Survey Campaign before the campaign starts running
  • By running a Holdout Test after the campaign is complete
  • By setting up a Holdout Test before the campaign starts running (CORRECT)
  • By running a Brand Survey test after the campaign is complete

Correct: Correct! A Holdout Test specifically measures the effect of your advertising on conversions. You must start it before the campaign begins.

9. Which of the following are not ways a Brand Lift Test on the Facebook platform may help an advertiser?

  • It shows the incremental lift in ad recall caused by a brand awareness focused campaign, thus helping to evaluate the effectiveness of that campaign
  • It tells you how you could improve your advertising creative to make your campaign more effective (CORRECT)
  • It shows you what people think of your ad (CORRECT)
  • It helps evaluate the incremental lift in conversions resulting from your campaign (CORRECT)

Correct: Correct! The Brand Lift Test is the best way to measure how much brand awareness an ad campaign has generated, this is not something measured by the test.

10. The group of people in an experiment that were not exposed to your ad is known as what?

  • The treatment group
  • The control group. (CORRECT)
  • The testers
  • The target audience

Correct: Correct! These are the people who did not receive “treatment”. Treatment here being exposure to your ad.

11. You want to run an experiment for your company’s advertising campaign. When designing your experiment, you randomly assign people to a control group and a treatment group. Studies that use this type of experiment are referred to as:

  • Incremental Impact
  • Brand Survey Test
  • Holdout Test
  • Randomized Control Trials (RCTs) (CORRECT)

Correct: Correct! Randomization is the process of assigning people to the different groups by chance, or randomly. By doing that, you distribute people with certain characteristics equally over the groups.

12. The difference between the treatment group and the control group that is likely to have been caused by the ad or campaign is known as what?

  • Holdout Exposure
  • Incremental Impact (CORRECT)
  • Intentional Treatment

Correct: That’s right! Incremental impact shows the difference in action between those who saw an ad and those who didn’t.

13. You’ve recently completed a brand lift test. Which of the following results will you receive? (Select all that apply)

  • Suggestions for better images and copy
  • Cost per view
  • Confidence level (CORRECT)
  • Cost per brand lift (CORRECT)

Correct: Correct! The test results also tell you what the confidence level is for your results. A confidence level of 90% or higher is good.

Correct: Correct! You also get a cost per brand lift, which reflects how much it costs you in advertising spend for each additional person who gives a desired response.

14. True or false: The Facebook Conversion API is used in place of the Facebook Pixel.

  • False (CORRECT)
  • True

Correct: That’s correct! The Facebook Conversion API and the Facebook Pixel can be used together.

15. You run the marketing for a high end clothing retailer and recently ran a holdout test for your company’s Facebook ad campaigns. In Facebook Ad Manager, you look at your overview and observe how many additional conversions happened as a result of your ad. This is called the:

  • Cost per brand lift
  • Purchase funnel
  • Sales lift
  • Conversion Lift (CORRECT)

Correct: Correct! The conversion lift tells you how your ad helped improve conversions. The conversion lift percentage also gives you the difference in conversions between the people who did and did not have the opportunity to see your ad during this test.

16. True or false: The Facebook Conversion API helps Facebook personalize and optimize your ads so that your ads are shown to people who are more likely to be interested in them.

  • False
  • True (CORRECT)

Correct: That’s correct! The Facebook Conversion API is designed to create a direct and reliable connection between marketing data and Facebook. This data is then used to improve ads on Facebook.

17. Imagine you work in the marketing department of a large electronics company. You are planning to set up a new conversion campaign on Facebook focused on generating online sales for computer monitors. Your manager asks you to make sure you can report on the effectiveness of your campaign once the campaign is completed. The first section of the Conversion Lift Test focuses on the test details. What are your options for what you can define for the test? (Select all that apply)

  • Competition
  • Duration (CORRECT)
  • Budget (CORRECT)
  • Audience (CORRECT)

Correct: Correct! This is an important aspect of the test that needs to be set before the test is conducted.

18. True or false: a randomized control test ensures that you have control over who is in the control group and who is in the treatment group.

  • True
  • False (CORRECT)

Correct: That’s correct! In an RCT, the process of grouping people in the control or treatment group is done randomly.

19. A campaign’s incremental impact is…

  • People assigned to the test group who may be exposed to a treatment.
  • The group for which the advertiser controls the ads they see.
  • Conversions or sales created by an ad campaign.
  • Any difference between the groups likely to have been caused by the ad or campaign. (CORRECT)

Correct: Correct! Well-designed Randomized Controlled Trials can show an ad campaign’s true impact. If there are significantly more conversions or sales from the test group than from the control group, we can confidently say that the advertising campaign worked and had a real impact.

20. Why does Facebook require spending at least $30,000 on a Brand Lift test?

  • In order to see a difference, you need to poll a large enough audience about whether or not they saw your ad. (CORRECT)
  • The servers required for this test are very expensive to maintain.
  • Each respondent is paid a small amount and you need at least 30,000 participants.

Correct: Correct! With this in mind, that means you have to have a certain budget to make the test work.

21. True or false? If you want to run an experiment to test your advertising effectiveness, you have to plan it before you create and run your advertising campaign.

  • False
  • True (CORRECT)

Correct: Correct! Experiments are carefully planned ahead of time. That’s because they involve creating groups of subjects that receive different treatments.

22. What is the final step in a conversion lift test?

  • Compare the actions in the test group vs the control group (CORRECT)
  • Check the conversions API
  • Reassign people from the control group to the test group
  • Design an ad for the test group and the control group

Correct: That’s right! The difference between the two groups is the conversion lift.

23. What is the conversion cycle?

  • How long you think it typically takes for people to convert (CORRECT)
  • The entire lifetime of a customer
  • The process of changing a viewer to a customer
  • When a former customer makes a second purchase

Correct: That’s right! Knowing how long this is an important part of planning your conversion lift test.

24. What is the question you ask to measure ad recall lift?

  • Do you recall seeing an ad for this brand online or on a mobile device in the last two days? (CORRECT)
  • If you had to pick a brand with the best ads, which would it be?
  • Do you recall any ad from this brand? What was it?
  • What is the last ad you recall seeing?

Correct: That’s right! This is a key metric in measuring brand lift.

25. Which of the following is NOT a good question to ask in a Brand Lift poll?

  • How likely are you to consider [X]?
  • Do you like brand [X]? (CORRECT)
  • How familiar are you with [X]?
  • Do you recall seeing an ad for [X]?

Correct: That’s right! This type of question doesn’t provide information specifically about brand lift.

26. True or false: If your test confidence level is below 90%, you should think about testing for a longer time, a higher budget, or both.

  • False
  • True (CORRECT)

Correct: That’s right! A confidence level below 90% isn’t ideal, but it could potentially be raised by spending more time and/or money on the test.

Previous – Week 2
Next – Week 4

Subscribe to our site

Get new content delivered directly to your inbox.

Quiztudy Top Courses

Popular in Coursera

Liking our content? Then, don’t forget to ad us to your BOOKMARKS so you can find us easily!

Subscribe