META MARKETING ANALYTICS PROFESSIONAL CERTIFICATE

Course 5: Marketing Analytics with Meta

Week 4: Optimizing Your Marketing Mix

Coursera Study Guide

Click to Enroll in Coursera Meta Marketing Analytics Professional Certificate

CONTENT

This week you will learn how to optimize your marketing by running effective A/B tests. You’ll also learn about Marketing Mix Modeling integrated with data you’ve gathered from Facebook Ads Manager and how it can be used to optimize the effectiveness of your marketing efforts.

Learning Objectives

  • Accurately integrate Facebook data in Marketing Mix Modeling
  • Explain how and when to apply Marketing Mix Modeling to optimize your marketing mix
  • Conduct an A/B test and evaluate the results
  • Design the test setup for an A/B test

PRACTICE QUIZ: OPTIMIZING ADS WITH A/B TESTS

1. True or false? When you are only testing two variations of one variable, you use an A/B test to determine which version will produce the best results.

  • False
  • True (CORRECT)

Correct: That’s right! A/B testing is the process of comparing two variants of an ad against each other to evaluate which one performs best.

2. In which of the following scenarios would an A/B test be helpful? (Select all that apply)

  • If you want to know whether your advertising was effective at increasing sales
  • If you want to optimize the copy (the text) in your ad (CORRECT)
  • If you would like to know which audience group your ad performs best in (CORRECT)
  • If you have two different images you could use in your ads and you want to know which image would generate most results (CORRECT)

Correct: That’s correct! By conducting an A/B test where you vary only the text in your ad you can understand whether a different version of the text works better and thus optimize the copy.

Correct: That’s correct! An A/B test can help you understand whether target audience A or B works better for your ad.

Correct: That’s correct! An A/B test can help you determine which image delivers best results.

3. In A/B testing in advertising, it’s best practice to only change one variable at a time between the version A and B of your ad. Why is that?

  • When you only vary one variable and leave all other aspects of the different versions of the ad identical, then you will know exactly which variable was responsible for the difference in results (CORRECT)
  • When you test only one variable at a time your chances of finding a winner are higher
  • It is easier to change only one variable at a time as it will reduce the time it takes to test
  • When you test only one variable at a time, the confidence level for your test will be higher

Correct: Correct! If you vary more than one variable, you won’t know which variable really made the difference.

4. The results of an A/B test on the Facebook platform include which of the following 2 elements?

  • The incremental lift of your ad compared to the control group and the winner of the test
  • The winner of the test (version A or version B) and the confidence level for the test (CORRECT)
  • The increase in brand awareness as a result of seeing your ad and the confidence level for the test
  • The number of results from version A and version B of the ad and the portion of the budget that was spent on each ad

Correct: That’s correct! Those are the two most important parts of the test results.

5. Which of the following is true when setting up an A/B test in Facebook?

  • There are two control groups
  • Facebook duplicates your ads and changes several variables
  • There is no control group (CORRECT)
  • The test group receives a treatment and the control group does not

Correct: That is correct! In an A/B test, your target audience will see either the A or the B version of your ad, but there is no control group (which would be a group that does not get to see any version of your ad).

6. What does the confidence level of an A/B test tell you?

  • It tells you how confident you can be that people were randomly assigned to the A and the B group
  • It tells you how confident you can be that your advertising was effective.
  • It tells you what the chance is of getting the same winner if you ran this test again. (CORRECT)
  • It tells you how likely it is that you can further improve your ad.

Correct: That is correct! In an A/B test, the confidence level gives you the percentage chance that you will get the same results if you run the same test again.

PRACTICE QUIZ: MARKETING MIX MODELING

1. True or false? You would run a marketing mix modeling study to show how your advertising on different platforms contributed to company sales.

  • T​rue (CORRECT)
  • False

Correct: Correct! A marketing mix modeling study is needed when you have advertising across multiple platforms, like TV, Radio, and Facebook.

2. Traditionally, what are the weaknesses associated with a marketing mix model? (Select all that apply.)

  • Inaccuracies
  • False Positives
  • Takes a long time (CORRECT)
  • Complex (CORRECT)

Correct: Correct! A marketing mix modeling study gathers data from multiple sources over a long period of time, making it both complex and time intensive. But, that also means the data is comprehensive and very valuable!

3. True or false: APIs and machine learning will eventually make marketing mix modeling irrelevant.

  • T​rue
  • False (CORRECT)

Correct: That’s right! These two innovations are helping to make marketing mix modeling simpler, making it more attainable and attractive for more marketers.

4. Imagine you work in the marketing department of a large clothing retailer. You want to run a marketing mix modeling study. Which of the following statements regarding the benefits of conducting this study are true? (Select all that apply)

  • Is solely used to determine on which platform your advertisement performed best.
  • It can help you analyze last year’s data to understand what worked. (CORRECT)
  • It can help plan how to best spend your money next year to improve results. (CORRECT)
  • It can predict what will work in the future for your company. (CORRECT)

Correct: Correct! When running a marketing mix modeling study, you’ll collect data related to any previous marketing actions and look at data on how much you spent and what results you got. 

Correct: Correct! After analyzing the data, you’ll have a report that tells you what the Return on Investment was for each one of the channels you used and how much each channel contributed to your final results.  This will help guide future marketing decisions.

Correct: Correct! Using information from the past, it’s possible to make educated guesses about what will happen in the future.

5. True or false: Likes, comments, and shares are great inputs to include in your marketing mix model.

  • T​rue
  • False (CORRECT)

Correct: That’s right! While this is good data to gather for other models and tests, for the purposes of a marketing mix model, it’s important to make sure your input data is comparable across channels. 

GRADED QUIZ: OPTIMIZING YOUR MARKETING MIX

1. True or false: In an advertising A/B test, it’s best practice to only change one variable at a time between versions A and B.

  • True (CORRECT)
  • False

Correct: Correct! If you vary more than one variable, you won’t know which variable really made the difference.

2. True or false: When you are only testing two variations of one variable, you use an A/B test to determine which version will produce the best results.

  • False
  • True (CORRECT)

Correct: Correct! A/B testing is the process of comparing two variants of an ad against each other to evaluate which one performs best.

3. You are working on a campaign with the objective to drive app downloads. You have created an audience on Facebook for your campaign. But, you also have a list of emails from people who have downloaded an app from you before, and you plan to create a custom audience using that list. Now, you would like to know which audience will be most likely to download your app. The new audience you created, or the custom audience of people who downloaded one of your apps before. Which study could help you?

  • You can run an A/B test (CORRECT)
  • You can use a Marketing Mix Modeling study
  • You can run a Brand Lift Test
  • You can run a Conversion Lift study

Correct: Correct! Because you are only changing one variable, in this case the audience, running an A/B test is the right way to go.

4. When setting up an A/B test in Facebook, what is the power of the test meant to tell you?

  • The likelihood that the results will be the same if the test is run more than once.
  • The likelihood that this test can drive more conversions.
  • The likelihood that this test can detect a difference in your ads if there is actually a difference to detect. (CORRECT)
  • The likelihood that this test will generate the desired results.

Correct: That’s right! Before you run your test, you’re given an indication of the estimated power of the test. Afterwards, you’re provided the confidence level of the results.

5. True or false: the confidence level you receive after running an A/B test on Facebook tells you the likelihood that the results will be the same if the test is run more than once.

  • True (CORRECT)
  • False

Correct: That’s right! A high confidence level means that running the test again should generate very similar results.

6. Jesse is not pleased with the confidence level they received on a report after running an A/B test. What should they do?

  • Start a new test, testing two different ads.
  • Give the campaign more time and resources to try to generate more impressions. (CORRECT)
  • Increase the number of variables being tested to increase the amount of data collected.

Correct: That’s right! More time and a higher budget allow for more observations in an A/B test making it easier to achieve a reliable result.

7. True or false: it’s a good idea to avoid running ad campaigns before and after an A/B test.

  • False
  • True (CORRECT)

Correct: That’s right! That way the results from either your A or B group won’t be influenced by other campaigns.

8. Jesse is attempting to analyze a campaign that ran on Facebook, YouTube, and TV. What type of analysis should they run?

  • Marketing Mix Modeling (CORRECT)
  • A/B Test Analysis
  • Digital Channel Analysis

Correct: That’s right! Marketing mix modeling takes data from different marketing channels and analyzes them together to get a more complete picture of all your advertising spending.

9. What are some of the developments in marketing mix modeling that may make this type of study cheaper to run in the future?

  • The use of pixels is making data collection faster and easier, reducing the price of MMM studies
  • Use of APIs for data collection and the use of machine learning for analysis. (CORRECT)
  • Agencies are getting better at running these models and the prices are dropping as a result.

Correct: That’s right! These innovations will make it faster and cheaper to run Marketing Mix Modeling studies in the future.

10. APIs and machine learning have many benefits in marketing mix modeling, including: (Choose all that apply)

  • More responses from participants
  • More granular insights (CORRECT)
  • Lower costs (CORRECT)
  • Faster turnaround time (CORRECT)

Correct: That’s right! This is one of the ways that innovation is helping marketing mix modeling improve.

11. Jesse is preparing an A/B test of an ad. They are planning on only changing a single thing between the two ad variants. Why is that?

  • When you test only one variable at a time, the confidence level for your test will be higher
  • It is easier to change only one variable at a time as it will reduce the time it takes to test
  • When you only vary one variable and leave all other aspects of the different versions of the ad identical, then you will know exactly which variable was responsible for the difference in results (CORRECT)
  • When you test only one variable at a time your chances of finding a winner are higher

Correct: Correct! If you vary more than one variable, you won’t know which variable really made the difference.

12. Imagine you are planning a campaign for a shoe brand on Instagram. You have several images of the shoes you want to promote, but you are not sure which image will work best. After a meeting with your team, you decide to run an A/B test. Version A includes an image of three shoes and a model with the caption “Put your best foot forward. Check out our summer collection.” There is a call to action button that says ‘order now’. Which of the following approaches follows best practice when developing your version B?

  • Make version B as different from version A as possible
  • Changing the caption and the image of the ad.
  • Changing the font and image of the ad.
  • Only vary the image creating version B. (CORRECT)

Correct: Correct! When you create your A and B version of what you are about to test, vary only one variable.

13. True or false: the power of a test tells you the likelihood that this test can find a difference in your ads if there is a difference to find.

  • False
  • True (CORRECT)

Correct: That’s right! Before you run your test, you’re given an indication of the estimated power of the test. Afterwards, you’re provided the confidence level of the results.

14. True or false: a good confidence level after an A/B test is at least 75%.

  • False (CORRECT)
  • True

Correct: That’s right! Ideally, your A/B test should have a confidence level of at least 90%.

15. After running an A/B test on Facebook, you find that you have a confidence level below 90%. What are some things you can do to raise the confidence level? (Choose all that apply)

  • Start with a new control group.
  • Test a different variable.
  • Run the test for a longer period of time. (CORRECT)
  • Increase the budget for your test. (CORRECT)

Correct: That’s right! This is one of the ways you can try to increase confidence in the results.

16. Jesse has a number of ads that are ready to launch leading up to and directly after an A/B test they’re planning on running. What should they do?

  • Hold off on running any tests before and after an A/B test. (CORRECT)
  • Run all the campaigns simultaneously to maximize the number of impressions.
  • Increase the number of ads being tested to increase the amount of information collected in the test.

Correct: That’s right! This is called the dark period before and after an A/B test. That way the results from either your A or B group won’t be influenced by other campaigns.

17. When including data from Facebook in a marketing mix model, what are some considerations? (Choose all that apply)

  • Facebook only offers a limited amount of data about a campaign
  • Include data that is comparable across channels (CORRECT)
  • Use a shorter time frame (CORRECT)
  • Put past campaign quality in context (CORRECT)

Correct: That’s right! Due to the nature of digital platforms this is an important consideration when working on a marketing mix model.

18. True or false: machine learning should be used sparingly with marketing mix modeling because it can drastically slow down analysis

  • True
  • False (CORRECT)

Correct: That’s right! Machine learning can help a marketing analyst iterate and improve a marketing mix model in real time.

19. Imagine you are planning a campaign for scented candles on Instagram. You have several images of the candles you want to promote, but you are not sure which image will work best. After a meeting with your team, you narrowed the choices down to two different images. One is an image of three candles on a dresser, the other one is an image of one candle only. You have already decided which copy will accompany the image. What should you do to help you decide on the right image to use?

  • Conduct an A/B test as part of the Instagram campaign in which you create two ads that are identical except for the image. (CORRECT)
  • Use a marketing mix modelling study to evaluate which images have worked best in the past
  • Conduct an A/B test in which you test a version A with the image of the three candles with a text that talks about the promotion and version B with the image of the single candle and copy that talks about the new scent of this candle
  • Conduct a Conversion Lift Test in which you check the effect of each of the images.

Correct: Correct! When you are only testing two variations of one variable, you use an A/B test to determine which version will produce the best results.

20. Jesse is running an A/B test on Facebook. They find that the estimated power of their test is 80%. Is this high enough to feel confident in the results?

  • No. (CORRECT)
  • Yes

Correct: That’s right! To have a good test, Jess should aim for a power of 90% or above. To increase the estimated power of a test, they can increase the budget or the time running the ads.

21. True or false: you can increase the confidence level received after a test by running the test for more time.

  • False
  • True (CORRECT)

Correct: That’s right! This is one of the ways you can try to increase confidence in the results. You could also increase the budget for your test.

22. Which of the following is a primary benefit of A/B testing different ads?

  • Finding what channel is most effective for delivering your ad.
  • Gauging brand awareness.
  • Finding the most optimal version of an ad. (CORRECT)

Correct: Correct! This type of detailed and specific testing is very useful for refining an ad.

23. True or false: marketing mix modeling is not useful for making predictions about future ad campaigns.

  • True
  • False (CORRECT)

Correct: That’s right! This is one of the strengths of marketing mix modeling. It can help you better understand your campaigns across channels and predict future outcomes.

24. After running an A/B test on Facebook, what is the confidence level of the test meant to tell you?

  • The likelihood that this test can find a difference in your ads if there is a difference to find.
  • The likelihood that the results will be the same if the test is run more than once. (CORRECT)
  • The likelihood that this test can drive more conversions.
  • The likelihood that this test will generate the desired results.

Correct: That’s right! A high confidence level means that running the test again should generate very similar results.

25. What are some of the things that marketing mix modeling can help with? (Choose all that apply)

  • Optimizing specific ads for different audiences.
  • Analyzing earlier data to discover what worked. (CORRECT)
  • Predicting what might work in the future. (CORRECT)
  • Planning future spending for improved results. (CORRECT)

Correct: That’s right! This is one of the ways marketing mix modeling can help you better understand your campaigns across channels.

26. True or false: all data is worth including in a marketing mix model even if there isn’t something to compare it to.

  • True
  • False (CORRECT)

Correct: That’s right! Some data like comments and shares aren’t good input for a marketing mix model.

27. What are some innovations that are helping to improve marketing mix modeling? (Choose all that apply)

  • More day to day social media usage. 
  • Streamlined consumer behaviors.
  • Machine learning. (CORRECT)
  • Manual data collection from APIs. (CORRECT)

Correct: That’s right! This is one of the ways that marketing mix modeling is improving and will continue to evolve with technology.

28. You are working on a campaign with the objective to drive app downloads. You have created an audience on Facebook for your campaign. But, you also have a list of emails from people who have downloaded an app from you before, and you plan to create a custom audience using that list. You also want to run ads on television and radio. Now, you would like to know which platform would be most effective for delivering your message. You have data from previous marketing campaigns and you work with a market research agency to help you with this. Which study could help you?

  • You can run a Marketing Mix Modeling study (CORRECT)
  • You can run a Brand Lift Test
  • You can run an A/B test
  • You can use a Conversion Lift study

Correct: Correct! This is a study that is often used by companies who are spending ad dollars across multiple channels. This gives them a solid high-level understanding of which channels work well for them.

29. When referring to an A/B Test, what is the dark period?

  • Time after your test where you don’t run ads.
  • Time before and after the test where you don’t run ads. (CORRECT)
  • Time during the test where you do not check the results.

Correct: That’s right! This is a good idea so the results from either your A or B group won’t be influenced by other campaigns.

30. You are working on a campaign with the objective to drive newsletter sign-ups. You have created an ad with copy that has previously tested well with a small focus group. But, you also have an ad with new copy written by the marketing team. Now, you would like to know which ad copy would be most effective for getting people signed up for your newsletter. Which study could help you?

  • You can run a Brand Lift Test
  • You can use a Conversion Lift study
  • You can run an A/B test (CORRECT)
  • You can run a Marketing Mix Modeling study

Correct: Correct! Because you are only changing one variable, in this case the old vs. new copy, running an A/B test is the right way to go.

31. There are different variables you can test in an A/B test. How many variables should you vary in one test?

  • Just one per test (CORRECT)
  • As many as you like
  • Two per test

Correct: That is correct!  By only varying one variable at a time, you can attribute the difference you see in results to the variable you changed.

32. What does the confidence level in an A/B test tell you?

  • The % of times the B version won over the A version in the test
  • The percent chance you will find the same winner if you ran the same test again. (CORRECT)
  •  The % difference in performance between the winner and the loser

Correct: That is correct. The confidence level tells you how confident you can be in your test and how likely you are to find the same results if you repeat the test with the same settings

33. In which situation would Marketing Mix Modeling be most beneficial?

  • For a large company who advertises on TV, the radio, YouTube, Facebook, and LinkedIn (CORRECT)
  • A small startup advertising their first product on Facebook and Instagram
  • A fashion company who only advertises on Instagram

Correct: Correct! A large company who has a large marketing budget and advertises on multiple media types would benefit from this type of analysis to determine where to allocate this budget.

Previous – Week 3
Next – Week 5

Subscribe to our site

Get new content delivered directly to your inbox.

Quiztudy Top Courses

Popular in Coursera

Liking our content? Then, don’t forget to ad us to your BOOKMARKS so you can find us easily!

Subscribe