META MARKETING ANALYTICS PROFESSIONAL CERTIFICATE

Course 4: Data Analytics Methods for Marketing

Week 3: Evaluating Advertising Effectiveness

Coursera Study Guide

Click to Enroll in Coursera Meta Marketing Analytics Professional Certificate

CONTENT

In week three, you’ll dig into using experiments to evaluate marketing effectiveness. You’ll also learn about A/B testing and how it can help you optimize your campaigns.

Learning Objectives

  • Describe the use of experiments to evaluate advertising effectiveness
  • Evaluate the results of an experiment and determine next steps
  • Assess the confidence of an experiment
  • Describe A/B testing and its application
  • Use observational methods to evaluate advertising effectiveness and describe the shortcomings

PRACTICE QUIZ: AD EFFECTIVENESS MEASUREMENT

1. SMART Goals are:

  • Systematic Marketing and Realty Time-Lines
  • Specific Measurable Achievable Realistic Time Bound (CORRECT)
  • Small Manageable Accurate Real Terrific
  • Simple Meaningful Accurate Relevant Targeted

Correct: Exactly! These are the attributes that make a good goal.

2. KPIs can help because they are specific metrics that can be used to measure progress. True or False?

  • False
  • True (CORRECT)

Correct: Exactly! KPIs are important because they allow you to measure progress towards your goal.

3. The two methods you can use to evaluate ad effectiveness are:

  • Observational methods and KPI evaluation
  • SMART Goal setting and Experimental methods
  • KPI evaluation and SMART Goal setting
  • Experimental and Observational Methods (CORRECT)

Correct: Exactly! Experimental and Observational are the two methods used to approach evaluating ad effectiveness.

4. If you are running an ad with a “click to subscribe” button, can you measure the effectiveness of the ad ONLY by counting the number of customers who subscribed by clicking the button?

  • Yes
  • No (CORRECT)

Correct: Exactly! This approach would underestimate the effectiveness of the ad, because some people would subscribe through other means after seeing this ad.

PRACTICE QUIZ: MEASURING AD EFFECTIVENESS WITH EXPERIMENTS

1. Results of an experiment include: (select all that apply)

  • A winner
  • Report output
  • Measure of confidence in the results (CORRECT)
  • Number of incremental results caused by your treatment (CORRECT)

Correct: Exactly! This is part of the results of an experiment.

2. What could the effect be of using ITT (Intention to Treat) on the incremental impact of your campaign results?

  • The incremental impact you observe may be lower (CORRECT)
  • Result remains the same

Correct: Correct! If you use intention to treat, it means that people who end up not being exposed to your ad still remain in your treatment group. So, on average, the incremental impact of your campaign may be lower as the people who did not see your ad are probably less likely to take action.

3. Randomization is the process of assigning people to groups based on a common trait that is selected at random. True or False?

  • True
  • False (CORRECT)

Correct: Exactly! If you followed this process, you would end up with similar people in each group, which could bias your results.

4. In an experiment, the group that did not receive a treatment is called what?

  • Null group
  • Treatment group
  • Control group (CORRECT)
  • Other

Correct: Exactly! The group that did not receive a treatment is called the control group.

5. A hypothesis is a proposed explanation for an observation. True or False?

  • False
  • True (CORRECT)

Correct: Exactly! This is the definition of a hypothesis.

GRADED QUIZ: EVALUATING ADVERTISING EFFECTIVENESS

1. SMART Goals are Not: (select all that apply)

  • Time Bound
  • Measurable
  • Ambiguous (CORRECT)
  • Relatable (CORRECT)

Correct: Exactly! This is not one of the attributes that make a good goal.

2. KPIs are… (select all that apply)

  • Useful only at the corporate level
  • Measurable (CORRECT)
  • Helpful in evaluating advertising effectiveness (CORRECT)
  • Specific Metrics (CORRECT)

Correct: Exactly! This is an attribute of KPIs.

3. Which of the following methods can be used for evaluating ad effectiveness? (Select all that apply)

  • SMART Goal setting
  • KPI evaluation
  • Experimental methods (CORRECT)
  • Observational methods (CORRECT)

Correct: Exactly! This is one of the methods for evaluating ad effectiveness.

4. SMART Goals are: (select all that apply)

  • Relatable
  • Specific (CORRECT)
  • Time Bound (CORRECT)
  • Measurable (CORRECT)

Correct: Exactly! This is one of the attributes that make a good goal.

5. KPIs have no practical use. True or False?

  • False (CORRECT)
  • True

Correct: Exactly! KPIs can help you measure progress or compare performance.

6. If you are running an ad with a “click to subscribe” button, and you measure the effectiveness of the ad ONLY by counting the number of customers who subscribed by clicking the button, you would be overestimating the results. True or False?

  • False (CORRECT)
  • True

Correct: Exactly! This approach would underestimate the effectiveness of the ad, because some people would subscribe through other means after seeing this ad.

7. Results of an experiment do NOT ALWAYS include: (Select all that apply)

  • Number of incremental results caused by your treatment
  • Measure of confidence in the results
  • Report output (CORRECT)
  • A winner (CORRECT)

Correct: Exactly! This is not part of the results of an experiment.

Correct: Exactly! This is not necessarily part of the results of an experiment.

8. Using ITT (Intention to Treat) in an experiment to measure the incremental impact of your campaign would not change the results. True or False?

  • True
  • False (CORRECT)

Correct: Correct! If you use intention to treat, it means that people who end up not being exposed to your ad still remain in your treatment group. So, on average, the incremental impact of your campaign may be lower as the people who did not see your ad are probably less likely to take action.

9. What does RCT stand for?

  • Randomized Controlled Trials (CORRECT)
  • Randomized Controlled Tests
  • Randomized Cautious Tests
  • Really Controlled Trials

Correct: Exactly! RCT stands for Randomized Controlled Trials.

10. In an experiment, the group that did receive a treatment is called what?

  • Null group
  • Control group
  • Treatment group (CORRECT)
  • Other

Correct: Exactly! The group that did receive a treatment is called the treatment group.

11. AB Testing does not use a control group. True or False?

  • True (CORRECT)
  • False

Correct: Exactly! AB tests have no control group.

12. Multi-Cell, Nested Cell, and Structure Cell are all types of test structures. True or False?

  • True
  • False (CORRECT)

Correct: Exactly! Structured Cell is not a type of test.

13. SMART Goals are:

  • Specific, Measurable, Achievable, Realistic, Time Bound (CORRECT)
  • Systematic, Marketing And Realty, Time-Lines
  • Simple, Meaningful, Accurate, Relevant, Targeted
  • Small, Manageable, Accurate, Real, Terrific

Correct: Exactly! These are the attributes that make a good goal.

14. What are two methods for evaluating ad effectiveness?

  • SMART Goal setting and Experimental methods
  • Observational methods and KPI setting
  • KPI evaluation and SMART Goal setting
  • Experimental and Observational Methods (CORRECT)

Correct: Exactly! Experimental and Observational are the two methods used to approach evaluating ad effectiveness.

15. If you are running an ad with a “click to subscribe” button, and you measure the effectiveness of the ad ONLY by counting the number of customers who subscribed by clicking the button, what would you expect the results to be?

  • There is not enough information to tell
  • Overestimated
  • Correct
  • Underestimated (CORRECT)

Correct: Exactly! This approach would underestimate the effectiveness of the ad.

16. The effect of using ITT (Intention to Treat) on the incremental impact of your campaign results in the incremental impact you observe being lower. True or False?

  • True (CORRECT)
  • False

Correct: Correct! If you use intention to treat, it means that people who end up not being exposed to your ad still remain in your treatment group. So, on average, the incremental impact of your campaign may be lower as the people who did not see your ad are probably less likely to take action.

17. AB Testing is not a good test to evaluate the effectiveness of your campaign, because…

  • It has no p-value
  • It cannot track data over time
  • AB Testing IS a good way to evaluate the effectiveness of your campaign.
  • There is no control group. (CORRECT)

Correct: Exactly! AB Testing has no control group.

18. Which of the following are NOT types of test structures?

  • Multi-Cell
  • Nested Cell
  • Structured Cell (CORRECT)
  • Single Cell

Correct: Exactly! This is not a type of test.

19. AB Testing is a good test to evaluate the effectiveness of your campaign. True or False?

  • False (CORRECT)
  • True

Correct: Exactly! AB Testing has many uses, but this is not one of them.

20. What do results of an experiment include? (Select all that apply)

  • Report output
  • A winner
  • Measure of confidence in the results (CORRECT)
  • Number of incremental results caused by your treatment (CORRECT)

Correct: Exactly! This is part of the results of an experiment.

21. Randomization does not impact the results of a trial. True or False?

  • True
  • False (CORRECT)

Correct: Exactly! Randomization plays a major role in the results of a trial.

22. In an experiment, the group that did not receive a treatment is called what?

  • Control group (CORRECT)
  • Treatment group
  • Other
  • Null group

Correct: Exactly! The group that did not receive a treatment is called the control group.

23. Results of an experiment include the number of incremental results caused by your treatment and the measure of confidence in the results. True or False?

  • False
  • True (CORRECT)

Correct: Exactly! These are the results of an experiment.

24. Experimental and Observational methods can both be used for evaluating ad effectiveness. True or False?

  • True (CORRECT)
  • False

Correct: Exactly! Experimental and Observational are both methods used to approach evaluating ad effectiveness.

25. Which of the following are types of test structures? (Select all that apply)

  • Structured Cell
  • Single Cell (CORRECT)
  • Multi-Cell (CORRECT)
  • Nested Cell (CORRECT)

Correct: Exactly! This is a type of test.

26. You are running an ad with a ‘click to purchase’ button.  Can you measure the effectiveness of the ad by counting the number of purchases that came from people clicking on the ad?

  • No, that would underestimate the effectiveness of the ad (CORRECT)
  • Yes, that would measure how many purchases the ad was responsible for.

Correct: That is correct, there may be people who saw the ad and later purchased the website without clicking on the ad.

27. Why are experimental methods better to measure advertising effectiveness than observational methods?

  • In experimental methods you have more control over how similar the different groups are that will be compared to evaluate results (CORRECT)
  • You need less budget for experimental methods.
  • Both methods don’t really differ.

Correct: that is correct!  Because you can carefully plan who does and does not see an ad, you have more control over how similar the groups are that you will be comparing. 

28. What could the effect be of using ITT (Intention to Treat) on the incremental impact of your campaign results?

  • Result remains the same
  • The incremental impact you observe may be lower. (CORRECT)

Correct: Correct!  If you use intention to treat, it means that people who end up not being exposed to your ad still remain in your treatment group. So, on average, the incremental impact of your campaign may be lower as the people who did not see your ad are probably less likely to take action.

29. There are two parts to experiment results, the number of incremental results from your treatment and what else?

  • A winner
  • A measure of confidence in the results (CORRECT)
  • A report output

Correct: Nice work! A measure of confidence is the second part of the results you get from an experiment.

Previous – Week 2
Next – Week 4

Subscribe to our site

Get new content delivered directly to your inbox.

Quiztudy Top Courses

Popular in Coursera

Liking our content? Then, don’t forget to ad us to your BOOKMARKS so you can find us easily!

Subscribe