META MARKETING ANALYTICS PROFESSIONAL CERTIFICATE

Course 3: Statistics for Marketing

Week 3: Designing Experiments and Testing Hypotheses

Coursera Study Guide

Click to Enroll in Coursera Meta Marketing Analytics Professional Certificate

CONTENT

In week three, you’ll dig into how to formulate and test appropriate hypotheses for your business goals. You’ll wrap up the week with part three of your capstone project.

Learning Objectives

  • Identify variables to test
  • Understand the Null Hypothesis, P-Values, and their role in testing hypotheses
  • Define the two types of statistical error
  • Formulate a hypothesis and align hypotheses with business goals
  • Identify actions based on hypothesis validation/invalidation
  • Differentiate observational methods and experiments
  • Perform a t-test to judge hypothesis validation/invalidation

PRACTICE QUIZ: EXPERIMENTAL DESIGN AND HYPOTHESES

1. In the context of this course, what is an evaluation question?

  • A question that motivates a study or experiment. (CORRECT)
  • A question about the accuracy of an experiment.
  • A question about the results of an experiment.

Correct: Good job! As discussed in the video lectures, all studies start with a question that needs to be answered. This is what we call the evaluation question.

2. What are possible sources for evaluation questions?

  • I.   Data
  • II.  Stakeholders
  • III. Managers
  • I
  • II
  • All three. (CORRECT)
  • III

Correct: Good job! Each one of these may be a source for evaluation questions.

3. Which of the following is/are parts of a hypothesis?

  • I.   What will change?
  • II.  How will it change?
  • III. What will cause the change?
  • I. and II.
  • I. and III.
  • All three are the parts of a hypothesis. (CORRECT)
  • II. and III.

Correct: Good job! Indeed, a hypothesis comes in these three parts. It is a tentative answer to the evaluation question.

4. What is a hypothesis?

  • It is the result of a statistical analysis.
  • It is another name for a theory.
  • It is a question that motivates a study.
  • It is a tentative answer to the evaluation question. (CORRECT)

Correct: Good job! A hypothesis is a tentative answer to the evaluation question.

5. What are the two types of studies used to test a hypothesis?

  • Experimental and actual.
  • Observational and experimental. (CORRECT)
  • Observational and simple.
  • Data driven and actual.

Correct: Good job! Observational studies involve observing what is happening passively. Experimental studies, on the other hand, involve directly affecting a process and observing the results.

6. Which of these is an example of an observational study?

  • Data mining. (CORRECT)
  • Repeated measures.
  • Random trials.
  • A/B test.

Correct: Good job! Data mining is an example of an observational study.

7. True or false: A/B testing is an example of an experimental study.

  • False
  • True (CORRECT)

Correct: Good job! A/B testing is an experimental study in which the subject population is divided into two groups, each of which is subjected to a different stimulus and their reactions observed.

8. What are the five steps in experimental design?

  • Question, hypothesis, measurement, solving, and analysis.
  • Question, hypothesis, define variables, measurement, and analysis. (CORRECT)
  • Question, theory, define variables, measurement, and analysis.
  • Prediction, question, hypothesis, define variables, and analysis.

Correct: Good job! These are the five steps in experimental design.

9. Which of the following is not a part of a hypothesis?

  • I.   What will change?
  • II.  How fast is it changing?
  • III. What will cause the change?
  • All of these.
  • II (CORRECT)
  • III
  • I

Correct: Good job! “How fast is it changing?” is frequently an important question. However, it is not a part of a hypothesis.

10. True or false: Simple surveying is an example of an experimental study.

  • False (CORRECT)
  • True

Correct: Good job! Simple surveying is not an example of an experimental study. Rather, it is an example of an observational study.

PRACTICE QUIZ: HYPOTHESIS AND AB TESTING

1. What is AB testing?

  • A test to determine if the analysis has bias.
  • A study that compares two unrelated things to see which performs better.
  • A test to determine the stakeholders of an analysis.
  • A study that compares two different versions of something to see which performs better. (CORRECT)

Correct: Good job! When comparing the effects of two different versions of something, you are performing an AB test.

2. When performing AB testing, the alternate hypothesis H1 makes what claim?

  • That there is no significant difference in the effect between A and B.
  • That A and B perform the same or similar.
  • That A performs better than B.
  • That there is a significant difference in the effect between A and B. (CORRECT)

Correct: Good job! The alternate hypothesis is the claim that there is a significant difference between two versions of the same thing.

3. In general, what do you conclude for a p-value where p > 0.05?

  • Accept H0, accept H1.
  • Reject H0, accept H1. (CORRECT)
  • Accept H0, reject H1.
  • Reject H0, reject H1.

Correct: Good job! In general, it is commonly accepted practice to reject the null hypothesis if the p-value is less than or equal to 0.05.

4. What does a 95% confidence interval represent?

  • The chance of making a type 1 error.
  • The range of values within which you may be 5% sure that the true mean falls.
  • The chance of making a type 2 error.
  • The range of values within which you may be 95% sure that the true mean falls. (CORRECT)

Correct: Good job! While the most common interval is 95%, other percent intervals can be used.

5. Which common hypothesis tests were mentioned in the video?

  • I. t-test
  • II. ANOVA
  • III. Chi-squared
  • All of these. (CORRECT)
  • II.
  • III.
  • I.

Correct: Good job! All three of these tests were mentioned in the video, but only the t-test was discussed in more detail.

6. True or false: The correct syntax for using a t-test in a spreadsheet is:

=ttest(Group1, Group2, Tails, Type)

  • False.
  • True. (CORRECT)

Correct: Good job! This formula takes four different parameters each corresponding to a different object in the parentheses, namely Group1, Group2, Tails, and Type.

7. When performing AB testing, the null hypothesis H0 makes what claim?

  • That there is a difference in the effect between A and B.
  • That B performs better than A.
  • That A performs better than B.
  • That there is no difference in the effect between A and B. (CORRECT)

Correct: Good job! The null hypothesis is the claim that two versions of the same thing perform with the same effectiveness.

8. True or false: The correct syntax for using a t-test in a spreadsheet is:

=ttest(Group, Tails, Type)

  • False. (CORRECT)
  • True.

Correct: Good job! The t-test requires two groups to be performed and this formula only indicates one group.

GRADED QUIZ: EXPERIMENTAL DESIGN AND TESTING

1. What is a type 2 error?

  • When both the null and alternate hypotheses are rejected.
  • When the null is falsely accepted. (CORRECT)
  • When the null hypothesis is falsely rejected.
  • When the alternate hypothesis is falsely accepted.

Correct: Good job! A type 2 error is also known as a false negative, and is when the null is falsely accepted.

2. Which of the following is not a step in experimental design?

  • Present to stakeholders (CORRECT)
  • Hypothesis
  • Select an analysis
  • Question

Correct: Good job! Presenting to stakeholders is going to happen, but it happens after the experimental study, not as a part of it.

3. Suppose that a car company wants to know if a new fuel additive increases the fuel efficiency of their vehicles. They undertake an AB test to investigate. What is the null hypothesis in this test?

  • That the additive decreases fuel efficiency significantly.
  • That the fuel additive does not affect the fuel efficiency significantly. (CORRECT)
  • That the additive is too expensive.
  • That the additive increases the fuel efficiency significantly.

Correct: Good job! The null hypothesis is the claim that there is no significant effect due to the additive.

4.

Alpha = 0.05

P-value = 0.04

With the information above, what conclusions can you draw?

  • There is no significant difference. Reject H0, reject H1.
  • There is a significant difference. Reject H0, accept H1. (CORRECT)
  • There is a significant difference. Accept H0, accept H1.
  • There is no significant difference. Accept H0, reject H1.

Correct: Good job! When the p-value from your study is less than or equal to the alpha level, you should accept the alternate hypothesis.

5. Repeated measures, AB testing, and randomized control trials are all studies of what type?

  • Experimental (CORRECT)
  • Observational
  • Statistics
  • Bias

Correct: Good job! All of these are examples of experimental studies since they require an analyst’s influence to be run correctly.

6. What is a statistical assumption?

  • Something that you know about the data.
  • An assumption about the target audience of the analysis.
  • Something that must be true for the analysis to be correct. (CORRECT)
  • An assumption about the outcome of an analysis.

Correct: Good job! Being aware of the assumptions made in a statistical analysis can help you to avoid faulty analysis.

7. What is the alpha for a 90% confidence interval?

  • 0.05
  • 90%
  • 0.10 (CORRECT)
  • 0.90

Correct: Good job! The alpha is the percent difference between the confidence interval and 100% in decimal form.

8. What is confirmation bias?

  • When one hypothesis is more reasonable than any others.
  • When you confirm a hypothesis that is not true.
  • When a study falsely confirms that the sample is representative of the population.
  • When an analyst specifically looks for data or other information to prove a hypothesis correct. (CORRECT)

Correct: Good job! Confirmation bias will always be a potential source of bias in a study.  Moreover, it is very difficult to see when it is affecting the validity of a study. Best practices in analysis always assume that in any study, confirmation bias is possible.

9. What is a type 1 error?

  • When the null hypothesis is falsely rejected in favor of the alternate hypothesis. (CORRECT)
  • When the alternate hypothesis is falsely rejected.
  • When both the null and the alternate hypothesis are rejected.
  • When the null hypothesis is falsely accepted.

Correct: Good job! A type 1 error is commonly called a false positive, since H1, being considered the positive result, is falsely taken to be true.

10. In experimental design, what is the order of the five steps?

  • 1) Hypothesis; 2) Required variables; 3) Question; 4) Select analysis; 5) Measurement
  • 1) Question; 2) Hypothesis; 3) Required variables; 4) Select analysis; 5) Measurement
  • 1) Question; 2) Hypothesis; 3) Required variables; 4) Measurement; 5) Select analysis (CORRECT)
  • 1) Question; 2) Required variables; 3) Hypothesis; 4) Select analysis; 5) Measurement

Correct: Good job! These are the correct five steps of experimental design in the correct order.

11. Which of these is not a statistical assumption?

  • Linearity
  • Bias (CORRECT)
  • Required sample size
  • Normality

Correct: Good job! Bias is not a statistical assumption. However, not accounting for statistical assumptions can lead to bias in an analysis.

12. When in doubt, what confidence interval and alpha should you use?

  • Confidence Interval: 95%; Alpha: 0.05 (CORRECT)
  • Confidence Interval: 90%; Alpha: 0.10
  • Confidence Interval: 99%; Alpha: 0.05
  • It doesn’t matter, but alpha should always be 0.05.

Correct: Good job! When in doubt, you should use a 95% confidence interval and an alpha of 0.05.

13. Which of the following are steps in experimental design?

  • I. Measurement
  • II. Hypothesis
  • III. Communicate
  • I. and II. (CORRECT)
  • II. and III.
  • I. and III.
  • All of these.

Correct: Good job! Measurement and hypothesis are both steps in experimental design.

14. A retail store is considering increasing their current discount on a product to see if it will sell more. They decide to perform an AB test. What is the alternate hypothesis?

  • That the greater discount leads to significantly more sales of the item. (CORRECT)
  • That the greater discount leads to no significant change in sales of the item.
  • That the item is popular.
  • That the item is underpriced.

Correct: Good job! We would expect that an increase in the discount will make the item more attractive to buyers and thus increase its sales. This is the alternate hypothesis.

15. What are five common types of bias?

  • Survey, Culture, Confirmation, Observation, and Selection (CORRECT)
  • Survey, Culture, Confirmation, Observation, and Unfair
  • Survey, Culture, Support, Observation, and Selection
  • Survey, Recording, Confirmation, Experimental, and Selection

Correct: Good job! These five types of bias are the most commonly encountered. There is almost always one that you will have to worry about, but often more than one of these can be present in a study.

16. A food wholesaler is looking to reduce its costs by shipping to its customers three days a week rather than five. However, the company believes that this may cause it to lose some customers. It decides to use an AB test to test their hypothesis. What is the null hypothesis of this test?

  • That shipping five days a week is a bad idea.
  • That shipping fewer days does cause a significant loss of customers.
  • That shipping three days a week is a bad idea.
  • That shipping fewer days does not cause a significant loss of customers. (CORRECT)

Correct: Good job! The null hypothesis is the claim that changing the shipping schedule will not result in a significant loss of customers.

17. What is observational bias?

  • When analysts don’t observe the differences in culture between themselves and the sample.
  • That observing people can cause them to act in a manner inconsistent with their true actions. (CORRECT)
  • When you can’t see what is happening.
  • When you select a sample by observing the population incorrectly.

Correct: Good job! Observational bias is very common when conducting studies on humans. Humans, when being watched, will have the tendency to act in ways they think would be acceptable by the observers, rather than how they would truly behave if they were not being watched.

18. What type of error falsely rejects the null hypothesis in favor of the alternate hypothesis?

  • Type AB error
  • Type 2 error
  • Type 1 error (CORRECT)
  • Analytic error

Correct: Good job! A type 1 error is when you accept the alternate hypothesis, also known as the positive hypothesis, when you shouldn’t.

19. Simple surveys, counting data, and data mining are all studies of what type?

  • Bias
  • Statistics
  • Observational (CORRECT)
  • Experimental

Correct: Good job! Observational studies consist of observing what is happening without directly influencing the outcome.

20. Assuming that the data sample is large enough for the analysis is an example of what kind of statistical assumption?

  • Required sample size (CORRECT)
  • Normality
  • Homogeneity of variance
  • Linearity

Correct: Good job! Required sample size is the assumption that the size of the sample is large enough for the sample to be representative of the population.

21. 

Alpha = 0.05

P-value = 0.08

With the information above, what conclusions can you draw regarding H0 and H1 in a hypothesis test?

  • There is a significant difference. Reject H0, accept H1.
  • There is no significant difference. Reject H0, reject H1.
  • There is no significant difference. Accept H0, reject H1. (CORRECT)
  • There is a significant difference. Accept H0, accept H1.

Correct: Good job! When the p-value from your study is greater than the alpha level, you should accept the null hypothesis.

22. How do observational studies differ from experimental studies?

  • In an observational study, the results of the study are watched in person, while for an experimental study, they are recorded on video.
  • In an experimental study, one does not influence the progress of the study. In an observational study, one directly influences the study and watches the effects of this influence.
  • They are not different.
  • In an observational study, one does not influence the progress of the study. In an experimental study, one directly influences the study and watches the effects. (CORRECT)

Correct: Good job! In an observational study the parties running the study do not interfere in the progress of the study. In an experimental study, this is not true.

23. What is the alpha for a 96% confidence interval?

  • 96%
  • 0.04 (CORRECT)
  • 0.96
  • 0.05

Correct: Good job! The alpha is the percent difference between the confidence interval and 100% in decimal form.

24.

Alpha = 0.06

P-value = 0.05

With the information above, what conclusions can you draw?

  • There is no significant difference. Accept H0, reject H1.
  • There is a significant difference. Accept H0, accept H1.
  • There is a significant difference. Reject H0, accept H1. (CORRECT)
  • There is no significant difference. Reject H0, reject H1.

Correct: Good job! When the p-value from your study is less than or equal to the alpha level, you should reject the null hypothesis.

25. What is the alpha for a 90% confidence interval?

  • 0.90
  • 0.05
  • 0.10 (CORRECT)
  • 90%

Correct: Good job! The alpha is the percent difference between the confidence interval and 100% in decimal form.

26. What kind of study is this?

  • Experimental Study (CORRECT)
  • Observational Study

Correct: Exactly! You are changing the color of the website to see if it influences the outcome. That makes this an experimental study.

27. Which of the following is not a part of Experimental Design?

  • Run the Study (CORRECT)
  • Question
  • Select an Analysis
  • Required Variables

Correct: Exactly! You can’t run the study until after the entire Experimental Design process is completed.

28. Does this mean that your hypothesis was wrong or correct?

  • Correct
  • Wrong (CORRECT)

Correct: Exactly! If there is no difference between the two subject lines, that means that the new subject line did not perform better than the old subject line.

29. A company is comparing the results of two different ad campaigns.  The results of their analysis are as follows:

Alpha = 0.05
P-Value = 0.04

With the information above, what conclusions can you draw?

  • There is NO difference. Accept the Null Hypothesis and reject the Alternative Hypothesis.
  • There is a significant difference! Reject the Null Hypothesis and accept the Alternative Hypothesis (CORRECT)

Correct: Exactly! The P-Value is smaller than the Alpha, so there is a significant difference

30. The Confidence Interval is influenced by Variance. True or False?

  • True (CORRECT)
  • False

Correct: Exactly! Standard Deviation, a measure of Variance, is part of the equation for Confidence Interval.

31. Using the default alpha of 0.05, p-value of 0.04 means…

  • There is a significant difference between the two versions. (CORRECT)
  • There is not a significant difference between the two versions.

Correct: Exactly! Since 0.04 is less than 0.05, there is a significant difference.

32. Is the above survey free of bias?

  • No (CORRECT)
  • Yes

Correct: Exactly! All three of these questions show signs of survey bias.

33. You interpret the results of your analysis to mean that there was no difference between the colors of tulips, but that isn’t correct. What type of error is this?

  • Type II Error (CORRECT)
  • Type I Error

Correct: Exactly! You said there was no difference when there was one.

34. Which of the following is not a common statistical assumption?

  • Linearity
  • Heterogeneity of Variance (CORRECT)
  • Minimum Sample Size
  • Normality

Correct: Exactly! Heterogeneity of Variance is not a common assumption, but Homogeneity of Variance.

Previous – Week 2
Next – Week 4

Subscribe to our site

Get new content delivered directly to your inbox.

Quiztudy Top Courses

Popular in Coursera

Liking our content? Then, don’t forget to ad us to your BOOKMARKS so you can find us easily!

Subscribe