UNDERSTANDING STATISTICS IN THE BEHAVIORAL SCIENCES 10TH EDITION BY PAGANO – TEST BANK

UNDERSTANDING STATISTICS IN THE BEHAVIORAL SCIENCES 10TH EDITION BY PAGANO – TEST BANK

CHAPTER 11

POWER

LEARNING OBJECTIVES

After completing Chapter 11, students should be able to:

1. Define power, both in terms of H1 and H0.

2 Define Pnull and Preal, and specify what Preal measures.

3. Specify the effect that N, the size of real effect and the alpha level have on power.

4. Explain the relationship between power and beta.

5. Explain why we never “accept” H0, but instead “fail to reject” or “retain” it.

6. Calculate power using the sign test.

7. Understand the illustrative examples, do the practice problems and understand the solutions.

DETAILED CHAPTER SUMMARY

I. Definitions

A. Definitions.

1 Power is the probability that the results of experiment will allow rejection of the null hypothesis if the independent variable has a real effect.
2. Power is the probability that the results of experiment will allow rejection of the null hypothesis if the null hypothesis is false.
3. Power is the probability of making a correct decision when H0 is false.

B. Preal.

1. Definition. Preal is the probability of a plus with any subject in the sample of

the experiment when the independent variable has a real effect. It is also the proportion of pluses in the population if the experiment was done on the entire population and the independent variable has a real effect.
2. Preal varies with the size and direction of the real effect.
3. Preal equals any value other than 0.50 (H1 nondirectional)

C. Pnull.

1. Definition. Pnull is the probability of getting a plus with any subject in the sample of the experiment when the independent variable has no effect.
2. Pnull = 0.50 (H1 nondirectional).

II. Power and Beta ()

A. Formula.

Power = 1 − beta

Beta = 1 − power

B. Relationship between power and beta.

1. As the power of an experiment increases, the probability of making a Type II error decreases.
2. Maximizing power minimizes beta.

C. Methods of increasing power.

1. Increase the size of effect of independent variable.
2. Increase sample size.

III. Identifying Correct State of Reality

A. Alpha and beta values.

1. Set alpha stringently (minimize Type I error).
2. Set beta low (minimize Type II error).

IV. Calculation of Power is a Two-Step Process

A. Step 1. Assume the null hypothesis is true (Pnull = 0.50) and determine the possible sample outcomes in the experiment that allow H0 to be rejected.

B. Step 2. For the value of Preal under consideration (e.g. Preal = 0.40) determine the probability of getting any one of the above sample outcomes. This probability is the power of the experiment to detect an effect size equal to Preal.

TEACHING SUGGESTIONS AND COMMENTS

Chapter 11 discusses power. Power is a difficult topic. Discussing power in conjunction with the sign test makes it easier than otherwise, but it is still difficult. The chapter is divided into two main parts; the first develops the concepts, and the second explains how to calculate power using the sign test. When I have time, I cover both, but often I only do the first part. The first part ends on p. 288; the second part begins with the section titled, Calculation of Power. Since this is a difficult chapter, I recommend you follow the textbook closely in your lectures. Specific comments follow.

1. What is power. The usage of the phrase “a real effect of the independent variable” given in italics on p. 278, namely, “By ‘a real effect of the independent variable’, we mean an effect that produces a change in the dependent variable,” needs a little elaboration. When analyzing the results of an experiment, the scores on the dependent variable obtained in the experimental condition will be different than those in the control condition. We need to decide if the differences in scores between conditions are due to chance, or to “the real effect of the independent variable.” If they are due to the real effect of the independent variable it is because the independent variable produced a change in the dependent variable. Thus, when we say the independent variable had a real effect, we mean that it produced a change in the dependent variable. In this context, a real effect of the independent variable can’t be that it has no effect on the dependent variable.

2. Pnull and Preal. Preal is a measure of the size and direction of the real effect of the independent variable. The real effect gets greater as its value deviates from 0.50. Thus, Preal = 0.00 indicates a real effect as large as Preal = 1.00. This feature of Preal can be confusing to students. It is important to make sure students understand this scaling. I recommend explaining it in conjunction with showing a transparency of Figure 11.1 on p. 279. Additionally, students need to understand that the X axis of this figure is a probability scale, and that all the probability values are values of Preal except 0.50, which is the value of Pnull. Thus, on this scale, all values of P < 0.50 and P > 0.50 are values of Preal. P = 0.50 is the value of Pnull.

3. Power analysis of the aids experiment. Table 11.2, p. 284, summarizes the effect of changing N and α on power. Be sure students understand this table.

4. Interpreting nonsignificant results. The major point of this section is that we can never conclude by “accepting H0” as true because the power to detect very weak, but real effects is always low. Instead, the proper conclusion when the results do not reach significance is to conclude by “retaining H0”, as a reasonable explanation of the data, but not accepting it as true. This is an important point that needs to be emphasized.

DISCUSSION QUESTIONS

1. Discuss how power fits into the hypothesis testing process. Why is power an important part of this process?

2. Discuss the relationship between power and N, size of real effect, and alpha level.

3. Very often nonscientists and even scientists when evaluating the conclusions of research, conclude that if the research fails to reject chance, the alternative hypothesis is false. In your opinion, is this a correct conclusion? Discuss.
4. Since conducting a powerful experiment is so valuable in the event that the results yield nonsignificant results, some scientists advocate that experimenters only conduct experiments that have high power. While on the surface this may seem like a sensible policy, as you think deeply about it, doubts may emerge. After giving the policy advocated due consideration, what is your opinion? Discuss.

5. Assume there are two experiments. The results of both are analyzed using the sign test and the same alpha level. The results of both experiments are significant (assume computations are correct). Both experiments have the same power over the full range of Preal values. In Experiment A, N = 10 and in Experiment B, N=100. In which of the two experiments would you have more confidence that the rejection of H0 has yielded the true state of reality? In which of the two experiments do you suspect the size of the real effect is larger? Discuss.

TEST QUESTIONS

Multiple Choice

1. With alpha and the effect of the independent variable held constant, as N increases _________.
a. power increases
b. the probability of a Type I error increases
c. the probability of a Type II error increases
d. power decreases

2. With alpha and N held constant, as the effect of the independent variable increases _________.

a. The probability of rejecting H0 increases
b. power increases
c. the probability of rejecting H0 remains the same
d. the probability of making a Type II error decreases
e. a, b, and d
f. a and b

3. With the effect of the independent variable and N held constant, as  gets more stringent _________.
a. power increases
b. power stays the same
c. power decreases
d. beta decreases

4. The advantage of a powerful experiment is that _________.
a. the probability of a Type I error is reduced.
b. you are more likely to detect the real effects of the independent variable, if there are any.
c. as power increases, alpha increases.
d. as power increases, N increases

5. If H0 is in reality false and it is legitimate to use a directional H1, which of the following will yield a more powerful test?
a. one-tailed alternative hypothesis
b. two-tailed alternative hypothesis
c. one-tailed or two-tailed alternative hypothesis will yield the same power
d. type of alternative hypothesis has no effect on power

6. If H0 is true and the probability of making a Type I error is 0.05, then the probability of making a correct decision is _________.
a. power
b. 0.05
c. 0.95
d. need more information

7. In an experiment in which H1 is nondirectional, and the state of reality is such that the probability of a plus = 0.50, the probability of making a Type II error is _________.
a. 1  alpha
b. 0
c. 1
d. need more information

8. Maximizing the power of an experiment _________.
a. minimizes alpha
b. minimizes beta
c. increases the probability of rejecting H0 when H0 is true
d. increases the probability of making a Type II error

9. If H0 is true, then the probability of rejecting H0 is limited by _________.
a. Power
b. 1  power
c. alpha
d. beta

10. Which of the following values of P represents the largest effect?
a. Preal = 0.20
b. Preal = 0.70
c. Preal = 0.75
d. Pnull = 0.50

11. If N = 4, Preal = 0.80 in the direction predicted, and  = 0.051 tail, using the sign test, the power is _________.
a. 0.9375
b. 0.0000
c. 1.0000
d. 0.0625

12. If H1 is nondirectional and the size of the effect of the independent variable decreases, the probability of making a Type I error _________.
a. decreases
b. increases
c. equals zero
e. need more information

13. With other factors held constant, as the effect of the independent variable decreases, power will _________ and the probability of a Type II error will _________.
a. decrease, increase
b. decrease, decrease
c. increase, decrease
d. increase, increase

14. The power of an experiment is affected by _________.
a. the alpha level
b. the sample size
c. the size of effect of the independent variable
d. all of the above

15. If the null hypothesis is false, the probability of making a correct decision is given by _________.
a. alpha
b. beta
c. 1  alpha
d. power

16. If the power of an experiment is 0.7496, the probability of making a Type II error is _________.
0.2498
a. 0.05
b. 0.2504
c. cannot tell without more information

17. You can only compute power when ________.
a. alpha > 0.05
b. H1 is given
c. Beta > 0.05
d. 1  power = 0.05

18. Power equals _________
a. alpha + beta
b. 1  beta
c. 1  alpha
d. alpha  beta

19. You are considering testing a new drug that is supposed to facilitate learning in mentally retarded children. Because there is relatively little known about the drug, you plan to use a nondirectional alternative hypothesis. Since your resources are limited, you can test only 15 subjects. The subjects will be run in a repeated measures design and the data will be analyzed with the sign test, using  = 0.052 tail. If the drug has a medium effect on learning such that Preal = 0.70, what is the probability you will detect the effect when doing your experiment.
a. 0.2968
b. 0.0176
c. 0.2969
d. 0.0352

20. You are considering testing a new drug that is supposed to facilitate learning in mentally retarded children. Because there is relatively little known about the drug, you plan to use a nondirectional alternative hypothesis. Since your resources are limited, you can test only 15 subjects. The subjects will be run in a repeated measures design and the data will be analyzed with the sign test, using  = 0.052 tail. What is the probability of a Type II error?
a. 0.7032
b. 0.7031
c. 0.9824
d. 0.9648

21. You are considering testing a new drug that is supposed to facilitate learning in mentally retarded children. Because there is relatively little known about the drug, you plan to use a nondirectional alternative hypothesis. Since your resources are limited, you can test only 20 subjects. The subjects will be run in a repeated measures design and the data will be analyzed with the sign test, using  = 0.052 tail. What is the power this time?
a. 0.4163
b. 0.2969
c. 0.6161
d. 0.2458

22. If the independent variable has a real effect, the probability of rejecting H0 is _________.
a. power
b. 1 – power
c. alpha
d. beta

23. One can increase power by _________.
a. increasing N
b. making alpha more stringent
c. increasing the effect of independent variable
d. a and c

24. If the power of an experiment is 0.3400, the probability of retaining H0 when H0 is false is _________.
a. 0.6600
b. 0.3400
c. 0.5000
d. 1.0000

25. Which of the following P values represents the strongest effect?
a. Pnull = 0.50
b. Preal = 0.70
c. Preal = 0.80
d. Preal = 0.

26. If beta = 0.7500, what is the power of the experiment?
a. 0.7500
b. 0.5000
c. 0.2500
d. 1.0000

27. Which of the following represents the null hypothesis condition for a nondirectional H1?
a. Preal = 0.49
b. Pnull = 0.50
c. Preal = 0.51
d. all of the above

28. In the case of N = 3, Preal = 0.90,  = 0.051 tail, using the sign test, the power is _________.
a. 1.0000
b. 0.1250
c. 0.8750
d. 0.0000

29. Which of the following may be false?
a. alpha + beta = 1
b. power + beta = 1
c. 1 – power = beta
d. 1 – beta = power

30. In a two-tailed test, which of the following would yield the same power as a size of effect represented by Preal = 0.90?
a. Preal = 1.00
b. Preal = 0.10
c. Preal = 0.80
d. none of the above

31. If H0 is false, the probability of making a correct decision is _________.
a. alpha
b. 1 – alpha
c. power
d. 1 – power

32. Assume that you have conducted an experiment and the tail probability for the results you obtained was 0.0900 and  = 0.05. You can conclude _________.
a. the independent variable has no effect
b. the independent variable has a small effect
c. you were unable to detect an effect of the independent variable with your experiment
d. the null hypothesis is false

33. Truth demands that scientists set _________.
a. alpha equal to 0.05
b. alpha equal to 0.01
c. a or b
d. not necessarily  = 0.01 or 0.05, but use their best judgment

34. In the sign test, if H0 is false, then Preal _________.
a. equals 0.05
b. equals 0.50
c. equals 1 – 0.05
d. does not equal 0.50

35. In an experiment, if the effect of the independent variable and alpha remain the same and N increases, _________.
a. power increases
b. power decreases
c. power remains the same
d. cannot be determined

36. In an experiment with N = 14 and Preal = 0.10, what is the power using  = 0.052 tail?
a. 0.8417
b. 0.0132
c. 0.0000
d. 0.0500

37. In an experiment with N = 14 and Preal = 0.10, what is the power using  = .012 tail?
a. 0.0500
b. 1.0000
c. 0.5847
d. 0.9868

38. An experiment with N = 18 is more powerful than an experiment with N = 17, all other things being the same.
a. true
b. false

39. In order to calculate power you must know _________.
a. alpha
b. N
c. P
d. the results of the experiment
e. all of the above
f. a, b and c

40. In an experiment with N = 6 and Preal = 0.15, what is the power using  = 0.012 tail?
a. 1.0000
b. 0.5000
c. 0.0000
d. 0.3771

41. Power is defined as ______.
a. the probability of rejecting H0 if H0 is false
b. the probability of accepting H1 if H1 is true
c. a and b
d. the probability of rejecting H0 if H0 is true

42. A real effect of the independent variable is defined as _________.
a. all possible effects including no change in the dependent variable
b. any effect that produces a change in the dependent variable
c. an effect that produces an increase in the value of the dependent variable
d. an effect showing no change in the dependent variable

43. Pnull is defined as _________.
a. the probability of getting a plus with any subject in the sample of the experiment when the independent variable has no effect in the direction predicted by a directional H1
b. the probability of getting a plus with any subject in the sample of the experiment when the independent variable has a real effect
c. the probability of getting a plus with any subject in the sample of the experiment when the independent variable has an effect in the direction predicted by a directional H1
d. the probability of getting a plus with any subject in the sample of the experiment when the independent variable has no effect.

44. Preal is defined as _________.
a. the probability of getting a plus with any subject in the sample of the experiment when the independent variable has a real effect
b. the probability of getting a plus with any subject in the sample of the experiment when the independent variable has no effect
c. the probability of getting a plus with any subject in the sample of the experiment when chance alone is at work
d. the probability of getting a plus with any subject in the sample of the experiment when the independent variable has a real effect in the direction predicted by the directional H1