Week 3 – Assignment: Evaluate Sampling Method and Sample Size
This assignment consists of three parts:
(1) Recommend the steps that should be taken to draw the particular sample described below. In addition, critically analyze the sampling plan.
a. A stratified sample of 75 doctors, 75 lawyers and 75 engineers who belong to a professional organization in that you belong to.
b. A simple random sample of 150 subscribers to your local newspaper.
c. A systematic sample of 250 from a subscriber list of a trade publication.
(2) Download the G*Power software provided (Link), and then use the software to submit the following: a. Calculate the sample size needed when given these factors:
· one-tailed t-test with two independent groups of equal size
· small effect size (see Piasta, S.B., & Justice, L.M., 2010)
· alpha =.05
· beta = .2
· Assume that the result is a sample size beyond what you can obtain. Use the compromise function to compute alpha and beta for a sample half the size. Indicate the resulting alpha and beta. Analyze the result and decide if the study should be conducted with a smaller sample size. Explain your rationale. If the study is not worth doing with the smaller sample, discuss possible tradeoffs that may make the sample size feasible for your study.
b. Calculate the sample size needed when given these factors:
· small effect size
· alpha =.05
· beta = .2
· 3 groups
· Assume that ANOVA (fixed effects, omnibus, one-way)
the result is a sample size beyond what you can obtain. Use the compromise function to compute alpha and beta for a sample approximately half the size. Give your rationale for your selected beta/alpha ratio. Indicate the resulting alpha and beta. Give an argument that your study is worth doing with the smaller sample.
(3) Describe the sampling method that would be appropriate for your intended research. Be sure to include the specification of the population, what would be used as a sampling frame, the method and the procedure that you would use to draw the actual sample.
Length: Your paper should be between 7-12 pages not including title and reference page. Feel free to include the results of the G* power analysis that will add length to the paper.
References: Include a minimum of five (5) scholarly sources. Your presentation should demonstrate thoughtful consideration of the ideas and concepts presented in the course and provide new thoughts and insights relating directly to this topic. Your response should reflect scholarly writing and current APA standards.
The two major decisions in designing your sampling plans are the sampling method and the sample size.
Given the desire to generalize the results of a quantitative study, researchers will use a probability procedure, if at all possible. This includes a simple random sample, systematic sample, stratified sample, cluster sampling. A common form of cluster sampling is area sampling where the clusters are the geographical area. The decision of what method is used is dependent on a number of factors including the cost, information, and knowledge of the population, accuracy and the time required.
Sample size determination for a probability sample is largely based on statistical theory. The factors that have to be specified to determine the appropriate size is the variability in the population, the degree of acceptable error and the confidence interval. Note that it is not the size of the population that is important but the degree of heterogeneity of the population. For instance, if everyone in the population was exactly the same with respect to what you are measuring, a sample of 1 would tell you all you need to know. While a researcher can determine the sample size necessary statistically, this may have to be modified due to other factors. For instance, if a survey was being conducted, to obtain the desired sample size given the level of precision and confidence desired, the initial sample may have to be larger due to the completion rate (the number of selected respondents who actually completes the interview or questionnaire) as well as the incidence rate (the percentage of people eligible for participation) in the population.
Unless you design your study adequately and select a sample of sufficient size, your design may be a set-up for a Type II error—failing to find a difference or a relationship that is really there—and your study may be largely a waste of time! You want to have a large enough sample to find a relationship among constructs that is really there and to be able to argue that the relationship is meaningful. At the same time, cost, and the ability to collect the desired number of sample elements have to be considered. There are four factors involved in calculating sample size:
1. Statistical test – Your sample size is partly a function of the statistical test you use. Some tests (e.g., Chi-squared) require larger samples to detect a difference than others (e.g., ANCOVA).
2. Expected/estimated effect size – The effect size is potency of the strength of the relationship you are investigating. In the language of statistics, an effect size is a difference between the mean scores of two groups divided by the pooled standard deviation. This is called Cohen’s d. You will calculate an effect size as part of the analysis of your data in order to determine that you have found something meaningful (not merely statistically significant). However, in advance of doing your study, you must estimate the effect size in your study.
3. Alpha. The alpha level is the probability of a Type I error—of rejecting the null, no difference, hypothesis when it is true—that you are familiar with. By convention, this is set at p=.05. The convention may not be your best guide. The null hypothesis is always false and can always be rejected with a large enough sample, so a .05 level may unnecessarily require you to have a larger sample than you need. It is best to use the literature and your judgment to justify an alpha level that makes sense for your study. This justification will involve looking at the danger of a Type I error versus the cost in resources of avoiding it.
4. Beta. The beta level is the probability of a Type II error—of accepting the null, no difference, and hypothesis when it is false, in other words, of failing to detect a difference when it is there. The main point of a power analysis is to have enough subjects and no more to detect a difference. As with alpha, you set beta based on a judgment. The convention is .2, which yields a power of .8 (1-beta). This is the lowest acceptable level.