Chapter+Three+-+Sampling+Concerns+and+Processes

=Samples and Populations=

The idea of this chapter is to make sure that you address sampling issues - even if you are conducting an action research project and your sample is "defined" by your classroom. This chapter should address the larger issues involved, especially in terms of the impact of sampling on research designs and outcomes. Consequently, the sections will focus on the definition of a sample, the strategies used in obtaining samples, as well as differences in perspectives based on research methodology. The discussion for this chapter should provide the reader with a clear understanding of sampling characteristics and techniques. Sampling is vital to your research because this is the part that helps you in answering your research question, or fairly testing your research hypothesis.

Who will provide the data? How useful is the data? Measurement soundness (reliability and validity)...if these are not sound then the results can lead to faulty conclusions and improper inferences.

I. Samples versus Populations
Populations, in regards to sampling, are divided into two groups for the purpose of research. The targeted population is the group you might target for your because they would meet your research needs. For example, if you were researching the effectiveness of the Ohio Graduation Test, your targeted population would be all the students who take the OGT. It is not feasible to expect every student in Ohio who has taken this test to be a part of your study. Therefore, you develop an accessible population, or those for whom it is feasible to participate in your study. That might mean one district or one high school from which you draw your sample. Your sample is the group of your accessible population that participates in your study.

====**Two Types of Generalization:** There are two types of generalizations, population and ecological. External validity means generalization. Problems with either population generalization or ecological generalization can threaten external validity.====

**1.) Population generalization**
====Population generalization is a form of generalization which focuses on the research participants themselves (people), apart from the setting. It is "the extent to which research findings extend beyond the sample of research participants who provided data." (Suter, 217). Are the people in a study representative of the general population? How well does the sample mirror the population? For example, if you wanted to do research on how physical activity effects academic achievement of elementary school students (grades K-3), but 90% of your research was on 2nd graders, it would be hard to generalize those results to all elementary (K-3) school students.====

====Ecological generalization (setting) is the extent to which research findings extend beyond the setting which produced sampled data (Suter, 218). It is the ability to apply the results outside of the study. The setting encompasses all aspects of the setting except the research subjects themselves. This can be the location of the setting, who the experimenter is (man or woman), and the type of setting where the research is being conducted (classroom vs. having an experiment in a white room, in person interviews vs. over the phone vs. emails)." Ecological generalization is no less important than population generalization, since problems with it can also threaten external validity." (Suter 217)====
 * 2.) Ecological Generalization**

II. Sampling Strategies
Sampling is important in order to integrate research into your paper. Depending on the information that you are looking for, and the type of research you are completing, it will help determine what type of sampling would be best suited for your research. The articles being collected for the research paper specify the sampling method utilized for their research. This infomation can be used as a point of guidance.

Qualitative research sampling focuses on a small, specific population. When you are doing a qualitative study, you are usually talking to people face to face or observing them. "Above all else, the sampling strategy in qualitative research is designed to yield "rich" data, with sources including a single person, a small group, and complex organizational sites." (Suter, 222). In this type of study, you may use the following techniques : 1.) Homogeneous samples which shows that everyone has the same characteristics; and 2.) Intensity samples which shows people in the sample have opposing characteristics. Snowballing is when you have a very small group for your sample and you use the few people you do have to find new people that fit your sample.
 * Qualitative Research Sampling:**

Quantitative research sampling focuses on a large sample and is based on the population. According to Suter, quantitative researchers use statistical methods and often assume participants have been randomly selected or assigned to experimental conditions. Usually quantitative researchers will use a variant of probability sampling because their participants ideally represent a larger target population.
 * Quantitative Research Sampling:**

Action research sampling is most often centered on practical problems within a setting such as a classroom, school, or district. It is also often used by teachers or other educational practitioners who wish to improve their practice or more fully understand it. Action research is often based on personal ideas or concerns to the researcher (something unique to a particular class or group), and the results of the study are not meant to be generalized beyond the sample group. Because of this, the sample and the population are the same. Because this type of research only applies to the particular class or group that is being studied, sample sizes are usually small. They often represent an individual class or a subgroup of a class. Action research is also an ideal place for collaboration between teachers because the process involves the dynamic processes of sharing, self-reflection, data collection, analysis, and action for change. (Suter, 2006). One strength of this type of research is that the findings are easily translated into practice.
 * Action Research Sampling:**

A. Random Sampling
Random Sampling is used to obtain appropriate numbers within target groups. (ppt 2.8) When conducting a study, it is important that the subjects of that study reflect the general population. This can be accomplished by using random sampling. With random sampling, every participant has an "equal opportunity of being part of the sample (p. 212). This is accomplished by using a table of random numbers. Possible participants would be numbered, and the table would be used to select the numbers (therefore selecting the participants). This is different than just drawing from a hat or flipping a coin. When drawing from a hat, the last names/numbers put in are likely to be the first ones chosen (because they are on top). With this method, every person does not have the same chance of being picked. (Suter, 2006).

Stratified Sampling is another method that can be used to gather data. One would use this type of sampling when you need to have an equal number of groups for each independent variable represented. For example, if you needed an equal number of male and female participants, you would take your random sampling of the population, put them into groups of male and female, then randomly select from each catergory. You could use this sampling with equal number or a proportional number of participants based on the purpose of your study. (ppt. 2.8 & 4.5). Stratified Sampling ensures equal representation of each category.

Another variant of random sampling is clusters. Clusters are intact groups that cannot easily be chopped into small units or individual students (Suter, 214). In educational research, most of the clusters encountered include classes, schools, and districts. Suter notes that "clusters can be randomly selected in the same way that individuals are randomly selected" (p. 215). For example, if someone was doing research on schools that utilize Positive Behavior Support, the schools could be numbered and then drawn in the same fashion as other random samples. This is called a "randomized cluster" (Suter, 215). Two staged sampling is suggested when clusters are very large. For example, from the randomly ten school districts, choose five principals. Cluster Sampling can be used if you do not have enough money or time to conduct a large quantitative sample. A random selection of groups of people would be appropriate for Cluster Sampling. For example, the city of Cleveland has many different zip codes throughout the city, with Cluster Sampling, the zip codes can be narrowed down to five zip codes of choice to use for the sample. Of the five zip codes, one can randomly select 50 individuals residing within those five zip codes to conduct the sample.

Multiple Stage is used by reasearcher when they need to select their samples in two or more stages. "For example, 60 schools may be selected randomly in a state, followed by the random selection of 20 classes within each of the 60 schools. This plan would be described as two-stage random." (Suter 215).

**B. Non-random Sampling**
In non-random sampling, not everyone has the chance of being chosen for this type of sample. There are several techniques used with non-random sampling. Convenience sampling is the most popular technique used in educational and social science research. This is when you choose a particular person or group of people for your sample based on your own reasons. For instance, 80 percent of psychological studies are done on undergraduate students. This is a large group of people that are readily available and are not opposed to participating in these studies. They are convenient to sample. Another example of convience sampling is magazine surveys or surveys conducted in malls or stores. The results may be biased because people are choosing to answer the surveys themselves and the population isn't represented equally. There is also a chance for sampling error because of the chance of non representative samples. For example, you may have only elderly people participating in your sample. Another example of Convenience sampling would be utilizing the students in your own classroom to conduct the sample. Convenience sampling is a self selection of participants in the sample.

Other types of non-random sampling are Purposive and Systematic. Purposive is when you are using certain criteria designed to exclude or include certain participants. For example, if you wanted to determine how many new teachers are passing the Praxis test on the first try, you would only sample teachers that have taken the test, not all teachers. It is important to know that your sample may suffer from attrition because people no longer meet the requirements of the study. You must keep your sample size large enough to make up for the possibility of attrition. The larger the sample, the more accurate it will reflect the population. Systematic sampling is used like quality control. You would sample every //one thousandth (kth)//person. Although these types of samples are not random, they are still valid.

III. Sample Size
There is no one number that constitutes the perfect sample size, and the determination is based on the type of study being conducted: group comparison versus correlation study. Saying this, however, traditional educational research studies typically have sample sizes of between 30 and 60 participants per group. This "magic number" derives from the fact that the numerical mean used as an index to indicate a central tendency stabilizes or becomes dependable based on a sample of at least 30 subjects. With the passage of No Child Left Behind (2001) ideas regarding the sample size in educational research became more rigorous: the rough rule of thumb became 150 participants in each group being compared. If the research compares entire classrooms or schools (as opposed to individuals) the recommended size for rigorous evidence is 25-30 schools or classrooms in each group. The dropout rate (the loss of subjects for any reason) cannot exceed 25% (Suter, 2006). The sample size allows you to know if there is a significant relationship between variables in the sample. The larger the sample, the more likely it is to uncover significant relationships among the variables if one exisits. The effect size will then allow you to know how strong or weak the relationship is among the variables. If a sample size is too small, the results may be null because you are not able to generalize the results to a larger population. Therefore, nothing new is learned and the research is not applicable. On the other hand, a large sample size is not always better if the sample does not contain a true representation of the population. For example, a study that observes 1 million girls and then generalizes the information saying that it applies for "all" children.

A. Quantitative versus Qualitative
In a quantitative research, researchers select a larger target population. Researchers also use the probability sampling technique to determine the probability of each member of the participants. Researchers use statistical methods in which participants are randomly selected. Structured techniques are used in this research, such as telephone interviews or street interviews. Quantitative designs are based on numerical data that is collected through a variety of methods, and generally seek to test a researcher’s hypothesis about the effect of an intervention, the relationship between variables, or differences between individuals or groups. This design is based on numerical variables, which can be indicated in the form of percentages or fractions. Dependent variable(s) of a study is (are) usually categorical.

In a qualitative research, on the other hand, researchers select one person or a small group of participants. The goal of this research is to have rich data. Researchers select participants for different purposes. Unstructured or semi-structured techniques are used. These include: individual depth interviews or group discussions. The names for different sampling strategies in qualitative research reveal varied purposes, all of which provide the richness in data that becomes the essential condition in qualitative research (Suter, 222). Qualitative research attempts to tell a story about the setting in which the data were collected, the characters who provide the information that is used as data, and a plot which describes the social interaction of the characters. Most of the data gathered in qualitative studies comes from observation and interview. In general, qualitative research designs using grounded theory involve: a focus or topic that is descriptive of some process or interaction; a design that emerges as the data is collected and analyzed; purposive sampling and generally small samples (in comparison to most quantitative designs); the participant’s context; and, a thick description or narrative of the results (i.e. “the story”) regarding the process. This design involves categorical variables, which can be gender or race. The independent variable(s) of a study is (are) usually categorical.

B. Generalizability
External validity “refers to how well finding extend, or generalize, beyond the sample to different people (population generalization) and settings (ecological generalization). (Suter, 2006) “ Educational generalizations as presented through research, have historically been difficult to translate, meaning what one may find it beneficial in one sample study and this may not be the case when applied to the next study or more importantly to a real-life situation. This is because no situation is without context. Suter refers to this notion as the “complex bundle” situation as in that educational situations are comprised of many variables such as student and teacher “culture, motivation, interest, and aptitude. (Suter 389)”

For example, if a study finds that establishing gay-straight alliances within a few public high schools decreases bullying, then it can be assumed that establishing gay-straight alliances would decrease bullying in high schools across the entire United States. One could take it a step further and state that it may also be assumed that if establishing gay-straight alliances decreases bullying in a high school setting, then establishing gay-straight alliances in middle school settings will decrease bullying in the same way. If study findings have been proven to be successful within the study, in order to be valid, the findings should also be successful in various environments outside of the study.

C. Type One and Type Two Error
The null hypothesis is a statistical assumption about the population from which the same class was drawn (Suter, 2006). When a researcher decides to either accept or reject this hypothesis, they may in fact be incorrect in their decision. This decision may have resulted due to a sampling oddity. This error, that researchers refer to as //sampling error// comes from a fluke within the sample. There are two types of errors; //type one// and //type two//.

//Type one error// is when the null hypothesis is wrongly rejected (Suter, 2006). It claims that there is a connection to the population, however there is not a relationship to it. This type of error is sometime referred to as //alpha// or //alpha error//. Unfortunately when this error occurs, the researchers are unaware of it until multiple studies accept the null hypothesis. This error does not happen very often, Suter says it is like winning the lttery. This is also not an error that a research should be faulted for because he/she cannot control over the" random oddities". Suter refers to this type of error as a false alarm.

//Type two error// is a statistical fluke in sampling that permits an incorrect acceptance of the null hypothesis (concluding there is no relationship in the population when in fact there is a relationship) (Suter, 2006). For example if you are studying the effects of anti-bullying programs on reducing bullying behaviors, you may have a group that is overrepresented by students who were unaware of the types of behaviors that would be considered bullying. If this is the case, you might not see a change in the frequency of observed bullying behaviors, which may lead you to believe that the program is ineffective. This type of error is sometimes called //beta// or //beta error//. Suter refers to this type of error as a missed sighting.

**A. Sampling error**
Sampling error are things that happen that are not completely under the researcher's control. For example, if the researcher sends out a questionnaire to one hundred people and only seventy-five respond, and of that seventy-five, fifty of the participants are males; there is nothing the researcher can do about it. It can also be a fluke in the sample - for example, if the researcher is looking at literacy in first-graders and the population he tests happens to have high reading comprehension. (Type I error) This error occurs from testing a sample, rather than an entire population. The researcher can control somewhat for sampling errors by making sure they take a random sampling of the population and by taking a large enough sample. It is more costly to look at a larger sample size, which makes it difficult for some researchers to do, but it gives a more accurate view of the population being tested.

**B. Sampling bias**
====Sampling bias occurs when the researcher allows his or her own opinions to affect who he or she choses for the sample. An appreciation of representative sampling might be gained from a brief description of faulty or “curious” sampling (Suter, 219). It is important to know the characteristics of the collector of the data, as well as the desired sample, in order to alleviate bias. For example, if a local school system received a $10,000 grant and wanted to maximize the money to most effectively benefit the school system, depending on target group, the circumstance may lead to a formation of bias. If the athletic boosters were asked, they would more likely indicate the need for money to put toward the football field, gymnasium, or uniforms. Whereas, if the collector inquired to the parents whose children were in liberal arts, the parents may advocate that the money be contributed to the redesign of the theatre. A conflict such as this, provides reason as to why a sample needs to represent the entire population, rather than target a certain group. Lack of representation generally leads to a bias.====

No matter who you are bias is a natural part of humans. Having a good description in your methods section as to your process of choosing your sample as well as how you administer your survey or interview is important to limit the amount of bias in your research. Also, it allows the reader to see any bias that may affect the use of the results.