Chapter+Five+-+Quantitative+Research+Designs

=Quantitative Research Methods=

The purpose of this chapter is to provide a discussion of the various research designs used in quantitative research studies. Each design should be clearly described, including procedural steps regarding sampling and data collection as well as the type of statistical analysis that would be used to determine the outcome(s) of a study using that design.

I. An Overview
A quantitative study tests specific hypotheses, usually stated in advance, and incorporates measures which can be analyzed statistically. Information on the age, gender, ethnicity,and/or socioeconomic status is usually stated in these studies to provide a clear picture of the data being collected. This type of research uses tables or charts to display findings that can (hopefully) be generalized beyond the sample to a wider population (inferential statistics). The researcher is distant in a sense from the study and thus guards against bias and other influences which may change the results (Suter, 41). Many quantitative research designs are true experimental or quasi-experimental (involving some type of intervention). These research designs are also highly structured and influence the type of data collection being used. The designs need to be set in place before the researcher can begin collecting data and the instruments being used for this type of research are typically standardized. Quantitative designs are based on numerical data that is collected through a variety of methods, and generally seek to test a researcher’s hypothesis about the effect of an intervention, the relationship between variables, or differences between individuals or groups. This design is based on numerical variables, which can be indicated in the form of percentages or fractions. Dependent variable(s) of a study is (are) usually categorical.
 * Constructs and Variables in Quantitative Research**:

Constructs, or an abstract theory or idea, suggest a more specific variable to be investigated with measuring tools presumed to represent traits or behaviors defined by the variables. Constructs can not be measured, however, constructs are represented by variables that can be measured. “The construct and variable are best understood within the context of measurement soundness, control, sampling, and statistical analysis” (Suter, 130). Educational research is challenging because educators are interested in complex abstractions. "The abstract dimensions that interest educational researchers are called constructs because they are constructed or invented labels-a shorthand way of describing many interrelated behaviors, all of which are postulated to represent the same trait or ability" (Suter, 106). Therefore, we can think of a construct as a label for a trait that we are presuming, it can be defined in many different ways. Interobserver agreement is very important in terms of "constructs". The behavior being observed may be viewed differently by multiple researchers, so it is not a wise practice to label a trait without interobserver agreement being .90 or above.

**II. True and quasi-experimental Designs**
True experiments are the best method to discover cause and effect relationships. Experimental designs are often touted as the most "rigorous" of all research designs or as the "gold standard" against which all other designs are judged. The addition of a control group is necessary to yield the best possible results. It is not necessary for a research design to have a control group in order for it to be a "true experimental" design, but having a control group helps with internal validity. True experiments have the best internal validity. Internal validity is at the center of all causal or cause-effect inferences. True experiments have independent and dependent variables. The independent variable "involves a manipulation coupled with random assignment of the subjects to groups" http://www.socialresearchmethods.net/kb/desexper.htm (Suter, 265).

The following are some examples of true experimental designs: __Randomized Posttest Control Group Design: R T Post---R C Post__ In this design participants are randomly assigned (R) to either the treatment group (T) or control group (C). The treatment group is given the intervention and the control group is not. Both groups are then given the same posttest. The posttest is given to assess the influence the treatment has had on the participants. The presence of the control group allows the researchers to subtract for extraneous influences allowing for purer results of the treatment effect (Suter, p. 266). __Randomized Pretest-Posttest Control Group Design: R Pre T PostR Pre C Post__ This design is the same as above except for the addition of a pretest given to everyone. The pretest acts as a baseline to compare the end results of the study. (p.268) __Randomized Matched Control Group: M R T PostM R C Post__ This design matches participants prior to random assignment. This is a good design to use if the sample size is small. Participants are given a pretest then matched based on the results. The two top scoring people are matched, then the next two, and so on down. One member of each pair is then randomly assigned to a group. The researcher is attempting to have both groups be similar before the treatment or control. Any differences in the posttest results can then be attributed to the intervention. (p.269)

The ABC design method is a example of a randomized pretest-posttest control group design. This design method takes a look at the antecedent, behavior, and consequence of the behavior. In this case, the antecedent would be the independent variable and consequence would be the dependent variable.

One of the main differences between true and quasi-experimental designs is that in quasi-experimental designs, the independent variables do not have random assignment. For example, if a class of students were selected to receive a treatment (an anti-bullying program), and another class of students were selected based on similarities (SES, race, age, gender, etc.) to act as a control, this would be a quasi-experimental design because the groups were already in existence prior to the manipulation meaning the subjects were not randomly assigned to the treatment or the control group. This means that it may be harder to control extraneous variables (Suter, 2006).

Quasi-experimental designs are used when randomization is either impossible or impractical. This, naturally, makes them easier to set up than true experimental designs. These designs take less time and effort to study and compare single subjects or groups of subjects that have naturally been organized than it does to set up random assignments on the subjects. Using quasi-experimental designs also minimizes threats to external validity. Quasi-experiments are natural experiments; this means that findings in one may be applied to other subjects and settings, which allows for generalizations to be made about populations. Finally, this experimental method is efficient in longitudinal research that involves longer time periods which are followed up in different environments.


 * Single-subject designs** are designs of the quasi-experimental research design. They can be applied when the number of individuals relate to a single group, such as an ESL classroom. To determine any changes in the research, these designs depend on gathering pretest information, which is called a baseline measure. They also depend on measuring the dependent variable or behavior, for a certain period of time, before the treatment. Periods of measurement are used in these designs to determine the change as well as the degree of change through the process of behavioral modification. The two applications of these designs are an ABAB design and a multiple baseline design.

http://www.psych.auckland.ac.nz/psych/Courses/306/MichaelD/SS3.htm



**III. Non-experimental Approaches**
When looking to avoid interpretations of cause and effect relationships, a non-experimental approach may be appropriate. Non-experimental designs lack an intervention or treatment component (Suter, 295). These designs are geared towards answering the "how" and "why" in regards to the success or failure of a research question. Non-experimental approaches require creativity as they do not interpret the cause and effect, but they do uncover relationships of interest. **There are 3 classes of non-experimental designs: correlational, descriptive, and causal comparative.**

A. Correlational Designs
Correlational research is very similar to casual comparative research except that it Correlational research does not examine differences between groups. Instead, it typically involves two or more variables within a single group. Correlational research studies attempt to establish a correlation between relationships. The research attempts to determine whether, and to what degree, a relationship between variables exists. Variables in correlational research must be expressed in numeric form, and the variables must be observed as they occur naturally. The purpose of a correlational research study is to examine relationships between variables and to make predictions based on the findings. In correlational research we must be careful not to assume that correlation always equals causation. For example, we can not say that all students that attend inner city schools do poorly in college or that wealthier school districts do better on standardized tests than poorer districts, so attending a wealthy district increases achievement. There are multiple factors that could influence these correlations. Unlike causal comparative designs, correlational designs do not establish cause and effect and interprets on theoretical grounds. Some advantages of correlational designs include, the ability to examine relationships between a large number of variables, the ability to examine the degree of the relationships and note the extent of differences. Aside from this, correlational design variables can also be measured through tests, surveys, questionnaires, and interview observation however; the data must be quantified so that it can be analyzed numerically.

Furthermore, correlation is evident in all types of research, including quantitative, qualitative, and action. Correlation is very effective in quantitative research. It refers to a linkage between variables, one that is revealed by any one of several non-experimental designs. In quantitative research correlation usually manifests in the form of scattergrams, and the primary method of data becomes the correlation coefficient. In qualitative research, correlation may include connections such as typologies, sequence charts, hierarchies, tables, and a variety of matrices, networks and displays." Still " action research borrows non-experimental, correlation designs from both qualitative and quantitative traditions to the extent that they assist reflective practice and 'actionable' research."

An advantage of correlational design is the ability to examine relationships between a large number of variables. The information in this type of research can be obtained about the degree of relationship as opposed to casual comparative designs that note the extent of differences.

B. Survey Designs
Surveys are typically used by researchers who want to obtain information from a group for the purpose of describing characteristics of that group (Suter, p.310). Surveys can take many forms, but most often take the form of a written questionnaire. The format can be changed to fit your personal needs. A researcher may simply have people check pre-selected answers, or they could include open-ended essay questions. Some commonly used survey formats include the Likert Scale (1 means strongly disagree, 5 means strongly agree), Rating Scales (never, rarely, sometimes, often, always), Forced Choice (one or the other), Semantic Differential ( compare two things: worse--better), Thurstone Scale (true, false, can't decide), and Adjective Checklist.

Let's look at an example of the elements of survey design using research regarding the effectiveness of school retention policies. Surveys for these purposes state specific questions concerning different people--students, teachers, parents, and school administrators. Thus, one survey would not be appropriate for general use; rather four different surveys asking the same general questions but geared toward the specific respondent would be necessary. This would help increase the effectiveness of the results. When looking at a survey design, it is important to remember who the sample is and what information is needed from them.

Surveys are often prone to respondent biases for several reasons. Survey participants may be embarrassed about actual answers and may not be honest when completing the survey. For example, a survey on school bullying may not indicate truthful responses about the bullying that the child has experienced because they feel ashamed and embarrassed that they haven't been able to stop or prevent the bullying. Additionally, if the survey participants have been the victims themselves some participants might not answer accordingly. One way to avoid respondent bias is to ensure the privacy/confidentiality of the study participants. Survey bias also occurs when the surveys are not self-evaluations. For example, the PEP-3 is an assessment tool that includes a caregiver survey. In the survey, caregivers are asked to rate behavior and skill levels of their nonverbal children. Bias may occur because parents naturally rate their children higher than average. Or, since parents are not experts in the field of special education, they may not completely understand the survey questions and may make inaccurate assumptions.

Surveys can also be used to determine the learning styles of students. A type of this survey would be created utilizing forced choice. A student could be asked "Do you prefer to read a book or participate in a discussion on a assigned topic?". A student who prefers to read books may be a visual learner, and a student who prefers to participate in a discussion may be a auditory learner.

C. Causal Comparative Designs
 Causal comparative research is a comparison of groups such as, classrooms, that allows inferences to be made based on the researchers hypothesis. With causal comparative designs at least two groups are being compared, for example, gender and race of classroom students. Gender and race varibles can be measured through tests, questionnaires, attitude surveys, and interview observation. In addition, causal comparative research designs are intended to search for causes or effects of group classifications formed by preexisting differences (Suter, 298). Sometimes referred to as an ex-post-facto (meaning prior to, retroactive, or going back to), this design looks to compare groups that exemplify differences. These differences are preexisting. An example would be comparing the graduation rates of traditional students to those of ESL students. The students cannot be assigned to these groups. The groups already exist and the design stays the same whether the researcher is interested in a cause or an effect. Causal comparative research designs are not intended to establish cause and effect relationships, however, they focus on either a presumed cause of an effect or a presumed effect of a cause. There is no intervention in casual comparative designs, all relationships occur naturally.

It can be easier to reach a higher understanding of the definition and purpose of a causal comparative design when it is compared to other types such as experimental studies or correlational studies. In a casual comparative design you are finding the cause for preexisting differences in groups of individuals. Since the cause has occurred already the questions being asked has to be studies in retrospect. Casual comparative designs try to identify the cause-effect relationship and usually have two or more groups with one independent variable, where correlational studies try to identify the relationship and have two or more variables, but one group. An example of causal comparative design would be "graduation rate for inner city students vs. suburban city students". When it comes to experimental studies the individuals are randomly selected where the independent variable can be manipulated. In casual comparative designs they are not randomly selected because they belong to groups with an independent variable that cannot be manipulated.

An advantage of the casual comparative design is that it allows for research of established groups. This allows for inferences to be made based on hypotheses.

IV. Statistical Analysis
Statistical approaches to quantitative research help deter potential threats to validity. A problem of error may occur during a researchers construction of a hypothesis, in which the researcher attempts to predict specific characteristics of or outcomes for a given population based on the study of a certain sample. Whenever a researcher makes such an inference about a population based on a sample, there is the possibility that the inference may be wrong or that there may be some individuals who are not reflective of the sample. Consequently, we have statistics to help make educated decisions regarding the validity of the research results, which are based on whether or not the hypothesis is correct.

The goal of these statistical tests is to make an inference about the hypothesis being tested and reach a conclusion about a larger population represented by the sample (339). Statistical analysis is geared towards a number that permits logical inference (a form of logic used in statistics that permits a conclusion about a population based on data collected from a sample) (340).


 * A. Descriptive Statistics**

Descriptive statistics describe the data that has been compiled from a study. The sample and measures of a study are summarized using descriptive statistics. Along with simple graphic analysis, descriptive statistics are the basis of quantitative analysis of data. Descriptive statistics simply describe what the data shows; in other words, what findings are indicated by the data. Descriptive statistics are used to present quantitative studies in a manageable form, which helps the data to be more easily understood. Examples of descriptive statistics include: race, gender, age, or socioeconomic status of study participants.

B. Inferential Statistics
Inferential statistics is a statistical reasoning that permits generalization beyond the sample to a larger population (Suter, 33). Most educational research studies deal with samples from larger populations, and inferential statistics allow researchers to determine whether or not the results that they got from a small sample can be applied to an entire population. The overall question being asked is: "how likely is it?" An example of inferential statistics can be seen if a positive behavior support plan is put in place in a classroom. After some time, if the plan is working for the students in one classroom, it is possible that it will work throughout the school building. However, making assumptions leave too much room for error. It is important to note that the results do not //prove// whether or not the results will occur in a larger population. They are meant to give the probability of that happening (Gay, Mills, & Airasian, 2007).

V. Summary
Quantitative research design falls into two main categories: experimental and non-experimental. Decisions regarding the type of quantitative research design have huge implications on how other researchers and readers of published research think about results. The types of research designs listed here are common varieties of design models, but hundreds of potentially useful designs exist. In general, experimental research designs study an intervention of some sort (Suter, 2006). Correlational studies let us draw conclusions about relationships, but true experimental designs (random assignment coupled with the use of manipulated independent variables-the intervention) let us draw conclusions about cause and effect. (Ormrod, 2000). Knowledge about teaching and learning should come from objective sources of information--psychological and educational research (Ormrod, 2000).