MAIN POINTS

Classic Experimental Research Design

The classic research design consists of two comparable groups: an experimental group and a control group. Randomization ensures the comparability of these groups. Both groups are pretested on the dependent variable; then the experimental group is exposed to the independent variable. Finally, a posttest of the dependent variable is conducted so that the groups can be compared. If the experimental group displays pretest posttest differences that are unlike those of the control group, then strong evidence exists that the independent variable affects the dependent variable. In general, social scientists use the experiment less widely than natural scientists, primarily because its rigid structure often cannot be adapted to social sciences research. Thus, social scientists frequently use designs that are weaker for drawing causal inferences but are more appropriate to the type of problems they examine.

Causal Inferences

In practice, the demonstration of causality involves three distinct operations: demonstrating covariation, which means that the variables involved are associated or correlated; eliminating spurious relations, meaning that the relationship cannot be explained by some third factor; and establishing the time order of the occurrences, meaning that the independent variable actually occurred before the dependent variable.

Components of a Research Design

The classic research design consists of three components: comparison, manipulation, and control. To the extent that other factors can be ruled out as rival explanations of the observed association between the variables under investigation, the results have internal validity.

Extrinsic factors that threaten internal validity are selection factors: initial differences between experimental and control groups that may account for the differences observed in the dependent variable. Intrinsic factors that threaten internal validity include changes in the individuals or the units studied that occur during the study period, changes in the measuring instrument, and the reactive effect of the observation itself. Major threats to internal validity are history, maturation, experimental mortality, instrumentation, testing, regression artifact, and interactions with selection. Extrinsic and intrinsic factors that threaten the internal validity of causal inferences may be controlled by three procedures: matching, randomization, and the use of control groups.

The external validity of research designs consists of the generalizability of research findings. The two main issues of external validity are the representativeness of the sample and reactive arrangements in the research procedure.

Extrinsic and intrinsic factors that threaten the internal validity of causal inferences may be controlled by several procedures. Scientists employ two methods of control to counteract the effect of extrinsic factors. The first, matching, controls for variables that are known to the investigator prior to the research operation. The second, randomization, helps to offset the effect of unforeseen factors. Using a control group helps counteract the effects of intrinsic factors.

Researchers control intrinsic factors by using a control group from which they withhold the experimental stimulus. Ideally, the control and experimental groups have been selected by matching or randomly so that they will have similar characteristics. The groups also experience identical conditions during the study except for their differential exposure to the independent variable. Thus, features of the experimental situation or external events that occur during the experiment are likely to influence the two groups equally and will not be confounded with the effect of the independent variable.

Most research is concerned not only with the effect of one variable on another in the particular setting studied but also with its effect in other natural settings and on larger populations. This concern is termed the external validity of research designs. The two main issues of external validity are the representativeness of the sample and the reactive arrangements in the research procedure.

External validity can be compromised when the experimental setting or the experimental situation does not reflect the natural setting or situation to which researchers wish to generalize. When a study is carried out in a highly artificial and planned situation, such as a laboratory, features of the setting might influence the individuals' responses.

Design Types

Four major types of research designs can be distinguished: experimental, quasi experimental, cross-sectional, and preexperimental.

The classic experimental design allows for pretest, posttest, and control group/experimental group comparisons; it permits the manipulation of the independent variable and thus the determination of the time sequence; and, most significantly, by including randomized groups, it controls for most sources of internal validity. However, this design is generally weak on external validity and does not allow for generalizations to be made to nontested populations. There are two variations of this design that are stronger in this respect.

The Solomon four group design yields results that are more generalizable than the classic experimental design. It involves the same features as the classic design plus an additional set of control and experimental groups that are not pretested, thus ruling out sensitization effects.

The posttest only control group design is a variation of both the classic design and the Solomon design; it omits the pretested groups altogether.

Factorial designs are more complicated than the classic experimental design and its variations, but they enable the researcher to study the simultaneous effects of two or more independent variables. The main advantage of factorial designs is that the generalizability of research findings is greatly increased. Another advantage is that this design allows for the systematic assessment of how two or more independent variables interact.