Home Analysis of Professional Literature
Class 4:
QUANTITATIVE RESEARCH DESIGNS:
Experimental, Quasi-Experimental, & Non-Experimental
James Neill
Last updated:
03 Aug 2003

Introduction

Quantitative Research Designs

Factors Influencing Internal and External Validity

Cook & Campbell's Method for Depicting Research Designs

Recommended reading

Quantitative Exam


Introduction

 

Having developed your research question, identified a reading list, and planned the outline for your literature review, we are now going to move on to look at quantitative research design.  What you should keep in my mind is that there are two key, current applications for developing your knowledge about quantitative research design:

(i) to look at how your research question can be examined through a variety of different designs;

(ii) to understand and describe the research designs used in the empirical literature which you are reading for your literature review.

 

Some Keywords:

  • quantitative research

  • experimental

  • quasi-experimental

  • non-experimental (descriptive)

  • internal validity

  • external validity

  • pre-test

  • post-test

  • longitudinal

  • panel

  • causal-comparative

  • cross-sectional

  • ex post facto

  • trend

  • cohort

  • correlational

  • causal


Quantitative Research Designs:
Experimental, Quasi-Experimental, and Non-Experimental

 

Quantitative research is commonly used to investigate research questions.  There is a potentially infinite array of possible quantitative research designs, and in the human sciences particularly, it can be difficult to do pure, experimental research.  Thus a great many adaptations of experimental designs, called quasi-experimental and non-experimental designs have been developed.  It is important to consider a range of possible quantitative research designs and their strengths and weaknesses, before adopting any particular design.

 

Read these introductory descriptions of quantitative research designs.


Factors Jeopardizing Internal and External Validity
(based on Campbell and Stanley (1963)

 

Campbell and Stanley (1963) define internal validity as the basic requirements for an experiment to be interpretable - did the experiment make a difference in this instance?  External validity addresses the question of generalizability - to whom can we generalize this experiment's findings?

 

Eight extraneous variables can interfere with internal validity:

1. History, the specific events occurring between the first and second measurements in addition to the experimental variables

2. Maturation, processes within the participants as a function of the passage of time (not specific to particular events), e.g., growing older, hungrier, more tired, and so on.

3. Testing, the effects of taking a test upon the scores of a second testing.

4. Instrumentation, changes in calibration of a measurement tool or changes in the observers or scorers may produce changes in the obtained measurements.

5. Statistical regression, operating where groups have been selected on the basis of their extreme scores.

6. Selection, biases resulting from differential selection of respondents for the comparison groups.

7. Experimental mortality, or differential loss of respondents from the comparison groups.

8. Selection-maturation interaction, etc. e.g., in multiple-group quasi-experimental designs

 

Four factors jeopardizing external validity or representativeness are:

9. Reactive or interaction effect of testing, a pretest might increase

10. Interaction effects of selection biases and the experimental variable.

11. Reactive effects of experimental arrangements, which would preclude generalization about the effect of the experimental variable upon persons being exposed to it in non-experimental settings

12. Multiple-treatment interference, where effects of earlier treatments are not erasable.


Cook & Campbell's Method for Depicting Research Designs

  • X = exposure of a group to an experimental variable or event, the effects of which are to be measured

  • O = some process of observation or measurement

  • Xs and 0s on the same line are applied to the same specified persons

  • Read left to right to indicate temporal order of events (Xs and Os vertical to one another occur simultaneously)

  • R = randomized selection/allocation


Next week in class you will be asked to present one, two, or three research designs for the class discussion.  You can use this presentation to help improve the quality of your answers for the exam which is due April 9.


Recommended Reading

Gall, M. D., Borg, W. R., Gall, J. P. (2003).  Educational research: An introduction. (7th Edition). White Plains, New York: Longman. Recommended: Skim read Part IV "Quantitative Research Design" (Chapters 10-13) (pp. 287 - 431).