Quantitative Research Design:
Sampling & Measurement
21 Feb 2003
In the previous class, we examined:
In the practical exercise (Quantitative Exam Task 1 - 5%) you developed three quantitative research designs (experimental, quasi-experimental, and non-experimental) for your research question, and considered the strengths and weaknesses, as well as practical issues, for each of these designs .
Now it is time to deal with two more important aspects of quantitative reserach design:
There are a wide range of possible options to consider when sampling. At all times, the purpose of the study needs to borne in mind and the various strengths of weaknesses, as as the practicality, of different sampling methods need to be weighed.
Sampling involves selecting individual units to measure from a larger population. The population refers to the set of individual units which the research question seeks to find out about. For example, my population of interest may be adolescent females in the United States. In particular, I may be interested to know whether socio-economic status influences American adolescent females' mental health. There are a variety of ways I could consider to obtain a representative sample. A sample is representative when it allows the results of the sample to be generalized to the population.
The sampling frame is the group of individuals who had a real chance of being selected for the sample. For example, if I use as my sampling frame the lists of students held by public and private schools in America from which to select a sample of adolescent females, then only students on those lists have a real chance of being selected. This may differ from the population to which I wish to generalize the results of my study (all adolescent females in the United States). In this case, my sample will almost certainly be bias because adolescent females with poor mental and with lower socio-economic status are probably less likely to be on school lists than other students.
As you can see from this example, the process of sampling, even when done systematically, can introduce potentially critical biases into a research study. Due our bias sampling technique we may enhance the risk of incorrectly concluding that there is no relationship between socioeconomic status and the mental health of adolescent females in the United States because we did representatively sample from adolescent females who weren't on school registers.
There are two main types of sampling - the key is whether or not the selection involves randomization. Randomization means that each unit within a sampling frame has an equal chance of being selected. By selecting randomly from a sampling frame, probability theory says that our sample, more often than not, should approximately represent the whole population. You can read more about sampling terminology and probability theory in social science research on Bill Trochim's Sampling Terminology page.
Probability sampling involves the use of randomization. These are the main types of probability sampling:
Read Bill Trochim's Probability Sampling for more detailed explanations of the random (probability) sampling methods.
Non-probability sampling does not involve the use of randomization. Therefore, to be considered representative, non-probability sampling methods cannot rely on the theory of probability (random theory). We can also use purposive or even accidental, haphazard, or convenience sampling to get a representative sample relying on other techniques than randomization.
In quantitative research, you end up with "numbers" after carrying out your research. These are analyzed, and then interpreted in light of the research question and other relevant theory and research findings.
In order to create the "numbers" for quantitative research (data), a measurement process needs to take place. In other words, you need to convert some human phenomenon (in the human sciences) accurately into numerical data.
The process of converting phenomena into data is called "measurement".
In the human sciences, much of what we try to measure is subjective (e.g., even concepts like "physical fitness" have fuzzy definitions). As a result, measurement becomes a difficult and complex issue, and noise is always created in the data due to inaccuracies in the process of measurement. Thus, it is vital to minimize noise in by using reliable and valid methods of measurement.
Avoid falling prey to "availability bias", which means using whatever measurement tools are on hand, or simply those which others have used previously. All instruments contain noise, and many contain a lot of noise. So, conduct a systematic search to identify possible instruments, evaluating each according to relevant criteria, such as: