Identifying the best research design to fit the question. Part 1: quantitative designs
Evidence-based nursing is about applying the best available evidence to a specific clinical question. Different clinical questions require evidence from different research designs. No single design has precedence over another, rather the design chosen must fit the particular research question.1 Questions focused on the cause, prognosis (course), diagnosis, prevention, treatment, or economics of health problems are best answered using quantitative designs, whereas questions about the meaning or experience of illness are best answered using qualitative designs. Many different quantitative and qualitative research designs exist, each with a specific purpose and with strengths and limitations. In this editorial, the most rigorous quantitative designs to address questions of prevention or treatment, causation, and prognosis will be outlined. The next editorial will describe the use of qualitative designs to address questions of meaning or experience.
Questions about the effectiveness of prevention and treatment interventions
The randomised controlled trial (RCT) is the strongest design for questions of whether healthcare interventions are beneficial (ie, do more good than harm). An RCT is a true experiment in which people are randomly allocated to receive a new intervention (experimental group) or to receive a conventional intervention or no intervention at all (control group). Because it is the play of chance alone that determines the allocation, the only systematic difference between the groups should be the intervention. Investigators follow participants forward in time (follow up) and then assess whether they have experienced a specific outcome (fig 1). The 2 most important strengths of RCTs are (1) the random allocation of participants to groups, which helps to ensure that the groups are similar in all respects except exposure to the intervention, and (2) the longitudinal nature of the study, whereby exposure to the intervention precedes the development of the outcome. These 2 features ensure that any differences in outcome can be attributed to the intervention. The disadvantages of this design include the cost of conducting a trial, the long period of follow up before patients experience the outcome, and the possibility that patients who agree to participate in a trial may differ from those to whom the results would be applied (generalisability).
If you are a school nurse who wants to find an effective intervention to prevent the initiation of smoking among adolescents, you should look for evidence from RCTs. An example of a trial is one in which schools are randomly allocated to the experimental group to receive an innovative intervention which is taught in small groups and allows young adolescents the opportunity to practise smoking avoidance behaviours, whereas those allocated to the control group receive traditional lectures about the ill effects of smoking. The students are followed up for several years and data are collected and compared on the number of students in each group who begin to smoke. In most RCTs, individuals are randomly allocated to groups. In this study, schools are the unit of randomisation to reduce the likelihood of students discussing their experiences of the intervention with students in the control group. To avoid such contamination, investigators will often randomise to units such as classrooms, schools, or communities.
If you are a primary care nurse practitioner and are wondering whether you should suggest nicotine gum to help smokers to stop smoking, again you should look for evidence from RCTs. In such trials, smokers are randomly allocated to nicotine gum (experimental group) or to placebo gum, which looks and tastes like nicotine gum but contains no active ingredients (control group). They are then followed up, and data are collected and compared on the number of participants in each group who stopped smoking.
Readers of this journal will notice that all of the treatment studies that have been abstracted have been RCTs. An example from this issue is the study by Macknin (p13) on the effectiveness of zinc lozenges for cold symptoms in children. There are occasions, however, when the evaluation of an intervention using an RCT is not ethical or feasible. In this case, we must rely on a less rigorous design such as the cohort analytic study (also known as a controlled trial). This study design is similar to the RCT in that there are comparison groups who receive and do not receive an intervention and they are followed up to determine who experiences the outcome of interest. The important difference between the 2 designs is the absence of random allocation to study groups; instead, participants most often select themselves or are selected by a clinician to receive the intervention (fig 2). This is an important limitation because groups may differ in ways other than exposure to the intervention. Group differences in outcomes at the end of the study may be because of differences in the groups that existed before the intervention began (baseline differences). The intervention therefore may appear to have had an effect on the outcome when, in fact, it was the initial differences in groups that influenced the outcome.
Continuing with the example of the school based smoking prevention programme, if schools were unhappy about being randomly allocated to groups, a cohort analytic study might be done rather than an RCT. In such a study, several schools are approached and asked if they would like to receive the innovative intervention (intervention group) or the traditional lectures (control group). Without random allocation to groups, it may be that schools that choose the innovative intervention differ from those that choose not to receive it in ways that may influence the outcome (eg, socioeconomic status or parental smoking habits). The study findings may show that students who received the innovative intervention were less likely to start smoking; however, this outcome may have been influenced by the baseline characteristics of the group rather than, or as well as, the intervention. In other words, this group may not have been as likely to smoke even if they did not receive the innovative intervention. Even if investigators document the group differences in baseline characteristics or use statistical techniques to adjust for the differences, other factors that were not considered may be responsible for differences in outcome.
Using a cohort analytic design for the nicotine gum example, smokers who want to stop smoking are offered nicotine gum and those who choose to take it (intervention group) are compared over time with those who choose not to take it (control group). Again, the major limitation of this design is that smokers who choose to take nicotine gum may differ from those who choose not to take it with respect to known and unknown baseline characteristics that may influence the outcome. Nicotine gum may appear to increase quit rates when, in fact, the increase may be because of variables such as higher motivation to stop smoking, younger age, or fewer years of smoking.
Questions about the cause of a health problem or disease
The RCT is the most rigorous design to determine whether some factor (exposure) causes an outcome. Using this design, participants are randomly allocated to be or not be exposed to a potential causative agent and then followed up to compare the number in each group who experience the outcome. In questions of causation, however, it may not be ethical or feasible to randomly allocate people to exposure to the causative agent. The next best evidence comes from cohort analytic studies. In this design, the investigator follows up people who are exposed to and not exposed to a causative agent. The major strength of RCTs and cohort analytic studies is that those entering the study have not yet experienced the outcome. The investigator is certain therefore that exposure to the causative agent (smoking) precedes the development of the outcome (lung cancer). This issue of temporality—that is, that the causative agent precedes the development of the outcome, is crucial for establishing a causal relation.2
When the outcome of interest is rare or takes a long time to develop, neither RCTs nor cohort analytic studies may be feasible. In these circumstances, a case control design is often used. In a case control design, patients with the outcome of interest (cases) and patients without the outcome of interest (controls) are identified and then the investigator determines whether they have had previous exposure to the causative agent (fig 3). The investigator is able to match the case and control patients on important variables that may influence the outcome (eg, age, sex, and other health conditions). In this way, the groups are as similar as possible and the specific effect of the causative agent on the outcome can be more confidently examined. The strengths of this design are that it allows the assessment of causation when the outcome is rare or takes a long time to develop, and that it includes a control group. The limitations are the difficulties in establishing that the exposure actually occurred before the outcome (temporality), obtaining accurate information about exposure to a causative agent which has occurred in the past (relies on accuracy of people's memory or on completeness and accuracy of medical records), and identifying a control group that is similar in all other factors that may have influenced the outcome.
When considering whether smoking causes lung cancer, it is unethical to randomise participants to smoke or to not smoke and then follow them up to determine who develops lung cancer; an RCT is therefore not possible. Cohort analytic studies have been done in which investigators followed up a group of smokers and a group of non-smokers, who were matched for as many other explanatory variables as possible. However, given the long time it takes before lung cancer develops, these studies took many years to complete. Case control studies are often a more feasible option when the disease in question is rare and when a cohort analytic study would be extremely large and costly to identify a sufficient number of people who develop the disease. Using a case control design for the smoking and lung cancer example, people with lung cancer are matched, for several important variables, to people without lung cancer. All participants are asked about their past smoking behaviour, and the number of smokers in each group are compared to see whether those with lung cancer were more likely to have smoked. An example of a case control study that examines a causal relation in this issue is the study by Woodward et al (p25) on whether parenting style increases hyperactivity in school aged boys.
Questions about the course of a health state or disease (prognosis)
When we are interested in the likelihood that people will experience or develop an outcome given their exposure to a disease, condition, or situation, the best design is a cohort study. An example of a prognosis question that appeared in a previous issue of Evidence-Based Nursing is whether infants who are preterm and small for gestational age are likely to have cognitive and motor delays during early childhood (Hutton et al, Jan 1998 issue, p 19). In this example, the condition is being preterm and small for gestational age. An example of a prognosis question that begins with a disease is how likely are patients with ulcerative colitis to develop bowel cancer? In a cohort study, one group of patients who are at a uniform point in the development or exposure of the disease or condition (eg, at first onset or at initial diagnosis) (inception cohort) and free of the outcome of interest are followed up to determine who develops the outcome (fig 4).
In the case of lung cancer, an example of a prognosis study is to assemble a group of patients who have just been diagnosed with lung cancer and follow them up to determine when certain symptoms appear or how long patients survive.
The longitudinal nature of the cohort design ensures that the disease, condition, or situation precedes the outcome. The disadvantages of this design are the expense and time required to follow up large numbers of patients until some develop the outcome.
Single studies versus systematic reviews
Questions about the prevention or treatment of disease, and the cause or course of disease are usually addressed by more than one study. To help practitioners and health policy makers keep up with the literature related to specific topics, systematic reviews of the literature are conducted. In a systematic review, eligible research studies are viewed as a population to be systematically sampled and surveyed. Individual study characteristics and results are then abstracted, quantified, coded, and assembled into a database that, if appropriate, is statistically analysed much like other quantitative data. The statistical combination of the results of >1 study, or meta-analysis, effectively increases the sample size and results in a more precise estimate of effect than can be obtained from any of the individual studies used in the meta-analysis. Rigorous systematic reviews provide a summary of all methodologically sound studies related to a specific topic. This is much more powerful than the results of any one single study and avoids the potential of bias if one only looks at some of the studies on a particular question.3 An example of a systematic review in this issue is the study by Tang (p18) about whether an intervention of dietary advice lowers blood cholesterol concentrations in people living in the community.
There are numerous quantitative research designs. The designs described above are the most rigorous for addressing questions about treatment, causation, and prognosis. When looking for studies related to these questions, awareness of these designs will help nurses to identify those studies and systematic reviews worth reading.