Evid Based Nurs 3:36-39 doi:10.1136/ebn.3.2.36

Estimating treatment effects: real or the result of chance?

  1. Trevor A Sheldon, DSc
  1. Department of Health Studies University of York, York, UK

      The Notebook in the January 2000 issue of Evidence-Based Nursing described how the outcomes of clinical trials are measured and summarised before analysis. We now discuss how we can tell, by using and interpreting statistical tests, if treatments have a real effect on health or if the apparent effects of treatments under trial are a result of chance.

      When critically reading a report of a clinical trial, one of the things we are interested in is whether the results of the study provide an accurate estimate of the true treatment effect in the type of patients included in the study.

      Sampling error

      Even if a study has been carried out in a methodologically sound (unbiased) way, a study result such as “5% more wounds healed in the treatment compared with the control group” does not necessarily mean that this is a true treatment effect. This finding could be a chance occurrence even when there is no true effect. To illustrate this, imagine that you are playing a game with dice. We know that, on average, each of the 6 numbers should come up an equal number of times in unbiased dice. However, when your friend throws 2 or even 3 sixes in a row, you are unlikely (depending on the friend) to infer that the dice are loaded (biased) or that he or she is cheating. Instead, you would probably conclude that this was just luck. This example shows us that even if there is no true effect (ie, the dice are not loaded), we can observe events that look like there is an effect, simply because of chance (sampling error). This is particularly the case when there are small numbers of observations. For example, if the number six came up in 2 out of 4 throws (ie, 50% of the …

      Responses to this article

      No Related Web Pages

      Free Sample

      This recent issue is free to all users to allow everyone the opportunity to see the full scope and typical content of EBN.
      View free sample issue >>

      EBN Journal Chat

      The EBN Journal Chat offers readers the opportunity to participate in discussion about research articles and commentaries from Evidence Based Nursing (EBN).

      How to participate >>

      Don't forget to sign up for content alerts so you keep up to date with all the articles as they are published.

      Navigate This Article