(A) Are the results of the study valid?
Screening Questions
1. Did the study address a clearly focused issue?
HINT: A question can be ‘focused’ In terms of
- The population studied
- The risk factors studied
- The outcomes considered
- Is it clear whether the study tried to detect a beneficial
or harmful effect?
Yes Can’t tell No
2. Was the cohort recruited in an acceptable way?
HINT: Look for selection bias which might compromise the generalisability of the findings:
- Was the cohort representative of a defined population?
- Was there something special about the cohort?
- Was everybody included who should have been included?
Yes Can’t tell No
Is it worth continuing?
Detailed questions
3. Was the exposure accurately measured to minimise bias?
HINT: Look for measurement or classification bias:
- Did they use subjective or objective measurements?
- Do the measurements truly reflect what you want them to (have they been validated)
- Were all the subjects classified into exposure groups using the same procedure
Yes Can’t tell No
4. Was the outcome accurately measured to minimise bias?
HINT: Look for measurement or classification bias:
- Did they use subjective or objective measurements?
- Do the measures truly reflect what you want them to (have they been validated)?
- Has a reliable system been established for detecting all the cases (for measuring disease
occurrence)? - Were the measurement methods similar in the different groups?
- Were the subjects and/or the outcome assessor blinded to exposure (does this matter)?
Yes Can’t tell No
5. (a) Have the authors identified all important confounding factors?
List the ones you think might be
important, that the author missed.
Yes Can’t tell No
(b) Have they taken account of the confounding factors in the design and/or analysis?
HINT: Look for restriction in design, and techniques e.g. modelling, stratified-, regression-, or sensitivity analysis to correct, control or adjust for confounding factors
Yes Can’t tell No
6. (a) Was the follow up of subjects complete enough?
Yes Can’t tell No
(b) Was the follow up of subjects long enough?
HINT: Consider
- The good or bad effects should have had long enough to reveal themselves
- The persons that are lost to follow-up may have different outcomes than those available for assessment
- In an open or dynamic cohort, was there anything special about the outcome of the people leaving, or the exposure of the people entering the cohort?
Yes Can’t tell No
(B) What are the results?
7. What are the results of this study?
HINT: Consider
- What are the bottom line results?
- Have they reported the rate or the proportion between the exposed/unexposed, the ratio/the rate difference?
- How strong is the association between exposure and outcome (RR,)?
- What is the absolute risk reduction (ARR)?
Yes Can’t tell No
8. How precise are the results?
HINT: Look for the range of the confidence intervals, if given.
Yes Can’t tell No
9. Do you believe the results?
HINT: Consider
- Big effect is hard to ignore!
- Can it be due to bias, chance or confounding?
- Are the design and methods of this study sufficiently flawed to make the results unreliable?
- Bradford Hills criteria (e.g. time sequence, dose-response gradient, biological plausibility, consistency)
Yes Can’t tell No
(C) Will the results help locally?
10. Can the results be applied to the local population?
HINT: Consider whether
- A cohort study was the appropriate method to answer this question
- The subjects covered in this study could be sufficiently different from your population to cause concern
- Your local setting is likely to differ much from that of the study
- You can quantify the local benefits and harms
Yes Can’t tell No
11. Do the results of this study fit with other available evidence?
Yes Can’t tell No
12. What are the implications of this study for practice?
HINT: Consider
- One observational study rarely provides sufficiently robust
evidence to recommend changes to clinical practice or
within health policy decision making - For certain questions observational studies provide the only
evidence - Recommendations from observational studies are always stronger
when supported by other evidence
Yes Can’t tell No