(A) Are the results of the study valid?
Screening Questions
1. Is the CPR clearly defined?
HINT:
- The type of patients to whom the CPR will be applied is clearly defined?
- The variables included in the rule are clearly defined?
- Is the outcome relevant and is it clinically reasonable? (The outcome can be expressed as a probablility or as course of action)
Yes Can’t tell No
2. The population from which the rule was derived included an appropriate spectrum of patients?
HINT: Consider
- Is it adequate the way the patients were selected?
- The spectrum of patient to whom the rule will apply is well represented?
Yes Can’t tell No
3. Was the rule validated in a different group of patients?
HINT:
- It is not good enough that the rule had a good performance on the patient group used to derive it. The rule should be validated in a set of patients different from those who served to derive the rule
- The validation was done in a group of patients similar to the one used to derive it
Yes Can’t tell No
Is it worth continuing?
Detailed questions
4. Were the predictor variables and the Yes Can’t tell No outcome evaluated in a blinded fashion?
HINT: Did;
- People evaluating the outcome know the predictor variables?
- People evaluating the predictor variables
know the outcome
Yes Can’t tell No
5. Were the predictor variables and the outcome evaluates in the whole sample
selected initially?
HINT:
- Are exclusions and drop outs well described and do the authors discuss the reasons for them?
- Sometimes the outcome cannot be measured in the same way in all patients
Yes Can’t tell No
6. Are the statistical methods used to construct and validate the rule clearly
described?
HINT:
- Were all important variables included and the positivity criteria explained?
- The statistical method is adequately described?
- Was the reliability of the rule considered?
Yes Can’t tell No
(B) What are the results?
7. Can the performance of the rule be calculated?
HINT: Rule –
| Outcome + | Outcome – | |
|---|---|---|
| Rule + | a | b |
| Rule – | c | d |
- Performance results can be presented as: Sens, Sp, + LR, -LR, ROC curve, calibration curves etc
- Sensitivity = a/(a+c)
- Specificity = d/(b+d)
- LR+ = sens/(1-sp)
- LR- = (1-sens)/sp
Yes Can’t tell No
8. How precise was the estimate of the treatment effect?
Did they try to refine the rule with other variables to see whether the precision could be improved or the rule simplified?
HINT: Think about
- The sample size and the number of Variables included in the CPR
- Is the rule robust? Has there been Any attempt to refine it?
Yes Can’t tell No
(C) Will the results help locally? / Are the findings applicable to the scenario?
9. Would the prediction rule be reliable and and the results interpretable if used for
your patient?
HINT: Consider
- Is your setting too different from that of the study?
Yes Can’t tell No
10. Is the rule acceptable in your case?
HINT: Consider
- The ease of use and the availability of the rule
and the costs - If the rule is reasonable from a clinical point
of view
Yes Can’t tell No
11. Would the results of the rule modify your decision about the management
of the patient or the information you can give to him/her?
HINT: Consider
- In addition to your opinion, might there be studies analysing the impact (in monetary terms or health results) of the rule?
- If nothing will change, the rule is at best useless in terms of benefit to the patients.
- How the initial estimation has changed after applying the rule, and the effect it has had on the action threshold.
Yes Can’t tell No