Explanation for the critical appraisal tool for RCTs with individual participants in parallel groups

How to cite: Tufanaru C, Munn Z, Aromataris E, Campbell J, Hopp L. Chapter 3: Systematic reviews of effectiveness. In: Aromataris E, Munn Z (Editors). JBI Manual for Evidence Synthesis. JBI, 2020. Available from https://synthesismanual.jbi.global

Answers: Yes, No, Unclear or Not/Applicable

Critical Appraisal Tool for RCTs (individual participants in parallel groups)

1. Was true randomization used for the assignment of participants to treatment groups?

The differences between participants included in compared groups constitute a threat to the internal validity of a study exploring causal relationships. If participants are not allocated to treatment and control groups by random assignment there is a risk that the allocation is influenced by the known characteristics of the participants and these differences between the groups may distort the comparability of the groups. A true random assignment of participants to the groups means that a procedure is used that allocates the participants to groups purely based on chance, not influenced by the known characteristics of the participants. Check the details about the randomization procedure used for allocation of the participants to study groups. Was a true chance (random) procedure used? For example, was a list of random numbers used? Was a computer-generated list of random numbers used?

2. Was allocation to groups concealed?

If those allocating participants to the compared groups are aware of which group is next in the allocation process, that is, treatment or control, there is a risk that they may deliberately and purposefully intervene in the allocation of patients by preferentially allocating patients to the treatment group or to the control group and therefore this may distort the implementation of allocation process indicated by the randomization and therefore the results of the study may be distorted. Concealment of allocation (allocation concealment) refers to procedures that prevent those allocating patients from knowing before allocation which treatment or control is next in the allocation process. Check the details about the procedure used for allocation concealment. Was an appropriate allocation concealment procedure used? For example, was central randomization used? Were sequentially numbered, opaque and sealed envelopes used? Were coded drug packs used?

3. Were treatment groups similar at the baseline?

The differences between participants included in compared groups constitute a threat to the internal validity of a study exploring causal relationships. If there are differences between participants included in compared groups there is a risk of selection bias. If there are differences between participants included in the compared groups maybe the ‘effect’ cannot

be attributed to the potential ‘cause’ (the examined intervention or treatment), as maybe it is plausible that the ‘effect’ may be explained by the differences between participants, that is, by selection bias. Check the characteristics reported for participants. Are the participants from the compared groups similar with regards to the characteristics that may explain the effect even in the absence of the ‘cause’, for example,  age, severity of the disease, stage of the disease, co-existing conditions and so on? Check the proportions of participants with specific relevant characteristics in the compared groups. Check the means of relevant measurements in the compared groups (pain scores; anxiety scores; etc.). [Note: Do NOT only consider the P-value for the statistical testing of the differences between groups with regards to the baseline characteristics.]