Evaluation of health interventions at area and organisation level

British Medical Journal, August 7, 1999 by Obioha C Ukoumunne, Martin C Gulliford, Susan Chinn, Jonathan A C Sterne, Peter G J Burney, Allan Donner

To allow for the correlation between subjects, the required standard sample size derived from formulas for individually randomised trials should be multiplied by a quantity known as the design effect or variance inflation factor.[2, 9] This will give a cluster level evaluation with the same power to detect a given intervention effect as a study with individual allocation. The design effect is estimated as

Deff = 1 ([n.sub.0]-1)[Rho]

where Deff is the design effect, [n.sub.0] is the average number of individuals per cluster and p is the intraclass correlation coefficient for the outcome of interest.

The intraclass correlation coefficient is the proportion of the total variation in the outcome that is between clusters; this measures the degree of similarity or correlation between subjects within the same cluster. The larger the intraclass correlation coefficient--that is, the more the tendency for subjects within a cluster to be similar--the greater the size of the design effect and the larger the additional number of subjects required in an organisation based evaluation, compared with an individual based evaluation.

Sample size calculations require the intraclass correlation coefficient to be known or estimated before the study is carried out.[12] If the intraclass coefficient is not available, plausible values must be estimated. A range of components of variance and intraclass correlations is reported elsewhere.[13, 14]

The number of clusters required for a study can be estimated by dividing the total number of individuals required by the average cluster size. When sampling of individuals within clusters is feasible, the power of the study may be increased either by increasing the number of individuals within clusters or by increasing the number of clusters. Increasing the number of clusters will usually enhance the generalisability of the study and will give greater flexibility at the time of analysis,[15] but the relative cost of increasing the number of clusters in the study, rather than the number of individuals within clusters, will also be an important consideration.

(6) Consider the use of matching or stratification of clusters where appropriate

Stratification entails assigning clusters to strata classified according to cluster level prognostic factors. Equal numbers of clusters are then allocated to each intervention group from within each stratum. Some stratification or matching will often be necessary in area based or organisation based evaluations because simple randomisation will not usually give balanced intervention groups when a small number of clusters is randomised. However, stratification is useful only when the stratifying factor is fairly strongly related to the outcome.

The simplest form of stratified design is the matched pairs design, in which each stratum contains just two clusters. We advise caution in the use of the matched pairs design for two reasons. Firstly, the range of analytical methods appropriate for the matched design is more limited than for studies which use unrestricted allocation or stratified designs in which several clusters are randomised to each intervention group within strata.[16] Secondly, when the number of clusters is less than about 20, a matched analysis may have less statistical power than an unmatched analysis.[17] If matching is thought to be essential at the design stage, an unmatched cluster level analysis is worth considering.[18] Stratified designs in which there are four or more clusters per stratum do not suffer from the limitations of the paired design.