ICCSF MACROS The ICCSF_.SPS macros implement the calculations for intraclass correlations and related statistics for assessing interrater reliability described in Shrout and Fleiss' 1979 Psychological Bulletin article (Vol. 86, No. 2, pp. 420-428). The models are as follows: In all three models, each of the N subjects/objects is rated by K raters. Model 1: Raters are a random sample from a specified population of raters, and each rater does not rate all subjects/objects. Therefore, each subject/object is rated by a potentially different set of raters. Model 2: Raters are a random sample from a specified population of raters, and each rater rates each subject/object. Model 3: Raters constitute the entire population of raters, and each rates each subject/object. Data structure: In each case, the data should be in a standard SPSS file format, with subjects/objects as cases (rows) and ratings as variables (columns). For models 2 and 3, each variable (column) will be a specific rater. For model 1, the placement of rating values among columns in each row is arbitrary, as the columns do not correspond to specific raters. Cases without full sets of ratings will be deleted from the file before calculating statistics. The three primary macros (ICCSF1.SPS, ICCSF2.SPS and ICCSF3.SPS) require an SPSS release of 6.0 or higher, including the MATRIX procedure, which will generally imply that you must have the Advanced Statistics module. The three abbreviated versions (ICCSF1A.SPS, ICCSF2A.SPS and ICCSF3A.SPS) require only a version of SPSS with macro capability and the MATRIX procedure. These are designed to produce the basic results (ICC estimates for individual rating and mean of K ratings and test of 0 population value) on systems prior to release 6.0, which do not offer the IDF functions used in the computation of confidence intervals. (Note that the test of 0 population value is the same for both single rating and mean of K rating ICC values.) The six macros produce the following results: ICCSF1.SPS--Full results for model 1: ICC estimate for a single rater, 95% confidence interval for the single rater estimate, F-test of null hypothesis of 0 population value, ICC estimate for the mean of K raters, 95% confidence interval for the mean of raters estimate and number of raters required to produce a lower bound on the 95% CI for the mean of raters estimate greater than or equal to .7, .8 and .9. ICCSF1A.SPS--Abbreviated results for model 1: ICC estimate for a single rater, ICC estimate for the mean of K raters and F-test of null hypothesis of 0 population value. ICCSF2.SPS--Full results for model 2: ICC estimate for a single rater, 95% confidence interval for the single rater estimate, F-test of null hypothesis of 0 population value, ICC estimate for the mean of K raters, approximate 95% confidence interval for the mean of raters estimate using Satterthwaite approximation for degrees of freedom and number of raters required to produce a lower bound on the 95% CI for the mean of raters estimate greater than or equal to .7, .8 and .9. ICCSF2A.SPS--Abbreviated results for model 2: ICC estimate for a single rater, ICC estimate for the mean of K raters and F-test of null hypothesis of 0 population value. ICCSF3.SPS--Full results for model 3: ICC estimate for a single rater, 95% confidence interval for the single rater estimate, F-test of null hypothesis of 0 population value, ICC estimate for the mean of K raters and 95% confidence interval for the mean of raters estimate. ICCSF3A.SPS--Abbreviated results for model 3: ICC estimate for a single rater, ICC estimate for the mean of K raters and F-test of null hypothesis of 0 population value. For details on derivation of statistics and interpretation, see the original Shrout and Fleiss article. The ICC estimates produced are ratios of unbiased estimators; they are biased but consistent estimates of the population values. A negative lower bound for the individual rating estimate confidence interval will result in missing values for the lower bound on the confidence interval for the mean of ratings estimate and the number of raters required for the lower bound to achieve values of .7, .8 and .9. A negative upper bound for the individual rating estimate will produce a missing value for the upper bound on the confidence interval for the mean of ratings estimate. An ICCSF_.SPS macro is most easily used by simply having it resident as a text file in your working directory and executing the following SPSS syntax: INCLUDE 'ICCSF_.SPS' . ICC VARS=varlist . Of course, you replace the underscore in the above syntax with the appropriate macro number (and letter, if relevant) designator. For simplicity, all six macros use the same syntax, and all create the same temporary files in order to prevent proliferation of temporary files. Upper or lower case text is acceptable. "VARS=" must be specified. Variables may be listed out, or the "TO" convention may be used. The macro first saves your working data file to a file named ic__tmp1.sav. This file is retrieved at the end of the macro run so that the macro may be used like a procedure in command syntax. The double underscore in the file name is an attempt to render unlikely the overwriting of an existing file. The same convention is used in creation of new variables. It is thus a good idea to avoid the use of double__underscores in file and variable names. The SET commands are used to minimize output; they may be changed or removed if you have problems running the macro in order to aid in identification of problem sources. A second temporary file, ic__tmp2.sav is also created. Multiple invocations of ICCSF macros in a single interactive session will produce the following warning on all runs after the first: >The macro name specified on the DEFINE command duplicates the name of a >previously defined macro. This instance will take precedence. This warning does not indicate a problem and may be ignored.