Impact of BPRS Interview Length on Ratings Precision in a Schizophrenia Trial

Poster Session Presented at: Annual American Society of Clinical Psychopharmacology Meeting, June 16th-19th, 2014, Hollywood FL.

Steven Targum, Chelsea Toner, J. Cara Pendergrass, Laura Zumpano, Philip Rauh, and Nick deMartinis

Background: Ratings precision is necessary to optimize signal detection in clinical trials, and is particularly important in studies that rely on subjective ratings. The ability of a second independent rater to replicate the scores given by the primary rater is one metric of ratings precision. We assessed ratings precision in a schizophrenia study using site-independent raters to blindly score audio-digital recordings of site-based interviews.

Methods: We examined ratings precision in a 12-week, double-blind, parallel-group study of PF-02545920 compared to placebo in patients with sub-optimally controlled symptoms of schizophrenia ( identifier NCT01939548). The study is currently being conducted at 26 trial centers in the United States. All patients consented to audio-digital pen recording of site-based interviews using the Brief Psychiatric Rating Scale (BPRS). Recorded interviews were electronically transmitted to Clintara LLC (Boston MA) via a secure website and distributed to 5 site-independent reviewers. These reviewers were blinded to the study site and visit and scored the BPRS based upon the audio recording and corroborative digital information they received. We analyzed “dual” ratings of the total BPRS score and the impact of BPRS interview length on scoring variability and discordance. Statistical analysis included intra-class correlation and Student’s t test.

Results: 392 BPRS interviews conducted at the screen and stabilization visits were recorded for “dual” scoring. The mean total BPRS scores were 50.4 ± 7.6 (SD) for the site-based raters and 49.2 ± 7.2 for the site-independent reviewer/raters (t= 2.34; p= 0.025). The total BPRS scores of the paired “dual” site-independent raters were highly correlated with the site-based scores (r=0.812), and the discordance rate (total score difference between site-based and site-independent raters ≥ 8 points) was less than 10% for all BPRS interviews.
Interview length significantly affected scoring discordance between site-based and site-independent ratings. The mean BPRS interview length was 21:05 ± 7:47 minutes ranging from 7 to 59 minutes. 89 interviews (22.7%) were conducted in less than 15 minutes. These “shorter” interviews yielded significantly greater “dual” scoring variability (p= 0.002) and absolute discordance (p=0.004) between site-based and site-independent raters than all other interviews.

Conclusion: Overall, “dual” scoring of the site-based BPRS interviews revealed a high correlation and minimal scoring discordance between site-based and blinded, site-independent raters. However, interview length had a significant impact on “dual” ratings concordance such that “shorter” interviews (< 15 minutes) were significantly more discordant than all other interviews in the sample. These data suggest that ratings precision may be compromised by short, incomplete interviews and may adversely affect the study outcome.