Increasing reliability and scoring accuracy for CNS trials

“A structured interview guide for global impressions: increasing reliability and scoring accuracy for CNS trials.” Ann Gen Psychiatry. 12(1) 2, 2013.

Targum SD, Houser C, Northcutt J, Little JA, Cutler AJ, Walling DP.

Background: The clinical global impression of severity (CGI-S) scale is a frequently used rating instrument for the assessment of global severity of illness in Central Nervous System (CNS) trials. Although scoring guidelines have been proposed to anchor these scores, the collection of sufficient documentation to support the derived score is not part of any standardized interview procedure. It is self evident that the absence of a standardized, documentary format can affect inter-rater reliability and may adversely affect the accuracy of the resulting data.

Method: We developed a structured interview guide for global impressions (SIGGI) and evaluated the instrument in a 2-visit study of ambulatory patients with Major Depressive Disorder (MDD) or schizophrenia. Blinded, site-independent raters listened to audio recorded SIGGI interviews administered by site-based CGI raters. We compared SIGGI-derived CGI-S scores between the two separate site-based raters and the site-independent raters.

Results: We found significant intraclass correlations (p = 0.001) on all SIGGI-derived CGI-S scores between two separate site-based CGI raters with each other (r = 0.768) and with a blinded, site-independent rater (r = 0.748 and r = 0.706 respectively) and significant Pearson’s correlations between CGI-S scores with all MADRS validity comparisons for MDD and PANSS comparisons for schizophrenia (p- 0.001 in all cases). Compared to site-based raters, the site-independent raters gave identical “dual” CGI-S scores to 67.6% and 68.2% of subjects at visit 1 and 77.1% at visit 2.

Conclusion: We suggest that the SIGGI may improve the inter-rater reliability and scoring precision of the CGI-S and have broad applicability in CNS clinical trials.

A cross-cultural comparison study of depression

“A cross-cultural comparison study of depression assessments conducted in Japan.” Ann Gen Psychiatry. 12(1) 9, 2013.
Targum SD, Nakagawa A, Sato Y.

BACKGROUND: The advent of global clinical trials has necessitated the use of English-based rating instruments in diverse cultures where English is clearly not the primary language. The cross-cultural applicability of rating instruments developed in one language with only one cultural group is an important issue in both research and clinical settings where these instruments might be used. We examined the cross-cultural applicability of the Montgomery-Asberg Depression Rating Scale (MADRS) in Japan.

METHODS: As part of a rater-training program for a clinical trial in Japan, we assessed inter-rater agreement using two videotaped MADRS interviews administered in Japanese and produced with English subtitles. We looked for possible interpretational variance that might have been generated by cultural differences between Japanese raters in Japan and English-speaking raters in the USA scoring the same interviews.

RESULTS: The US and Japanese raters demonstrated high inter-rater agreement and no significant scoring difference on the total MADRS score. The subtitles in English did not adversely affect the overall scoring. We separately analyzed the 10 individual items from each of the two MADRS interviews used for rater training. Of the 20 items, 18 were concordant between the US and Japanese raters. In one interview, the US raters scored lassitude significantly higher (p = 0.013) and the inability to feel significantly lower (p = 0.037) than the Japanese raters, reflecting a possible interpretational difference on these items.

CONCLUSION: Although developed in Europe, these findings support the general applicability of the MADRS to assess the severity of depressive symptoms in Japan. We did note significant scoring differences on 2 of the 20 individual items, suggesting a possible cultural difference. It is possible that more interviews might have revealed more interpretational differences. These findings highlight the need for cultural familiarity when assessing psychiatric patients.

Innovations in Clinical Neuroscience

“Psychiatry in the Ukraine.”Innovations in Clinical Neuroscience. 10 (4), 2013.
Targum, S, Chaban, O, & Mykhnyak, S.

“A comparative analysis between site-based and centralized ratings and patient self-ratings in a clinical trial of Major Depressive Disorder.” J Psychiatr Res, 2013
Targum S, Wedel PC, Robinson J, Daniel DG, Busner J, Bleicher LS, Rauh P & Barlow C.

Abstract: We compared scores from three different ratings methods in a clinical trial of patients with Major Depressive Disorder (MDD). The Quick Inventory of Depressive Symptoms (QIDS-SR16) was compared to site-based clinician and centralized (site-independent) ratings of the Inventory of Depressive Symptoms (IDSc30). An extracted QIDSc16 was used for a matched comparison with the QIDS-SR16. Patient self-ratings were more depressed at baseline than either site-based ratings (p = 0.131) or centralized ratings (p = 0.005), but significantly less depressed at the end of double-blind treatment than either site-based (p = 0.006) or centralized ratings (p = 0.014), and after 12 weeks (site-based ratings: p = 0.048; centralized ratings: p = 0.004). The matched comparisons with patient self-ratings revealed ICC of r = 0.55 (site-based raters) and r = 0.49 (centralized raters) at baseline. After baseline, the correlations between the two different clinician ratings and patient self-ratings improved to r-values between 0.78 and 0.89. At the end of double-blind treatment, site-based raters separated the combination treatment from placebo on the IDSc30 (p = 0.030) whereas neither centralized ratings nor patient self-ratings achieved statistical significance. Alternatively, patient self-ratings separated the combination treatment from buspirone (p = 0.030) whereas neither clinician rating method achieved significance. A “dual” scoring concordance range reduced the placebo response rate and increased the drug effect between the combination treatment and placebo. These findings reveal scoring variability between each of the three ratings methods and challenge the reliability of any single method to accurately assess symptom severity scores, particularly at baseline. The use of “dual” scoring criteria may help to confirm symptom severity scores and improve ratings precision, particularly prior to enrolling subjects into CNS trials.

A structured interview guide for global impressions: increasing reliability and scoring accuracy for CNS trials.

Pendergrass, C. J., Rauh, P, Toner, C. & Targum, S. “Factors Affecting Subject Selection for Clinical Trials Across CNS” Poster Session Presented at:  American College of Neuropsychopharmacology Annual Meeting, May 28th-31st , 2013, Hollywood FL.

Targum SD, Houser C, Northcutt J, Little JA, Cutler AJ, Walling DP. “A structured interview guide for global impressions: increasing reliability and scoring accuracy for CNS trials.Ann Gen Psychiatry. 12(1) 2, 2013.

Abstract

Background: The clinical global impression of severity (CGI-S) scale is a frequently used rating instrument for the assessment of global severity of illness in Central Nervous System (CNS) trials. Although scoring guidelines have been proposed to anchor these scores, the collection of sufficient documentation to support the derived score is not part of any standardized interview procedure. It is self evident that the absence of a standardized, documentary format can affect inter-rater reliability and may adversely affect the accuracy of the resulting data.

Method: We developed a structured interview guide for global impressions (SIGGI) and evaluated the instrument in a 2-visit study of ambulatory patients with Major Depressive Disorder (MDD) or schizophrenia. Blinded, site-independent raters listened to audio recorded SIGGI interviews administered by site-based CGI raters. We compared SIGGI-derived CGI-S scores between the two separate site-based raters and the site-independent raters.

Results: We found significant intraclass correlations (p = 0.001) on all SIGGI-derived CGI-S scores between two separate site-based CGI raters with each other (r = 0.768) and with a blinded, site-independent rater (r = 0.748 and r = 0.706 respectively) and significant Pearson’s correlations between CGI-S scores with all MADRS validity comparisons for MDD and PANSS comparisons for schizophrenia (p- 0.001 in all cases). Compared to site-based raters, the site-independent raters gave identical “dual” CGI-S scores to 67.6% and 68.2% of subjects at visit 1 and 77.1% at visit 2.

Conclusion: We suggest that the SIGGI may improve the inter-rater reliability and scoring precision of the CGI-S and have broad applicability in CNS clinical trials.

Development of a clinical global impression scale for fatigue

“Major Depressive Disorder with Mixed Features: Interim Baseline Characteristics of Subjects Enrolled in a 6 week, Double-Blind, Placebo Controlled Trial of Lurasidone.” Poster Session Presented at:  American College of Neuropsychopharmacology 51st Annual Meeting, December 2nd-6th, 2012, Hollywood FL.
Suppes, T, Cucchiaro, J, Pikalov, A, Werner, P, Targum, S & Loebel, A.

“Development of a clinical global impression scale for fatigue.” J Psychiatr Res, 46(3) 370-4, 2012.
Targum SD, Hassman H, Pinho M, Fava M.

Abstract: Physical, cognitive, and affective components of fatigue are often associated with depression and other Axis I psychiatric disorders. We developed two, single item global assessment scales to specifically evaluate symptoms of fatigue. 101 subjects visiting a clinical trial site consented to participate in this reliability and validity study. Diagnoses included Major Depressive Disorder, Bipolar disorder, and schizophrenia. There were two clinic visits during which the modified Clinician and Patient Impressions of Fatigue rating instruments were administered in conjunction with the MGH cognitive and physical functioning questionnaire (MGH-CPFQ), a validated patient-rated 7-item scale. CGI-Severity and PGI-Severity for fatigue were well correlated at two separate visits (p < 0.00005). At visit 1, the mean CGI-S for fatigue was 3.33 ± 1.53 (SD) and the PGI-S for fatigue was 3.57 ± 1.70 (r = 0.75; p = 0.000). At visit 1, the total MGH-CPFQ was 21.66 ± 6.92. Both CGI-S and PGI-S measures for fatigue were highly correlated with the MGH-CPFQ: CGI-S (r = 076; p < 0.00005); PGI-S (r = 0.62; p < 0.00005). Both the PGI-S and CGI-S for fatigue revealed temporal stability and convergent validity for the MGH-CPFQ (r = 0.83 for CGI-S and 0.73 for PGI-S). There was high internal consistency between the two independent CGI raters at visit 2 as demonstrated by a kappa statistic = 0.971 (CGI-S) and 0.868 (CGI-I) and Cronbach’s alpha = 0.998 (CGI-S) and 0.941 (CGI-I). As shown here, the modified CGI and PGI instruments for fatigue are reliable measures of fatigue and both measures are validated with the MGH-CPFQ instrument.

A possible role for neurogenesis in drug discovery

“An exploratory study of combination buspirone and melatonin SR in Major Depressive Disorder (MDD): A possible role for neurogenesis in drug discovery.” J Psychiatr Res,46(12) 1553-63, 2012.

Fava M, Targum SD, Nierenberg AA, Bleicher LS, Carter TA, Wedel PC, Hen R, Gage FH, Barlow C.

Abstract: We used in vitro neurogenesis-based human neural stem cell (hNSCs) assays and rodent in vivo behavioral assays to identify potential novel antidepressants. A combination of buspirone and melatonin displayed antidepressant activity in these assays whereas neither buspirone nor melatonin alone showed any antidepressant-like profile. After evaluating numerous combination ratios, we determined that low dose buspirone 15 mg combined with melatonin-SR 3 mg yielded optimal antidepressant efficacy in our pre-clinical platform. The low dose of buspirone suggested that antidepressant efficacy might be achieved with only minimal adverse event liability. Based on these data, we conducted an exploratory 6-week, multi-center, double-blind, randomized, placebo- and comparator-controlled study of the combination of buspirone and melatonin in subjects with acute Major Depressive Disorder (MDD). The combination treatment revealed a significant antidepressant response in subjects with MDD on several measures (Clinical Global Impression of Severity and Improvement, Inventory of Depressive Symptomatology) compared to either placebo or buspirone 15 mg monotherapy. These preliminary findings have clinical implications and suggest that a platform of pre-clinical neurogenesis matched with confirmatory behavioral assays may be useful as a drug discovery strategy.

A global measure to assess switching antipsychotic medications in the treatment of schizophrenia

“A global measure to assess switching antipsychotic medications in the treatment of schizophrenia.” Hum Psychopharmacol, 27(5) 455-63, 2012.
Targum SD, Pestreich L, Reksoprodjo P, Pereira H, Guindon C, Hochfeld M.

OBJECTIVES: It is common practice to switch antipsychotic medications in the treatment of patients with schizophrenia to enhance clinical efficacy and/or reduce drug-related side effects. The conventional Clinical Global Impression (CGI) of severity scale is a well-understood measure to track switching effects but does not differentiate between the severity of clinical symptoms and the impact of side effects..
METHODS:
We developed a CGI-switch instrument that contains distinct global severity scales for clinical efficacy, safety and/or tolerability, and a third unified (integrated) CGI severity score to assess these interrelated assessments. An integrated Clinical Global Impression of Change was also created to assess global clinical change relative to the initiation of treatment.

RESULTS: Interrater reliability conducted as part of a rater-training program for a clinical study (Novartis protocol CIL0522D; clinitrials.gov identifier: CT01207414) revealed high interrater agreement (Cronbach’s alpha?=?0.945). Data were collected from 1066 CGI assessments during the course of the trial. CGI raters easily grasped the utility of the instrument. The distinction made between efficacy and safety/tolerability facilitated serial tracking of each condition during the course of treatment.

CONCLUSION: The modified CGI-switch instrument is a simple, reliable, and practical metric to assess the benefits, if any of switching antipsychotic medications in patients with schizophrenia

A Novel Strategy for Treating Alzheimer’s Disease

“Deep Brain Stimulation: A Novel Strategy for Treating Alzheimer’s Disease.” Innov Clin Neurosci. 9(11-12) 10-17, 2012.
Lyketsos CG, Targum SD, Pendergrass JC, Lozano AM.

Abstract
Recent studies have identified an association between memory deficits and defects of the integrated neuronal cortical areas known collectively as the default mode network. It is conceivable that the amyloid deposition or other molecular abnormalities seen in patients with Alzheimer’s disease may interfere with this network and disrupt neuronal circuits beyond the localized brain areas. Therefore, Alzheimer’s disease may be both a degenerative disease and a broader system-level disorder affecting integrated neuronal pathways involved in memory. In this paper, we describe the rationale and provide some evidence to support the study of deep brain stimulation of the hippocampal fornix as a novel treatment to improve neuronal circuitry within these integrated networks and thereby sustain memory function in early Alzheimer’s disease.

Overwork suicide in Japan: a national crisis

“Overwork suicide in Japan: a national crisis.” Innov Clin Neurosci. 9(2) 35-8, 2012.
Targum SD, Kitanaka J.

Introduction

I recently attended an international conference on affective disorders held in Tokyo and learned about the high rate of suicide in Korea and Japan. Currently, Korea has the highest annual suicide rate in the world. It is noteworthy that the increasing suicide rate is directly correlated with the annual increase in Korea’s gross national product. Japan has the third highest rate of annual suicide (after Hungary) and has witnessed a staggering 30,000 deaths per year for the past decade related in part to the sustained economic recession. It has not been easy for these proud and tradition-based cultures to acknowledge the relationship between suicide and depression, but the enormous social and economic consequences of the deaths have fostered an unusual partnership between the government and psychiatry to confront this urgent bio-psycho-social issue. Whereas suicide was once “normalized” by many Japanese as an act of free will, the alarmingly high suicide rates have contributed to a new conceptualization of suicide as a genuine mental illness and a new category called “overwork” suicide to describe people who take their lives simply because of working too hard. Although overwork suicide represents only a small fraction of the total suicides in Japan, this new category has had a marked social and political impact in the country.

During the conference, I met Junko Kitanaka, PhD, a medical anthropologist trained at the University of Chicago and McGill University and currently an Associate Professor in the Department of Human Sciences at Keio University (Tokyo). She has researched depression in Japan and written extensively about overwork suicide and the responses taken by both the government and psychiatrists to deal with this dilemma. Here, she presents some of her compelling research work and perspective on overwork suicide.

Educating the public through the courtroom

“Educating the public through the courtroom: efforts of a forensic psychologist.” Innov Clin Neurosci. 9(7-8) 48-50, 2012.
Targum SD, Ebert R.

Introduction

Courtroom drama has always fascinated the public. Fictional dramas, movies, and theater are often based on thinly disguised true stories as well as “real” celebrity cases. In recent times, many of these cases have become prime time television entertainment. Sometimes, the focus of attention commands national headlines when bizarre acts are committed by mentally ill individuals, such as the shooting of Representative Gabrielle Giffords by a mentally ill man a year ago or the shooting of President Reagan years ago.

Unfortunately, the perpetrator is often cast as evil by the press, and the genuine plight of the mentally ill is rarely addressed. According to Ronald Ebert, PhD, a forensic psychologist practicing in Boston, Massachusetts, the presumption of evil, rather than sensitivity to mental illness, is often dominant in the “real” courtroom as well.

Dr. Ebert got his doctorate from Pennsylvania State University in State College, Pennsylvania, and joined the clinical staff at McLean Hospital in Boston, Massachussetts. While at McLean, he became Assistant Medical Director at the Bridgewater State Hospital where for eight years he ran a forensic unit to evaluate and treat dangerous mentally ill men. During that time, he began to provide testimony on behalf of these patients as well. He has been a forensic psychologist for the past 35 years.

I spoke with Dr. Ebert about his work in the courtroom and his efforts to educate the public about the realities of mental illness.