Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Biomedical engineering 1 ________________________________________________________________________ 1. INTRODUCTION Schizophrenia is a chronic, severe, and disabling brain disease. Approximately 1% of the population develops schizophrenia during their lifetime. While schizophrenia is found worldwide, more than 2 million Americans suffer from the illness in a given year. The severity of the symptoms and long-lasting, chronic pattern of schizophrenia often cause a high degree of disability. Although schizophrenia affects men and women with equal frequency, the disorder often appears earlier in men, usually in the late teens or early twenties, than in women, who are generally affected in the twenties to early thirties. People with schizophrenia often suffer a range of symptoms. Their speech and behavior can become disorganized that they may be incomprehensible to others. Available treatments can relieve many symptoms, but most people with schizophrenia continue to suffer some symptoms throughout their lives; it has been estimated that no more than one in five individuals recovers completely. Research is gradually leading to new and safer medications through better understanding of the complex causes of the disease. Approaches include the study of molecular genetics while new imaging methods of the brain’s structure and function hold the promise of new insights into the disorder. The first signs of schizophrenia often appear as confusing changes in behavior. The sudden onset of severe psychotic symptoms is referred to as an “acute” phase of schizophrenia. Less obvious symptoms, such as social isolation or withdrawal, or unusual speech, thinking, or behavior, may precede, be seen along with, or follow the more obvious psychotic symptoms. CURRENT METHOD OF DIAGNOSIS In 2004 the European Brain Council estimated that in Ireland the cost of schizophrenia was €37million of a total brain disorders cost of €260million. Currently t here are no objective biomarkers for the diagnosis of schizophrenia. Mental health professionals currently use the presence of specific symptom clusters during clinical interviews as a means of making a diagnosis of psychiatric disorders. People with schizophrenia often show “blunted” or “flat” affect. This refers to a severe reduction in emotional expressiveness. A person with schizophrenia may not show the signs of normal emotion, perhaps may speak in a monotonous voice, have diminished facial expressions, and appear extremely apathetic. ________________________________________________________________________ Biomedical engineering 2 ________________________________________________________________________ The person may withdraw socially, avoiding contact with others; and when forced to interact, he or she may have nothing to say, reflecting “impoverished thought.” Motivation can be greatly decreased, as can interest in or enjoyment of life. These problems with emotional expression and motivation are symptoms of schizophrenia. ________________________________________________________________________ Biomedical engineering 3 ________________________________________________________________________ 2. AIM The aim is to develop a new approach to diagnose and to disease manage serious mental illnesses (SMI) such as schizophrenia using the acoustic analysis of speech. The spontaneous speech of schizophrenia patients with negative symptoms (e.g. flat affect, loss of interest) contains many hesitations and pauses. Acoustic analysis demonstrates that the length of pauses, independent of other linguistic or paralinguistic parameters, is strongly associated with the clinician's opinion of the patient's negative symptoms. [1, 2] Employing automatic acoustic analysis will result in the development of specific and sensitive acoustic diagnostic biomarkers for schizophrenia. The ability to provide acoustic diagnosis would see significant advances in the following areas: Advanced Medical Diagnostics. An automation of patient diagnosis would reduce with the need to wait for appointments with either clinic or hospital staff. With the availability of acoustic signal processing on easily acquired speech signals, high quality assessments can be carried out remotely, providing diagnostic prescreening of patients with a variety of psychiatric disorders. This will result in prioritizing those in need of the most immediate attention resulting in the use of available medical resources in an efficient and effective manner. Advanced Personalized Care through Disease Management – Comparison of a client’s physiological data automatically generated from biomedical signal processing with local and international databases will increase the disease management capacities of primary care physicians and specialists. Such information processing will enhance treatment adherence and allow continuous monitoring of a variety of health variables remotely. Anticipated benefits of this acoustic diagnosis approach include: Provide critical diagnostic information objectively, efficiently, cost-effectively and non-invasively Reduce the time spent and the need for medical personnel to make diagnoses Manage mental health professional’s time efficiently ________________________________________________________________________ Biomedical engineering 4 ________________________________________________________________________ Reduce travel times and costs for patients Provide medical specialists with patients specific audio signal interpretations Empower patients to become proactive participants in their health management. In this thesis, Chapter 3 overviews the literature in voice characteristics and their important in psychiatric diagnosis. Chapter 4 will give a brief look in prior acoustic investigations on schizophrenia. In Chapter 5 the audio database is presented and the methods used in the development of the classifier discussed. The classifier performance is presented in Chapter 6. Following in Chapter 7 is a discussion of the findings. Chapter 8 draws conclusion from this study. ________________________________________________________________________ Biomedical engineering 5 ________________________________________________________________________ 3. SPEAKING BEHAVIOR AND VOICE SOUND CHARACTERISTICS OF SCHIZOPHRENICS PATIENTS Symptoms such as affective flattening, blunted affect, the inability to experience or express a normal range of affective responses, emotional dullness, pervasive apathy, poverty of speech and psychomotor retardation have been constituents of clinical definitions of schizophrenia throughout this century and have led to the negative-positive model of schizophrenia. In this model, delusions, hallucinations and florid formal thought disorders are conceptualized as pathological excesses, whereas negative symptoms are conceptualized as pathological deficits. Based on this heuristic division, negative symptoms have been theorized to be unresponsive to antipsychotic medication, to be associated with a deteriorating course, and to reflect some structural brain abnormalities that would mediate the hypothesized association between negative symptoms and poor prognosis. Recent findings, however, have indicated that positive symptoms are not uniformly responsive to treatment, and that negative symptoms are not universally immutable. Specifically, during recovery from an acute episode of schizophrenia, the key symptoms tend to disappear first while symptoms at lower levels persist longer. Non-specific and affective symptoms tend to recur as prodromal symptoms of relapse. Furthermore, several authors have reported a diagnostic non-specificity of negative symptoms, particularly with regard to various clinical forms of depression which occur relatively commonly in schizophrenic patients and there is also some overlap between the specific phenomena of depressive illness and negative symptoms in schizophrenia. [1] 3.1. Formal thought disorder In the psychiatric literature, many of the abnormalities of language in schizophrenia are lumped together as formal thought disorder (a disorder in the form of thought, not the content). ________________________________________________________________________ Biomedical engineering 6 ________________________________________________________________________ Perhaps most commonly it is the moment-to-moment, logical sequencing of ideas which is at fault. At other times, the mechanisms of language production themselves appear to be disturbed, so that the meaning of individual words and phrases is obscured. At still other times, the fault seems to be at the level of discourse: individual words, sentences, and sequences of thought make sense, but there is no discernible thread to longer verbal productions. Florid formal thought disorder is a relatively uncommon finding in acute schizophrenia, though it is somewhat more common in chronic cases. Manifestations of formal thought disorder include poverty of content (failure to express sufficient information), loss of goal (slippage away from the intended topic), clanging (chaining together similar sounding words as if distracted by them), and other kinds of incoherence and unintelligibility. 3.2. First study of linguistic structure The study of schizophrenic language disorder by linguists began with Chaika (1974), who studied a single patient who spoke normally for weeks at a time, her deviant language coinciding with what her psychiatrists term psychotic episodes. Stripped of some mid-1970s theoretical terminology, and condensed somewhat, the abnormalities that Chaika observed were: (1) Failure to utter the intended lexical item; (2) Distraction by the sounds or sense of words, so that a discourse becomes a string of word associations rather than a presentation of previously intended information; (3) Breakdown of syntax and/or discourse; (4) Lack of awareness that the utterances are abnormal. Of these, (2) is most characteristic of schizophrenia; (1) and (3) resemble ordinary speech errors, and (4) resembles some forms of aphasia. ________________________________________________________________________ Biomedical engineering 7 ________________________________________________________________________ 3.3. Schizophrenic language vs. normal speech errors Fromkin responded that except for the disruption of discourse which can be attributed to non-linguistic factors, all the features [of schizophrenic language] are prevalent in normal speech as exemplified by speech errors and “slips of the tongue” Mistaken lexical choices and minor scramblings of syntax are common in everyday speech. Indeed, speech errors are often triggered by the sounds or senses of recently uttered words, and speakers are commonly unaware of their fumbles. Thus, (1), (2), (3), and (4) of Chaika’s core abnormalities are all disposed of, except for derailment of discourse, which Fromkin considers extra linguistic. This claim has not held up. Although there are obvious similarities between Chaika’s samples of schizophrenic language and Fromkin’s corpus of speech errors, there are also obvious differences. Normal speakers make occasional errors like those seen in schizophrenia, but not whole strings of errors. A representative patch of gibberish from Chaika’s patient comprised 9 syllables, and uncorrected speech errors of such length and unintelligibility do not occur in normal speech. What’s more, normal speakers, when an error is pointed out, immediately correct it; speakers with schizophrenia do not. Moreover, Chaika’s patient would commonly string together 10 or 20 sentences connected, as far as one can see, only by word associations; ordinary people, even when plagued by speech errors, do not do this. Normal speech errors are momentary deviations from a discourse plan that is immediately resumed. 3.4. Loss of voluntary control In later work, Chaika argues that schizophrenic language disorder is fundamentally a loss of voluntary control over the speech generation process. Indeed, according to Chapman, patients sometimes say in retrospect that this is exactly what happened—they couldn’t control their speech. This echoes the main theme of the Schneiderian first-rank ________________________________________________________________________ Biomedical engineering 8 ________________________________________________________________________ symptoms, which is loss of control over the train of thought. Note however that Chaika’s original patient apparently lacked such insight. Chaika argues that loss of voluntary control ties together a wide range of observed phenomena, depending on which part of language production goes out of control—most often discourse organization, but often lexical retrieval, and sometimes pronunciation or syntax. It fits well into a more general conception of schizophrenia as degradation of communication between mental subsystems. 3.5. Schizophrenic language disorders vs. aphasia How much do the language disturbances of schizophrenia resemble the aphasia caused by stroke, traumatic brain injury, or neurological conditions such as epilepsy? Researchers agree that there are important differences, but beyond that, discussion of the issue has been complicated by the heterogeneity of both schizophrenia and aphasia. Aphasia-like symptoms are episodically observed in only a small proportion of subjects considered to be schizophrenics whereas the aphasia produced by stroke or brain injury is in most cases constantly present. Patients with aphasia have normal thoughts and express them with difficulty; those with schizophrenia have unusual thoughts (or disorganized discourse plans) and express them with comparative ease. Pinard and Lecours compare schizophrenic language to Wernicke’s aphasia (including jargon aphasia), a disorder in which the patient speaks fluently but unintelligibly. Their main findings: (1) Schizophasic discourse often has a preferred theme or preoccupation; aphasic discourse rarely does. (2) Speakers with schizophrenia often jump from one subject to another based on the sounds or associations of words they have uttered (association chaining or glossomania). This is seldom observed in jargon aphasia; it requires lexical mastery well beyond that of most aphasics, as well as remarkable control of prosody. ________________________________________________________________________ Biomedical engineering 9 ________________________________________________________________________ (3) Schizophasic discourse often includes rare words, evidence of a large, intact vocabulary; jargon aphasia, even when very fluent, shows a restricted vocabulary. (4) Schizophrenic speech can include conscious creation of new words (neologisms) and consciously constrained discourse in which the speaker is well aware that the speech is unusual, whether or not others can understand it. Aphasic speakers who produce fluent unintelligible discourse do not seem to be fully aware of what they are doing, and if they create new words, it is as if by accident 3.6. Andreasen’s 18-point scale The standard account of schizophrenic language today is that of Andreasen, whose Thought, Language, and Communication (TLC) scale provided a foundation for subsequent research and clinical practice. The scale comprises 18 symptoms: (1) poverty of speech (2) poverty of content (wordy vagueness) (3) pressure of speech excessive speed or emphasis) (4) distractibility (by stimuli in the environment) (5) tangentiality (partly irrelevant replies) (6) loss of goal (7) derailment (loss of goal in gradual steps) (8) circumstantiality (numerous digressions on the way to the goal) (9) illogicality (10) incoherence (“word salad”, severely disrupted structure) (11) neologisms (novel made-up words) (12) word approximations (coined substitutes for existing words) (13) stilted speech (pompous or overly formal style) (14) clanging (15) perseveration (16) echolalia ________________________________________________________________________ Biomedical engineering 10 ________________________________________________________________________ (17) blocking (sudden stoppage) (18) self-reference (talking about oneself excessively) 3.7. Liddle’s TLI Liddle et al. simplified Andreasen’s TLC index into a Thought and Language Index comprising 8 symptoms, which factor analysis divided into 3 groups with strong, cleanly separate factor loadings. They found that the disorganization and impoverishment symptom groups were approximately orthogonal rather than bipolar (neither one is the opposite of the other; they can coexist in any combination). Perseveration and distractibility correlated with each other but were independent of both impoverishment and disorganization. Another noteworthy result was that the scores in the healthy control group were not negligible, i.e., the abnormalities were found, in mild form, in non-patients. The triad of Liddle et al. resembles other classifications of symptoms of schizophrenia going back all the way to the 19th-century division into catatonia (negative symptoms), paranoia (odd thoughts), and hebephrenia (disorganization). 3.8. Chen’s CLANG scale In a paper that deserves to be better known, Chen et al. present an alternative to Andreasen’s TLC scale and its derivatives. Their CLANG (Clinical Language) scale comprises 17 symptoms classified according to levels of linguistic structure. Ranging beyond Andreasen’s and Liddle’s scales, CLANG includes disturbances of fluency, voice quality, and articulation. It is a fuller evaluation of speech, not just “thought” or discourse. ________________________________________________________________________ Biomedical engineering 11 ________________________________________________________________________ The factor analysis by Chen et al. found three major kinds of language dysfunction in schizophrenic patients, “syntactic,” “semantic,” and “production.” “Syntactic” dysfunction affects the structure of language on all levels, including lexical access. “Semantic” dysfunction affects the ability to map thoughts onto language and pursue a communicative goal; it corresponds closely to the traditional definition of thought disorder. “Production” dysfunction comprises poverty of speech, lack of details, and lack of intonation; it is associated with the negative symptoms of schizophrenia. There are also factors for “pressure” and “prosody” (each cleanly self-contained, even though “prosody” is only one symptom) and two symptoms, dysarthria and excessive details, that are unclassified because they have minor loadings in several factors. Ceccherini-Nelli and Crow got a different factor analysis in which the “syntactic” and “semantic” factors fell together into one, perhaps because of different scoring criteria. Their second factor, “poverty,” corresponds closely to the “production” of Chen et al. Their third factor, “excess,” comprises syntactic perseveration (excess syntactic constraints) and excessive detail (interpretable as semantic perseveration). [3] 3.9. Acoustic Speech Characteristics Predominantly negative-symptom schizophrenia (NSZ) can be characterized by a number of atypical reductions in observed behavior, including communication behavior. Classically observed outward manifestations in spoken (e.g., linguistic production) and pragmatic/paralinguistic (e.g., body language, intonation, and prosody) aspects of communication are related to affective flattening, anhedonia, alogia, and avolition (the Diagnostic and Statistical Manual of Mental Disorders summarized characterization of NSZ), implying an effect on both affect and cognition. Atypical communication can have an effect on clinical assessment, as it may influence rater scoring on a number of items from subjective behavioral rating scales (e.g., PANSS) including items such as blunted ________________________________________________________________________ Biomedical engineering 12 ________________________________________________________________________ affect, emotional withdrawal, poor rapport, social withdrawal, reduced flow of conversation, and motor retardation. Although these rating scales are invaluable in their ability to help clinicians assess symptomatology, the addition of objective and quantifiable measures of disease severity and therapeutic treatment response are desirable and possible through speech and voice acoustic measurement. In the most basic terms, physical quantitative measurements of communication behavior, using aspects of frequency, intensity, and time, support and extend the clinical impressions of atypical communication used clinically. This adds clinical value by providing repeatable quantification of observed symptomatology. The literature surrounding acoustic investigations in persons with schizophrenia has revealed a number of consistent themes. Flat affect, alogia, and asociality (as measured by the PANSS scale) are strongly related to restricted speech output, monotonous speech, pause in speech, energy variation, utterance duration, and inflection. In addition, acoustic measures have shown great promise in identifying treatment response by demonstrating a larger treatment effect than those seen with traditional rating scales. It has also been demonstrated that specific measures of acoustic inflection are sensitive enough to differentiate between antipsychotic compounds (Olanzapine vs. Haloperidol; Remoxipride vs. Haloperidol), whereas rating scales were not able to detect this difference. Bidirectional changes in the speech acoustic characteristics between drug conditions have also led researchers to the conclusion that different mechanisms of drug action that may be at work, though rating scales were not able to make this distinction. Acoustic measures were able to separate the different drug groups at outcome while the rating scales failed to show a difference. In a recent acoustic investigation by Wisniecki et al. have been successfully demonstrated measurable differences in cognitive behavior and motor slowing by comparing persons with NSZ to a control group. Speech pause behavior in a simple counting and picture description task have indicated that average pause length was indicating of motor retardation, whereas global measures of pause were indicative of the increased cognitive linguistic demands of the picture description task. [4] ________________________________________________________________________ Biomedical engineering 13 ________________________________________________________________________ 4. ACOUSTIC INVESTIGATIONS ON SCHIZOPHRENIA Based on a sample of 42 chronic schizophrenic patients and 42 carefully matched controls, Stassen et al. investigated potential relationships between acoustic variables on the one hand, and negative syndromes, positive syndromes and affective disturbances, on the other. A set of 12 acoustic variables automatically assessed in a standardized experimental setting allowed an almost perfect discrimination between schizophrenic patients and normal subjects. Acute side-effects of medication did not explain this finding. However, the question of whether the observed changes in speaking behavior and voice sound characteristics were caused by long-term neuroleptic treatment, for example, as a consequence of tardive dyskinesia, could not be answered by the investigation. In view of a biological validation of the negative-positive model of schizophrenia, the reliability of various psychopathological subscales was tested through repeated assessments at 14 day intervals. Stassen et al. found most psychopathology scores to be sufficiently stable and reproducible over time, thus representing a suitable basis for the estimation of severity with respect to the negative and positive component of schizophrenia. Using the first measurements as training samples and the second measurements 14 days later as test samples, discriminant analysis yielded conclusive proof of a close relationship between acoustic variables and the severity of the negative and positive component of schizophrenia. In particular, by means of "objective" acoustic variables and under the constraint of reproducibility, 75.9% of patients were correctly classified as low or high scorers with respect to the negative syndrome, 71.9% of patients with respect to the positive syndrome, and 79.4% of patients with respect to their depressive symptomatology. [1] To asses speaking behavior and voice sound characteristics following acoustic variables were used: (1) mean pause duration (only pauses >250ms are included) (2) number of pauses (only pauses >250ms are included) (3) mean pause duration per second ________________________________________________________________________ Biomedical engineering 14 ________________________________________________________________________ (4) mean utterance duration (5) total recording time (6) total length of pauses (only pauses >250ms are included) (7) total length of utterances (8) mean energy per second (9) variation of energy per second (dynamics) (10) mean energy per syllable (11) variation of energy pre syllable (dynamics) (12) mean vocal pitch (13) variation of mean vocal pitch (intonation) (14) F0-amplitude (intensity of mean vocal pitch relative to the intensity of overtones (timbre)) (15) F0-6dB-bandwidth (intonation) (16) F0-contour (timbre) All these variables have turned out to be highly stable over time and to be sensitive enough to distinguish between emotionally neutral and emotionally charged texts read out loud by the same individuals. All speech signals were inspected visually and marked with and artifact code if necessary so that disturbed intervals could be removed prior to data analysis. [5] Literature review, of speaking behavior of schizophrenic patients and acoustic investigation on schizophrenia, was presented in this and previous Chapter. Following Chapter will inform reader about data and methods used in developing of system capable discriminating between schizophrenic and non-schizophrenic subjects. ________________________________________________________________________ Biomedical engineering 15 ________________________________________________________________________ 5. DATA AND METHODS The system implementation is comprised of three main parts - data acquisition, feature extraction and classification. Audio Data Acquisition Feature extraction Classifier Decision Fig. 5.1 – System implementation 5.1. Audio data The audio sample included 33 subjects. Of these, 15 subjects represented schizophrenic patients under various medications and 18 subjects belonged to control group. The sample was comprised of 19 male subjects and 14 female subjects with a mean age of 39.6 years (SD 13.2 years, range 24-80 years). The average illness duration since diagnosis was 5.6 years (SD 5.4 years, range 1-17 years). All subjects from patients and control group were tested on EEG to confirm their condition or mental health. See Appendix A for patients and controls details. All audio files were recorded in a quiet room on a minidisc recorder with direct digital 16-bit sampling, at a sampling rate of 22 kHz. Although all effort was taken to keep constant microphone distance during acquisition, this was not the case in all recordings. Emotionally neutral text of around 3 min. length from a children’s book was selected for the recording, especially chosen for its verbal and semantic simplicity. [5] (See Appendix B) ________________________________________________________________________ Biomedical engineering 16 ________________________________________________________________________ 5.2. Acquisition All audio files were processed in GoldWave v.4.26 to reduce ambient noise, where silent part in each file was selected and copied to the clipboard. Than function Noise Reduction was applied on whole file based on data in the clipboard. Pauses longer than 250ms were removed at the beginning and the end of the file, so they do not affect the results. Further, all speech signals were inspected visually, so that disturbed parts could be removed before data analysis. Furthermore, normalization was performed on all speech signals so as to ensure equal conditions for energy measurement. This was necessary due to variations of distance between the speaker and the recording microphone. 5.3. Feature extraction Feature extraction was carried out using MATLAB executed processing algorithms. 5.3.1. Time statistics The recordings were analyzed, so only speech signals above a certain threshold were classified as voice segments. Only pauses greater than 250ms were included, whereof all segments below threshold lasting less than 250ms were added to voiced parts of signal. Following acoustic variables were extracted to obtain time statistics: (1) Mean Pause Duration (2) Number of Pauses (3) Mean Pause Duration per Second (4) Mean Utterance Duration (5) Total Recording Time ________________________________________________________________________ Biomedical engineering 17 ________________________________________________________________________ (6) Total Length of Pauses (7) Total Length of Utterances 5.3.2. Energy measures The energy of a discrete time signal is defined as def Ex x ( n) 2 (5.1) n A signal is called as energy signal if 0 Ex (5.2) “Mean Energy per Second” and “Standard Deviation of Mean Energy per Second” were estimated for each speech signal. [6] 5.3.3. Vocal Pitch Estimation For the vocal pitch estimation the Variable Length Average Magnitude Difference Function [7] was used. This function provides fast and efficient pitch estimation algorithm which can be implemented in real-time. The main principle of AMDF is to find the pitch by comparing the similarity between the original signal and its shifted version. The VLAMDF algorithm uses variable-length speech samples and is defined as E VLAMDF ( ) 1 1 s ( n) s ( n ) (5.3) n 0 where min , min 1 ,..., max ________________________________________________________________________ Biomedical engineering 18 ________________________________________________________________________ The pitch is estimated according to the minimum of the waveform and is obtained by the equation max T pVLAMDF arg MIN E VLAMDF ( ) min (5.4) The speech recording was divided into 40ms frames. Then, based on certain conditions to eliminate unvoiced and speech onset or offset frames, appropriate frames were chosen and pitch was estimated in these frames. Further, “Mean Vocal Pitch” and “Standard Deviation of Mean Vocal Pitch” were estimated. Fig. 5.2 –Waveform of VLAMDF, 1(a) the original waveform, 1(b) the waveform of VLAMDF ________________________________________________________________________ Biomedical engineering 19 ________________________________________________________________________ 5.4. Classification Based on histograms of features, extracted from all speech signals for schizophrenic and control group, which show an approximate Gaussian distribution a classifier based on Linear Discriminant Analysis [8] was chosen to differentiate between control and schizophrenia audio samples. Cross-fold validation was used to estimate classification accuracies. 5.4.1. Linear Discriminant Analysis Training of the LD classifiers proceeded as follows. Let x be a column vector containing d feature values which is to be assigned to one of two classes. Assume there are N1 feature vectors available for training the classifier from class 1 and N2 feature vectors from class 2. The nth feature vector for training in class k is designated as x n( k ) . Training involves determining the class-conditional mean vectors 1 and 2 using 1 1 N1 (1) xn N1 n1 2 1 N2 N2 x n 1 ( 2) n (5.5) (5.6) and the common covariance matrix ∑ using 2 Nk 1 ( xn( k ) k )( xn( k ) k )T N1 N 2 2 k 1 n1 (5.7) The common covariance matrix is employed since within class covariance is similar and offers better representation for the limited normal training set. After determining the k ’s and Σ from the training data, a feature vector x is classified by assuming values ________________________________________________________________________ Biomedical engineering 20 ________________________________________________________________________ for the prior probability of class 1, 1 (note that 2 1 1 ), and calculating the discriminant value, y using 1 1 y ( 1 2 )T 1 x ( 1 2 )T 1 ( 1 2 ) log 2 ( 1 ) 1 (5.8) The posterior probabilities for classes 1 and 2 are then calculated using exp( y ) exp( y ) 1 (5.9) P(2 | x) 1 P(1 | x) (5.10) P(1 | x) The final classification is obtained by choosing the class with the highest posterior probability estimate from (5.10). 5.4.2. Assessing classifier performance The cross-validation scheme [9] was used for estimating the classifier performance. The variance of the performance estimates was decreased by averaging results from multiple runs of cross validation where a different random split of the training data into folds is used for each run. In this study ten repetitions of fifteen-fold cross-validation were used to estimate classifier performance. For each run of cross fold validation the number of normal and abnormal cases were equal. Classifier performance was measured using sensitivity, specificity, positive predictivity, negative predictivity and the overall accuracy. These measures were calculated as per the definition of true positives (TP), true negatives (TN), false positives (FP) and false negatives (FN) presented in Table 5.1. ________________________________________________________________________ Biomedical engineering 21 ________________________________________________________________________ Sensitivity TP (TP FN ) (5.11) TN FP (5.12) Positive Predictivity TP (5.13) Specificit y TN (TP FP) Negative Predictivity TN (5.14) (TN FN ) Overall Accuracy (TP TN ) (5.15) (TP TN FP FN ) TABLE 5.1 DEFINITIONS OF TRUE POSITIVES, TRUE NEGATIVES, FALSE POSITIVES AND FALSE NEGATIVES. True Classification Pathology PREDICTED CLASSIFICATION Pathology Normal Normal True Positive False Negative TP FN False Positive True Negative FP TN 5.4.3. Receiver Operator Characteristics (ROC) Receiver operator characteristic curves (ROC curves) provide valuable information on a test’s ability to discriminate between two classes over the complete spectrum of decision thresholds. The ROC curve is a graph of sensitivity vs. (100% - specificity) as the a-priori probabilities of the two classes are swept between zero and one. It provides information on clinical usefulness since it presents a trade-off in costs between false ________________________________________________________________________ Biomedical engineering 22 ________________________________________________________________________ positives and false negatives and can be used to decide the threshold for different clinical requirements e.g. screening vs. pre-surgical diagnosis. The area enclosed by the ROC plot is a metric against which other similar tests can be compared [10]. The area was computed using the trapezoidal rule. Fig. 5.3 –Interpretation of ROC curve This Chapter presented some of the voice features that can be automatically extracted from audio voice files, along with a pattern classification scheme. The next chapter will use these features to discriminate between patients and controls. ________________________________________________________________________ Biomedical engineering 23 ________________________________________________________________________ 6. RESULTS The values for all features, all patients and controls were input to the subject independent LDA classifier to assess their performance to distinguish between schizophrenic and non-schizophrenic subjects. In the ROC graphs below, the Sensitivity, the Specificity, the Positive Predictivity, the Negative Predictivity, the Overall Accuracy and the Area Under the ROC Curve are shown for all features individually and for the combination of all features. The reader is reminded that classifier performance is not dependent on Overall Accuracy but on a combination of all these classifier metrics. ________________________________________________________________________ Biomedical engineering 24 ________________________________________________________________________ 6.1. Number of Pauses Classifier performance: 1) Sensitivity = 66.67% 2) Specificity = 46.67% 3) Positive Predictivity = 55.56% 4) Negative Predictivity = 58.33% 5) Overall Accuracy = 56.67% 6) Area Under the ROC Curve = 0.591 Fig. 6.1 – ROC curve for feature “Number of Pauses” ________________________________________________________________________ Biomedical engineering 25 ________________________________________________________________________ 6.2. Mean Pause Duration Classifier performance: 1) Sensitivity = 60.00% 2) Specificity = 53.33% 3) Positive Predictivity = 56.25% 4) Negative Predictivity = 57.14% 5) Overall Accuracy = 56.67% 6) Area Under the ROC Curve = 0.596 Fig. 6.2 – ROC curve for feature “Mean Pause Duration” ________________________________________________________________________ Biomedical engineering 26 ________________________________________________________________________ 6.3. Mean Pause Duration per Second Classifier performance: 1) Sensitivity = 73.33% 2) Specificity = 60.00% 3) Positive Predictivity = 64.71% 4) Negative Predictivity = 69.23% 5) Overall Accuracy = 66.67% 6) Area Under the ROC Curve = 0.622 Fig. 6.3 – ROC curve for feature “Mean Pause Duration per Second” ________________________________________________________________________ Biomedical engineering 27 ________________________________________________________________________ 6.4. Mean Utterance Duration Classifier performance: 1) Sensitivity = 40.00% 2) Specificity = 80.00% 3) Positive Predictivity = 66.67% 4) Negative Predictivity = 57.14% 5) Overall Accuracy = 60.00% 6) Area Under the ROC Curve = 0.631 Fig. 6.4 – ROC curve for feature “Mean Utterance Duration” ________________________________________________________________________ Biomedical engineering 28 ________________________________________________________________________ 6.5. Total Recording Time Classifier performance: 1) Sensitivity = 60.00% 2) Specificity = 86.67% 3) Positive Predictivity = 81.82% 4) Negative Predictivity = 68.42% 5) Overall Accuracy = 73.33% 6) Area Under the ROC Curve = 0.764 Fig. 6.5 – ROC curve for feature “Total Recording Time” ________________________________________________________________________ Biomedical engineering 29 ________________________________________________________________________ 6.6. Total Length of Pauses Classifier performance: 1) Sensitivity = 66.67% 2) Specificity = 53.33% 3) Positive Predictivity = 58.82% 4) Negative Predictivity = 61.54% 5) Overall Accuracy = 60.00% 6) Area Under the ROC Curve = 0.533 Fig. 6.6 – ROC curve for feature “Total Length of Pauses” ________________________________________________________________________ Biomedical engineering 30 ________________________________________________________________________ 6.7. Total Length of Utterances Classifier performance: 1) Sensitivity = 80.00% 2) Specificity = 80.00% 3) Positive Predictivity = 80.00% 4) Negative Predictivity = 80.00% 5) Overall Accuracy = 80.00% 6) Area Under the ROC Curve = 0.836 Fig. 6.7 – ROC curve for feature “Total Length of Utterances” ________________________________________________________________________ Biomedical engineering 31 ________________________________________________________________________ 6.8. Mean Energy per Second Classifier performance: 1) Sensitivity = 53.33% 2) Specificity = 100.00% 3) Positive Predictivity =100.00% 4) Negative Predictivity = 68.18% 5) Overall Accuracy = 76.67% 6) Area Under the ROC Curve = 0.871 Fig. 6.8 – ROC curve for feature “Mean Energy per Second” ________________________________________________________________________ Biomedical engineering 32 ________________________________________________________________________ 6.9. Standard Deviation of Mean Energy per Second Classifier performance: 1) Sensitivity = 53.33% 2) Specificity = 100.00% 3) Positive Predictivity = 100.00% 4) Negative Predictivity = 68.18% 5) Overall Accuracy = 76.67% 6) Area Under the ROC Curve = 0.876 Fig. 6.9 – ROC curve for feature “Standard Deviation of Mean Energy per Second” ________________________________________________________________________ Biomedical engineering 33 ________________________________________________________________________ 6.10. Mean Vocal Pitch Classifier performance: 1) Sensitivity = 40.00% 2) Specificity = 46.67% 3) Positive Predictivity = 42.86% 4) Negative Predictivity = 43.75% 5) Overall Accuracy = 43.33% 6) Area Under the ROC Curve = 0.453 Fig. 6.10 – ROC curve for feature “Mean Vocal Pitch” ________________________________________________________________________ Biomedical engineering 34 ________________________________________________________________________ 6.11. Standard Deviation of Mean Vocal Pitch Classifier performance: 1) Sensitivity = 66.67% 2) Specificity = 53.33% 3) Positive Predictivity = 58.82% 4) Negative Predictivity = 61.54% 5) Overall Accuracy = 60.00% 6) Area Under the ROC Curve = 0.622 Fig. 6.11 – ROC curve for feature “Standard Deviation of Mean Vocal Pitch” ________________________________________________________________________ Biomedical engineering 35 ________________________________________________________________________ 6.12. Performance of classifier for all features combined Classifier performance: 1) Sensitivity = 86.67% 2) Specificity = 93.33% 3) Positive Predictivity = 92.86% 4) Negative Predictivity = 87.50% 5) Overall Accuracy = 90.00% 6) Area Under the ROC Curve = 0.920 Fig. 6.12 – ROC curve for all features combined ________________________________________________________________________ Biomedical engineering 36 ________________________________________________________________________ From the time statistics features, the best performance was demonstrated by the feature “Total Length of Utterances” with the Overall Accuracy of 80% and the Area Under the ROC Curve equal 0.836. This was followed by “Total Recording Time” with the Overall Accuracy of 73.3% and the Area Under the ROC Curve equal 0.764. For all other features, the Overall Accuracy was proximately 60% and the Area Under the ROC Curve ranged from 0.53 to 0.63. This Chapter presented the results of the classifier performance individually for all extracted features and for all features combined. Next Chapter will discuss further possibilities of improvement of here presented system. ________________________________________________________________________ Biomedical engineering 37 ________________________________________________________________________ 7. DISCUSSION This work presented a system for discrimination between schizophrenic and nonschizophrenic subjects based on audio analysis of speech signals. Based on the results, it can be seen that schizophrenic patients speak slowly and produce longer words in comparison to healthy subjects This explains why the features “Total Length of Utterances” and “Total Recording Time” provided one of the highest performance in discrimination between schizophrenic and non-schizophrenic subjects. Features “Mean Energy per Second” which represents the average value of energy in each second of the speech signal and “Standard Deviation of Mean Energy per Second”, representing the standard deviation of previous feature for whole speech signal and dynamics of the speech, both performed well with the Overall Accuracy of 76.7% and the Area Under the ROC Curve equal 0.871 and 0.876. The increase of energy in patients’ voices could be a side-effect of medication, as it has been reported that patients under sedation tend to speak louder and to produce utterances with a greater variation of loudness. [1] The feature “Mean Vocal Pitch”, representing the average value of vocal pitch measured in each second of the recording, did not perform satisfactorily with the Overall Accuracy of 43.3% and the Area Under the ROC Curve equal 0.453. “Standard Deviation of Mean Vocal Pitch”, which reflects subjects’ intonation, showed better performance than the previous feature with the Overall Accuracy of 60% and the area under the curve equal 0.622. Classifier performance for all features combined was represented by the Overall Accuracy of 90% and the Area Under the ROC Curve equal 0.920, which allowed very good discrimination between schizophrenic patients and the control group. ________________________________________________________________________ Biomedical engineering 38 ________________________________________________________________________ 7.1. New Features New features should be explored, which, when combined with the actual measured time statistics, energy and pitch features may increase system performance. In the study of Stassen et al. [1], the “F0-amplitude” and “F0-contour” features had the highest contribution on overall performance of their discriminant function and demonstrated that the patients’ voice sound characteristics were primarily distinctive in comparison with healthy subjects. 7.2. A larger database A larger database of schizophrenic patients’ and healthy controls’ speech signals, containing various types of schizophrenia and greater number of samples, would contribute to a more robust classification system. 7.3. Full automation and remote diagnostic Remote diagnostic and full automated system study of vocal fold pathologies assessment, as presented in Moran et al. [11], could easily be developed. In this way, high quality assessments could be carried out remotely, resulting in the use of available medical resources in an efficient and effective manner, as well as providing psychiatrists with up to date results with minimal processing. 7.4. Real-time implementation Performance of the algorithm to extract features was assessed. Time statistics for the calculation of “Number of Pauses”, “Mean Pause Duration”, “Mean Pause Duration per Second”, “Mean utterance Duration”, “Total Recording Time”, “Total Length of ________________________________________________________________________ Biomedical engineering 39 ________________________________________________________________________ Pauses” and “Total Length of Utterances” was assessed. The analysis of a audio file with sampling frequency of 11 kHz and of 3 minutes in duration was found to take 30 minutes to process on computer with 2GHz AMD Turion64 CPU and 1GB of RAM. This may not be satisfactory for clinical use. This temporal performance metric was also carried out on resampled, downsampled and decimated version of the original file. Based on results downsampling the original file showed the best results with negligible difference in the feature values compared to the original. Following downsampling to 3675 Hz, the feature extraction time reduced to approximately 8 minutes. Further downsampling to 2756 Hz reduced the processing time to 6 minutes. Further downsampling to 2205 Hz resulted in processing time of 4.5 minutes, again with negligible differences. With use of more a powerful computer the time needed for analysis maybe shortened and the analysis possibly real-time. The processes of resampling and decimating yielded poor quality data with respect to pattern classification and resulted in poor discrimination perfromance. See Appendix C for examples of this processing. 7.5. Longitudinal study Regular (monthly) voice recording could be established to investigate if the features extracted in this study are stable over time. Later, when the effects of medication on schizophrenic patients’ voice could be investigated, regular voice recording could serve as a control of prescribed medication. ________________________________________________________________________ Biomedical engineering 40 ________________________________________________________________________ 8. CONCLUSION In this study, the possibility to use audio analysis as a basis for objective diagnosis in psychiatry was presented by building a system capable of discriminating between schizophrenic patients and healthy control subjects. Very good discrimination with accuracy of 90% was achieved. An increase of number of specific extracted features may yield even better results. Further optimalization of feature extraction and implementation of a remote and fully automated system capable of real-time analysis may reduce the cost currently associated with assessment of schizophrenia and also help psychiatrists with objective measures on the efficiency of the medication they prescribe to their patients. ________________________________________________________________________ Biomedical engineering 41 ________________________________________________________________________ BIBLIOGRAPHY [1] Stassen H. H., Alber M., Püschel J., Scharfetter Ch., Tewesmeier M. & Woggon B. (1995). Speaking behavior and voice sound characteristics associated with negative schizophrenia. Journal of Psychiatric Research 1995 Jul-Aug;29(4):277-96 [2] P.R.Lott, S. Guggenbuhl, A.Schneeberger, A.E.Pulvar H.H. Stassen, “Linguistic Analysis of the Speech Output of Schizophrenic, Bipolar and Depressive Patients”, Psychopathology, No. 35, pp 220-227, 2002 [3] Covington M. A., He C., Brown C., Naci L., McClain J. T., Fjordbak B. S., Semple J., Brown J. (2005). Schizophrenia and the structure of language: linguist’s view. Schizophrenia Research 2005 Sep 1, 77(1):85-98 [4] Wisniecki A., Cannizzaro, M., Cohen, H., & Snyder, P. (2005). Speech Impairments in Neurodegenerative Diseases/Psychiatric Illnesses. In K. Brown (Ed.), Encyclopedia of Language and Linguistics, 2nd edition. Oxford : Elsevier. pp. 358-364 [5] Stassen H. H., Kuny S., Hell D. (1998). The speech analysis approach to determining onset of improvement under antidepressants. European Neuropsychopharmacology 1998 Dec; 8(4):303-10 [6] Deller J. R., Proakis J. G., Hansen J. H. L., Discrete-Time Processing of Speech Signals. Prentice-Hall, Upper Saddle River, New Jersey 07458, 1993 [7] X. D. Mei, J. Pan, S. H. Sun (2001). Efficient algorithms for speech pitch estimation. Proceedings of 2001 International Symposium on Intelligent Multimedia, Video and Speech Processing. Hong Kong, pp. 421-424. [8] Duda R. O., Hart P. E., Stork H. G., Pattern Classification. New York: WileyInterscience, 2000 ________________________________________________________________________ Biomedical engineering 42 ________________________________________________________________________ [9] R. Kohavi, “A study of cross validation and bootstrap for accuracy estimation and model selection,” Proc. 14th Int. Conf on Art. Intel., pp. 1137-1143, 1995. [10] M.H. Zweig, G. Campbell, “Receiver Operating Characteristic (ROC) Plots: A Fundamental Evaluation Tool in Clinical Medicine”, J. Clinical Chemistry, Vol. 39, No. 4, 1993. [11] Moran R. J., Reilly R. B., de Chazal P., Lacy P. D. (2006). Telephony-Based Voice Pathology Assessment Using Automated Speech Analysis. IEEE Transactions on Biomedical Engineering, Vol. 53, No. 3, March 2006 ________________________________________________________________________