Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
International Journal for Quality in Health Care 2001; Volume 13, Number 3: pp. 257–264 An evaluation of the QSP and the QPP: two methods for measuring patient satisfaction JÖRGEN NATHORST-BÖÖS1, INGRID M. E. MUNCK2, INGEMAR ECKERLUND3 AND CAROLINA EKFELDT-SANDBERG4 1 Department of Obstetrics and Gynaecology, Karolinska Institute, Karolinska Hospital, 2The Swedish Agency for Administrative Development, 3National Board of Health and Welfare and 4SPRI, Box 70487, Stockholm, Sweden Abstract Background. Patient satisfaction is a function of several variables addressing reasons why it is important to use methods in which these different factors can be isolated and their importance analysed. Objective. In this project, two methods using this approach were used: the ‘Quality from the Patient’s Perspective’ and the ‘Quality, Satisfaction, Performance’ models. The aim of the present study is to evaluate these two different methods with respect to application, strengths and weaknesses. Design. In the Quality from the Patient’s Perspective model, the patient judges the different domains in two dimensions: perceived reality and subjective importance. The Quality, Satisfaction, Performance model uses a multivariate analysis to capture the patient’s priorities. Four hundred and sixty forms for each model were distributed to a random sample of patients at the Department of Obstetrics and Gynecology at Karolinska Hospital. Main measures. The quality factors ‘treatment by the nurse’, ‘participation’, ‘information’, ‘environment’ and ‘accessibility’ were measured. Results. On both forms, ‘medical care’, ‘treatment by the doctor’ and ‘access to nursing treatment’ received high scores in ‘perceived reality’ while ‘accessibility’ and ‘participation’ received low scores. ‘Subjective importance’ measured directly and indirectly, respectively, in the two models showed high values for ‘medical care’ and ‘treatment by the doctor’. Conclusion. The advantages of the Quality from the Patient’s Perspective model are that it has a comprehensive and solid question bank. The Quality, Satisfaction, Performance model’s advantage is its immediate usefulness and its clear graphic presentation. An integration and further development of these two approaches may prove useful. Keywords: patient satisfaction, patient questionnaire, quality from the patient’s perspective, quality satisfaction performance In the past decade, quality issues in health care have gained increasing interest. Several tools have emerged to continuously monitor hospital care processes and to improve and control different areas of care. To date, most studies have focused on medical and economic criteria, but attempts have also been made to include the customer’s or patient’s judgement about the care that health services supply. Beside democratic and ethical reasons for assessing patient’s views on health care, the findings from such studies can lead to better use of allocated resources. There are other important medical priorities such as that satisfied and well-informed patients find it easier to follow medical instructions thereby eliminating unnecessary medical visits. Continuous measurement of our clients’ opinions about products and services given by the medical community also provides a basis for quality assurance and different forms of benchmarking. There are, however, several problems concerning methodological issues in these studies. In many standardized surveys a high satisfaction rate of 75–90% is often found [1]. Many authors have suggested that the high level of patient satisfaction found is due to measurement errors causing a different kind of bias, such as social desirability, reluctance Address reprint requests to J. Nathorst-Böös, Department of Obstetrics and Gynaecology, Karolinska Hospital, Box 140, 171 76 Stockholm, Sweden. E-mail: [email protected] International Society for Quality in Health Care and Oxford University Press 257 J. Nathorst-Böös et al. Figure 1 Spectrum of influences on ‘patient satisfaction’. to express a negative opinion, the wording of questions and non-specific questions [2–5]. The lack of differentiation seen seems to indicate that patient satisfaction is not easily operationalized as a discrimination measure [1]. Patient satisfaction is a function of several variables (Figure 1) and it is therefore of great importance to use multivariate methods in which the different factors can be isolated. These validation problems have been approached by communicating with patients to identify the issues that concern them most acutely and to determine how they perceive and interpret the services they receive. This is often done with focus groups or by one-on-one interviews with patients prior to measuring their satisfaction. In this way, questions that are important to patients can be elucidated and used in questionnaires. After identification of the patients’ concerns, these can be grouped into specific domains, such as medical, physical, emotional etc. [6–10]. In order to reduce the number of questions and to detect the structure in the relationships between the variables, the traditional factor analysis or the more general statistical methodology, structural equation modelling, may be used [8]. Another methodology problem in these studies concerns the relative importance of the different categories. It is often implied that it is those aspects of satisfaction with which patients are least satisfied that should be considered as priorities for improvement, i.e. patient satisfaction surveys seldom take into account patients’ priorities among different variables. Deciding what is a priority by choosing the domain with the lowest score also ignores the possibility of improving satisfaction generally. It is therefore important that the method used captures the patient’s judgement of the different domains incorporated [11]. In this project, two methods using this approach have been used. In the Quality from the Patient’s Perspective (QPP), the patient judges the different domains in two dimensions: perceived reality and subjective importance. The Quality, Satisfaction, Performance (QSP) model uses a multivariate analysis to capture the patient’s priorities. The two models are described in more detail in the Materials and methods section. The aim of the present study was to evaluate the two different methods with respect to application, strengths and weaknesses. 258 Materials and Methods The QPP model The QPP model is based on the premise that the quality of care can be understood by evaluating two aspects of care provision: the resource structure of the care organization and the patient’s preferences. The resource structure of the care organization consists of the personnel and the qualities of the physical and administrative environment while the patient’s preferences have a rational and a human aspect [10]. The QPP questionnaire was developed from the findings of a study using personal interviews [12]. In this instrument, each question is judged in two dimensions; perceived reality and subjective importance. The question bank in the QPP form originates from interviews where factors of importance for patient satisfaction with care have been identified [10]. In this study, 35 interviews were conducted consisting of openended and individually adapted follow-up questions covering the following themes: (i) Issues of importance with regard to the care the patient received; (ii) what the patient perceived as positive or negative in connection with the care; (iii) whether the patient felt that anything was lacking during the period of care; and (iv) whether the patient wished to change anything regarding the care. The interviews lasted approximately 60–90 minutes and were tape-recorded and later transcribed verbatim for consecutive analysis. Nine hundred indicators were sorted into 27 categories. The model was then operationalized using factor-analysis into a questionnaire consisting of 56 questions. The point of departure for measurement are these quality factors: (i) medical care; (ii) treatment by the doctor; (iii) treatment by the nurse; (iv) participation; (v) information; (vi) environment; and (vii) accessibility. We designed two main QPP forms for outpatient assessment: a long and a short form. In this study the long form was used. In the QPP approach, each question was judged in two ways: (i) perceived reality on a four-point scale (quite correct, almost correct, not quite correct, not correct at all, not applicable) and (ii) subjective importance on a four-point scale (of utmost importance, of great importance, of little importance, of no importance, not applicable). An example of a QPP question is given in Table 1. An evaluation of QSP and QPP: two methods measuring patient satisfaction Table 1 Example of a question from the QPP model Waiting time at clinic ........................................................................................................................ I was . . . Perceived reality Subjective importance ....................................................................................................................................................................................... . . . treated without any delay Quite correct 4 Of utmost importance 4 during my visit at the clinic Almost correct 3 Of great importance 3 Not quite correct 2 Of little importance 2 Not correct at all 1 Of no importance 1 Not applicable X Not applicable X Table 2 Example of a question from the QSP model How satisfied are you with the waiting time at the clinic? ............................................................................................................................................................................................................................. Not satisfied Very Do not at all satisfied know 1 2 3 4 5 6 7 8 9 10 Φ Φ Φ Φ Φ Φ Φ Φ Φ Φ Φ A personal quality index (PQI) was calculated for each question by placing the response scores into the following formula: PQI=subjective importance×(2×perceived reality – subjective importance). The formula gives an index that can vary between –8 and 16. High values indicate good patient satisfaction and low values suggest that action should be taken in order to improve satisfaction [12]. Figure 2 The relationship between the three concepts ‘quality factors’, ‘patient satisfaction’ and ‘goals’ in the QSP model. The QSP model The QSP model originates from a method used to assess customer satisfaction. The model has been developed and adapted for assessing patient satisfaction. Studies have been performed in eye clinics and dermatological clinics [13]. The QSP model consists of three parts that are later combined to give one score for each respondent: (i) Patient satisfaction is rated on a global, standardized scale and assessed by means of three questions. This forms the Patient Satisfaction Index’ (PSI) which is estimated from the weighted average of the three questions. (ii) Experienced quality of different dimensions or factors of quality are a priori regarded as important in explaining patient satisfaction. Examples of such factors are information, waiting-time and satisfaction with medical care. Three or four questions were formulated to cover each factor. (iii) Experienced quality of factors is defined as the clinic’s goals. Examples of these goals are increased confidence and willingness to come back. An example of a question from the QSP questionnaire is given in Table 2. All questions were judged on a 10-point scale. The statistical QSP model consists of two parts: (i) Part I. A structural model describes how the latent variables patient satisfaction and the different quality factors and goals are related to each other. The relationship between the three concepts quality factors, patient satisfaction, and goals is described in Figure 2. This model is used for the statistical analysis. (ii) Part II. Each concept in the model is measured with a latent variable built up as a factor related to several questions and this constitutes the measurement part of the model. The relationships between the different quality factors and patient satisfaction are expressed as equations. This means that a regression coefficient is calculated describing the importance of each factor on the PSI. The regression factor can be interpreted in terms of the importance of the factor studied while at the same time controlling for the other factors, i.e. keeping the rest of the factors constant. The results can be presented in a coordinate system with the factor under study on the y-axis and its impact on the PSI shown on the x-axis. 259 J. Nathorst-Böös et al. The statistical method used is a multivariate analysis based on the principles used in the Partial Least Squares (PLS) method [14]. The study was conducted at the Department of Obstetrics and Gynecology at Karolinska Hospital, a university outpatient clinic with approximately 4000 visits each year. The catchment area of the clinic is the northern part of Stockholm, serving approximately 800 000 inhabitants. Apart from the problems often observed at gynecological clinics, such as bleeding disorders, Pap smear tests and HRT treatment, most of the clinic patients are healthy. Table 3 The eight domains and 37 questions used in the QPP and QSP questionnaires Domain Medical care Waiting time Treatment by the doctor Main measures The first step was to perform a pilot study with open-ended questions in order to see what issues were regarded as important by the patients. A simple questionnaire form was given to 100 patients who were asked to indicate three things they considered good and three things they considered bad. From this assessment the most important domains and questions were identified. Suitable questions were then selected from the QPP questionnaire. This process was carried out in conjunction with the nurses and doctors at the clinic. It was considered important to make the questionnaire as simple as possible and to exclude questions which seemed inappropriate to the outpatient setting. The questions were formulated in a similar way and arranged in the same order as in the QPP and QSP questionnaires. The 37 questions finally chosen covered eight different factors. The questions and factors are presented in Table 3. An additional 19 questions from the QSP form covered Goals and General satisfaction. The questionnaires were handed out by the receptionist to all women attending the outpatient clinic and were alternated among the patients. Every other patient received a QPP questionnaire and the remaining patients received a QSP questionnaire. The women were requested to fill in the forms during their visit but those who for various reasons (lack of time being the most frequent excuse) chose to fill in the forms after their visit received an stamped addressed envelope so that they could return their completed form by post. The number of forms distributed to each group was 460. Two hundred and sixty (63%) of the QSP questionnaires and 247 (54%) QPP questionnaires were returned. After sorting out the forms with incomplete answers, it was found that 60% of the QSP questionnaires and 52% of the QPP questionnaires had been returned satisfactorily. Statistical analysis The PQI was calculated for the QPP from the formula described earlier. The QSP was analysed using the PLS method. Furthermore, a Structural Equation Modelling (SEM) was performed for both methods using STREAMS 1.7 (Structural Equation Modelling Made Simple) [15]. By use of the SEM technique, different features of the application of the two methods could be evaluated. The aspects brought up in this study were: (i) confirmatory one-factor models for the quality factors were tested for goodness-of-fit and (ii) reliability estimates for the sum-index of variables for each 260 Treatment by the nurse Information Participation Environment Accessibility Question area Examination Medical treatment Doctor’s skill Nurse’s skill On the telephone During the appointment Showed you respect Gave you sincere answers Treated you in a positive way Listened to you Showed commitment and care Showed you respect Gave you sincere answers Treated you in a positive way Listened to you Showed commitment and care Examinations Blood sampling Treatment Drugs Results of tests Where to turn with questions Choose time for appointment Choose doctor Discuss alternative treatments Choose if students should be present or not during examination Finding your way to the clinic Tidiness at clinic Cosiness at clinic Comfort at clinic Atmosphere Possibility of speaking with the doctor in private Telephone hours for making an appointment Reaching the nurse for an appointment Doctor’s telephone hours Reaching the doctor by phone the clinic’s opening hours quality factor were calculated [3]. Regression coefficients for the relationships between the quality factors and ‘patient satisfaction’ were estimated with standard errors taking measurement errors in the 56 questions underlying the seven quality factors into account. An evaluation of QSP and QPP: two methods measuring patient satisfaction Table 4 Background data of the QSP and QPP questionnaire respondents QSP QPP Respondents n=260 n=247 ....................................................................................................................................................................... Median age (age range) 42 (14–80) 41 (11–80) Swedish as native language 86% 90% What level of education do you have? Elementary school, primary school or similar 22% 20% Middle school, junior-high or similar 24% 23% High school 21% 20% University or college 34% 37% Was this your first visit to the clinic? Yes 26% 20% No, I have been here before occasionally 21% 30% No, I have been here several times before 53% 50% How would you rate your present state of health? Very good 30% 27% Rather good 48% 52% Neither good nor bad 16% 14% Rather bad 5% 6% Very bad 1% 1% Results Patients’ background data including age, native language, education, present state of health and previous visits to the clinic are presented in Table 4. The women’s ratings of the different factors in the QPP and QSP questionnaires are presented in Table 5. In the QPP model, the results regarding perceived reality, subjective importance, personal quality index are presented using both the PLS and the SEM-analysis techniques. Treatment by the doctor and medical care received the highest scores in the perceived reality domain. Accessibility received the lowest score. The PQI was calculated according to the formula given above, and the scores ranged between 3.1 and 14.1. The patients placed highest value on ‘treatment by the doctor’ and ‘medical care’ but ‘accessibility’ and ‘participation’ received low scores. In the QSP, the total PSI received a relatively high score of 81.8 out of a possible 100. Also in the QSP model ‘treatment by the doctor’ and ‘medical care’ received high scores while ‘participation’ and ‘accessibility’ received low scores. As previously, a calculation was performed for every factor studied in order to explain the variance in the PSI. This means that a high value indicates high importance on the PSI and vice versa. As a comparison, the regression coefficient was also calculated for the QSP data using the SEM-technique (Table 5). In Figure 3 the mean values of the factors are plotted on the y-axis and their impact on the PSI is shown on the xaxis as described above. The results can be illustrated in a coordinate system with the ranking value on the y-axis and its impact on the total PSI on the x-axis. Factors with high scores and large impact on the total PSI are placed in quadrant I, while factors with high scores but low impact are placed in quadrant IV. In the same way, factors with low scores but large impact are placed in quadrant II, while factors with low scores and low impact are placed in quadrant III. ‘Treatment by the doctor’ and ‘medical care’ were placed in quadrant I, which consequently means that these factors had both high impact and ranking. Also ‘environment’ turned out to have a relatively high impact on the PSI while its ranking was moderate. Accessibility was ranked as the lowest factor but had a significant impact on the PSI. The three remaining factors displayed no significant impact on the PSI. The variance in the PSI explained by the seven quality factors was as high as 79%. This indicates that about 62% of the PSI variance was captured by the QSP questionnaire. The alternative, SEM method was a two-step procedure: (i) testing and adjusting a SEM model to the QSP data, and (ii) estimation of regression coefficients with the so called Maximum Likelihood Method (MLM) taking measurement errors into account [16]. The results are presented in Table 5 and can be compared with the PLS regression coefficients. As seen from the figures, results were found to be similar to the PLS data. Conclusion Measuring patient satisfaction has become increasingly popular. It is important to discuss how these results can be integrated into the health care process in order to increase patient satisfaction. Early in the planning of a study it is therefore important to integrate in what context the results will be used. This should, if possible, include the headings: ‘goals’, ‘relevant questions’, ‘interpretation’ and ‘intervention’. The aim of the present study was to perform an evaluation of the two different methods with respect to the proposed 261 262 n.s., not significant. Medical care Treatment by the doctor Treatment by the nurse Information Participation Environment Accessibility 3.64 3.68 3.45 3.33 2.68 3.23 2.53 (3.59–3.69) (3.60–3.76) (3.40–3.50) (3.27–3.39) (2.62–2.74) (3.17–3.29) (2.49–2.57) 3.76 3.83 3.51 3.72 3.27 3.20 3.23 (3.73–3.79) (3.78–3.88) (3.47–3.55) (3.69–3.75) (3.23–3.31) (3.14–3.26) (3.19–3.27) 13.0 13.5 11.8 11.7 7.8 10.4 6.0 8.81 9.03 8.87 8.49 7.44 8.03 6.26 (8.73–8.89) (8.92–9.14) (8.79–8.95) (8.40–8.58) (7.34–7.54) (7.93–8.13) (6.16–6.36) (mean±95% CI) 2.5 (1.3–3.7) 3.2 (1.8–4.6) 0.5± n.s. 0.5± n.s. 0.6 (0–1.4) 2.7 (1.7–3.7) 1.4 (0.6–2.2) (mean±95% CI) 4.5 (1.7–7.3) 1.8 (0.4–3.2) 0.4 (0–1.2) 0.8±n.s. 0.9 (0–3.1) 2.4 (1.4–4.4) 1.1 (0.5–1.7) QSP SEM QPP QPP QSP Importance Importance Perceived reality Subjective importance QPP Experienced quality PLS regression SEM regression Quality factors (mean±95% CI) (mean±95% CI) PQI (mean±95% CI) coefficient coefficient ................................................................................................................................................................................................................................................................................................................ Table 5 The women’s judgement of the different factors in the QPP and QSP questionnaires J. Nathorst-Böös et al. An evaluation of QSP and QPP: two methods measuring patient satisfaction Figure 3 The mean value of the different quality factors in the QSP model and their impact on the PSI. field of application, and the methods’ strengths and weaknesses. Sixty-three per cent of the QSP and 54% of the QPP forms were returned indicating that the QPP form was somewhat more time consuming to complete, resulting in a higher dropout rate. Regarding the SEM analysis of the dimensions of quality; that is of the seven quality factors, the two methods generated the same factor structure. As expected, a lower reliability index was found for the four-point QPP scale than for the 10-point QSP scale. This means that for this group of women patients a better measure is achieved if the more differentiated 10-point scale is used. For other patients, for example elderly people, the four-point response scale may be preferred, as it is simpler and quicker to answer and generates the same answer pattern, although it may have a somewhat higher measurement error rate. The ease of interpreting the results and developing a basis of action is different for each model. In the QSP, this is incorporated directly into the method of questioning which is built upon a structural equation model that shows how quality factors, patient satisfaction and the clinic’s goals are related to each other. The equation results in a series of coefficients of importance indicating the different factors’ relative impact on patient satisfaction. These relationships can subsequently be presented graphically and the presentation describes the mutual relationship of the factors involved and gives a good indication for how intervention can be made. The high level of explained variance in the PSI indicates that the modelling of the QSP data has given reliable results. In this study the same questions concerning experienced reality were used in both the QSP and the QPP. The QPP form also included the subjective judgement of importance. The QPP model has its basis in orally conducted interviews aimed at investigating what issues in health care are considered important by patients. This process has resulted in a question bank where questions are grouped into specific categories and from which suitable questions can be selected. This firststep procedure is important; it addresses the issue of validating the questionnaire and is not included in the QSP model. In the QPP model, a PQI is directly calculated on the basis of the patient’s estimation of subjective importance and experienced reality. The properties of the formula resulting in this index are, however, inconsistent and the same index can be reached by means of several combinations of importance and reality factors, which makes interpretation difficult. The index gives no information on how patient satisfaction changes if quality improvement is made. As a contrast between these two methods, analysis of these additional data specific to the QPP was set as a secondary priority, mainly because this method showed a tendency towards estimating most subjective importance factors as ‘very important’ or ‘important’. In summary, the advantages of the QPP model are that it has a comprehensive and solid question bank. This consists of 56 questions extracted by means of multiple regression analysis from 900 indicators. The QSP model’s advantage is its immediate usefulness and its clear graphic presentation. The importance of the different factors are grounded in the patients’ values but can be extracted from the model. An integration and further development of these two approaches could prove useful. References 1. Carr-Hill R. The measurement of patient satisfaction. J Public Health Med 1992; 14: 236–249. 2. Raphael W. Do we know what patients think? A survey comparing the views of patients, staff and committee members. Int J Nursing Studies 1967; 4: 209–223. 3. Locker D, Dunt D. Theoretical and methodological issues in sociological studies of consumer satisfaction with medical care. Soc Sci Med 1978; 12: 283–292. 4. French K. Methodological considerations in hospital patient opinion surveys. Int J Nursing Studies 1981; 18: 7–32. 5. Carr-Hill R, Dixon P, Thompson A. Too simple for words. Health Serv J 1989; 99: 728–729. 6. Meredith P, Wood C. Aspects of patient satisfaction with communication in surgical care: confirming qualitative feedback through quantitative methods. Int J Qual Health Care 1996; 8: 253–264. 7. Ware JE, Sherbourne CD. The MOS 36-Item short form health survey (SF-36): 1 Conceptual framework and item selection. Med Care 1992; 30: 473–483. 8. Hays RD, Larson C, Nelson EC, Batalden PB. Hospital quality trends: a short form patient based measure. Med Care 1991; 29: 661–663. 9. Tarlov A, Ware JE, Greenfield S et al. The Medical Outcomes Study: An application of methods for monitoring the results of medical care. J Am Med Assoc 1989; 262: 925–939. 10. Wilde B, Starrin B, Larsson G, Larsson M. Quality of care from 263 J. Nathorst-Böös et al. a patient perspective – a grounded theory study. Scand J Caring Sci 1993; 7: 113–120. 11. Scott A, Smith RD. Keeping the customer satisfied: Issues in the interpretation and use of patient satisfaction surveys. Int J Qual Health Care 1994; 16: 353–359. 12. Wilde B, Larsson G, Larsson M, Starrin B. Quality of care. Development of a patient-centered questionnaire based on a grounded theory model. Scand J Caring Sci 1994; 8: 39–48. 13. Eckerlund I, Jönson B, Tambour M, Westlund AH. Change oriented patient questionnaires – testing a new method at three 264 departments of ophthalmology. Int J Health Care Quality Assurance 1997; 10: 254–259. 14. Fornell C, Cha J. Partial Least Squares. In Bagozzi RP, ed., Advanced Methods of Marketing Research. Cambridge: Blackwell, 1994: pp.52–78. 15. Gustaffson HE, Stahl PA. STREAMS User’s Guide Version 2.5. Gothenburg: Multivariate Ware, 1999. 16. Hox JJ, Becger TM. An Introduction to Structural Equation Modelling. Family Science Review 1998; 11: 354–372. Accepted for publication 23 February 2001