Download An evaluation of the QSP and the QPP: two

International Journal for Quality in Health Care 2001; Volume 13, Number 3: pp. 257–264
An evaluation of the QSP and the QPP:
two methods for measuring patient
satisfaction
JÖRGEN NATHORST-BÖÖS1, INGRID M. E. MUNCK2, INGEMAR ECKERLUND3 AND
CAROLINA EKFELDT-SANDBERG4
1
Department of Obstetrics and Gynaecology, Karolinska Institute, Karolinska Hospital, 2The Swedish Agency for Administrative
Development, 3National Board of Health and Welfare and 4SPRI, Box 70487, Stockholm, Sweden
Abstract
Background. Patient satisfaction is a function of several variables addressing reasons why it is important to use methods in
which these different factors can be isolated and their importance analysed.
Objective. In this project, two methods using this approach were used: the ‘Quality from the Patient’s Perspective’ and the
‘Quality, Satisfaction, Performance’ models. The aim of the present study is to evaluate these two different methods with
respect to application, strengths and weaknesses.
Design. In the Quality from the Patient’s Perspective model, the patient judges the different domains in two dimensions:
perceived reality and subjective importance. The Quality, Satisfaction, Performance model uses a multivariate analysis to
capture the patient’s priorities. Four hundred and sixty forms for each model were distributed to a random sample of
patients at the Department of Obstetrics and Gynecology at Karolinska Hospital.
Main measures. The quality factors ‘treatment by the nurse’, ‘participation’, ‘information’, ‘environment’ and ‘accessibility’
were measured.
Results. On both forms, ‘medical care’, ‘treatment by the doctor’ and ‘access to nursing treatment’ received high scores in
‘perceived reality’ while ‘accessibility’ and ‘participation’ received low scores. ‘Subjective importance’ measured directly and
indirectly, respectively, in the two models showed high values for ‘medical care’ and ‘treatment by the doctor’.
Conclusion. The advantages of the Quality from the Patient’s Perspective model are that it has a comprehensive and solid
question bank. The Quality, Satisfaction, Performance model’s advantage is its immediate usefulness and its clear graphic
presentation. An integration and further development of these two approaches may prove useful.
Keywords: patient satisfaction, patient questionnaire, quality from the patient’s perspective, quality satisfaction
performance
In the past decade, quality issues in health care have gained
increasing interest. Several tools have emerged to continuously
monitor hospital care processes and to improve and control
different areas of care. To date, most studies have focused
on medical and economic criteria, but attempts have also
been made to include the customer’s or patient’s judgement
about the care that health services supply. Beside democratic
and ethical reasons for assessing patient’s views on health
care, the findings from such studies can lead to better use
of allocated resources. There are other important medical
priorities such as that satisfied and well-informed patients
find it easier to follow medical instructions thereby eliminating
unnecessary medical visits. Continuous measurement of our
clients’ opinions about products and services given by the
medical community also provides a basis for quality assurance
and different forms of benchmarking.
There are, however, several problems concerning methodological issues in these studies. In many standardized surveys
a high satisfaction rate of 75–90% is often found [1]. Many
authors have suggested that the high level of patient satisfaction found is due to measurement errors causing a
different kind of bias, such as social desirability, reluctance
Address reprint requests to J. Nathorst-Böös, Department of Obstetrics and Gynaecology, Karolinska Hospital, Box 140,
171 76 Stockholm, Sweden. E-mail: [email protected]
 International Society for Quality in Health Care and Oxford University Press
257
J. Nathorst-Böös et al.
Figure 1 Spectrum of influences on ‘patient satisfaction’.
to express a negative opinion, the wording of questions and
non-specific questions [2–5]. The lack of differentiation seen
seems to indicate that patient satisfaction is not easily operationalized as a discrimination measure [1]. Patient satisfaction is a function of several variables (Figure 1) and it
is therefore of great importance to use multivariate methods
in which the different factors can be isolated.
These validation problems have been approached by communicating with patients to identify the issues that concern
them most acutely and to determine how they perceive and
interpret the services they receive. This is often done with
focus groups or by one-on-one interviews with patients prior
to measuring their satisfaction. In this way, questions that
are important to patients can be elucidated and used in
questionnaires. After identification of the patients’ concerns,
these can be grouped into specific domains, such as medical,
physical, emotional etc. [6–10]. In order to reduce the number
of questions and to detect the structure in the relationships
between the variables, the traditional factor analysis or the
more general statistical methodology, structural equation modelling, may be used [8].
Another methodology problem in these studies concerns
the relative importance of the different categories. It is often
implied that it is those aspects of satisfaction with which
patients are least satisfied that should be considered as
priorities for improvement, i.e. patient satisfaction surveys
seldom take into account patients’ priorities among different
variables. Deciding what is a priority by choosing the domain
with the lowest score also ignores the possibility of improving
satisfaction generally. It is therefore important that the method
used captures the patient’s judgement of the different domains
incorporated [11].
In this project, two methods using this approach have
been used. In the Quality from the Patient’s Perspective
(QPP), the patient judges the different domains in two
dimensions: perceived reality and subjective importance. The
Quality, Satisfaction, Performance (QSP) model uses a multivariate analysis to capture the patient’s priorities. The two
models are described in more detail in the Materials and
methods section. The aim of the present study was to
evaluate the two different methods with respect to application,
strengths and weaknesses.
258
Materials and Methods
The QPP model
The QPP model is based on the premise that the quality of
care can be understood by evaluating two aspects of care
provision: the resource structure of the care organization and
the patient’s preferences. The resource structure of the care
organization consists of the personnel and the qualities of the
physical and administrative environment while the patient’s
preferences have a rational and a human aspect [10]. The
QPP questionnaire was developed from the findings of a
study using personal interviews [12]. In this instrument, each
question is judged in two dimensions; perceived reality and
subjective importance. The question bank in the QPP form
originates from interviews where factors of importance for
patient satisfaction with care have been identified [10]. In
this study, 35 interviews were conducted consisting of openended and individually adapted follow-up questions covering
the following themes: (i) Issues of importance with regard
to the care the patient received; (ii) what the patient perceived
as positive or negative in connection with the care; (iii)
whether the patient felt that anything was lacking during the
period of care; and (iv) whether the patient wished to change
anything regarding the care.
The interviews lasted approximately 60–90 minutes and
were tape-recorded and later transcribed verbatim for consecutive analysis. Nine hundred indicators were sorted into
27 categories. The model was then operationalized using
factor-analysis into a questionnaire consisting of 56 questions.
The point of departure for measurement are these quality
factors: (i) medical care; (ii) treatment by the doctor; (iii)
treatment by the nurse; (iv) participation; (v) information;
(vi) environment; and (vii) accessibility.
We designed two main QPP forms for outpatient assessment: a long and a short form. In this study the long form
was used. In the QPP approach, each question was judged
in two ways: (i) perceived reality on a four-point scale (quite
correct, almost correct, not quite correct, not correct at all,
not applicable) and (ii) subjective importance on a four-point
scale (of utmost importance, of great importance, of little
importance, of no importance, not applicable). An example
of a QPP question is given in Table 1.
An evaluation of QSP and QPP: two methods measuring patient satisfaction
Table 1 Example of a question from the QPP model
Waiting time at clinic
........................................................................................................................
I was . . .
Perceived reality
Subjective importance
.......................................................................................................................................................................................
. . . treated without any delay
Quite correct
4
Of utmost importance
4
during my visit at the clinic
Almost correct
3
Of great importance
3
Not quite correct
2
Of little importance
2
Not correct at all
1
Of no importance
1
Not applicable
X
Not applicable
X
Table 2 Example of a question from the QSP model
How
satisfied are you with the waiting time at the clinic?
.............................................................................................................................................................................................................................
Not satisfied
Very
Do not
at all
satisfied know
1
2
3
4
5
6
7
8
9
10
Φ
Φ
Φ
Φ
Φ
Φ
Φ
Φ
Φ
Φ
Φ
A personal quality index (PQI) was calculated for each
question by placing the response scores into the following
formula: PQI=subjective importance×(2×perceived reality
– subjective importance). The formula gives an index that
can vary between –8 and 16. High values indicate good
patient satisfaction and low values suggest that action should
be taken in order to improve satisfaction [12].
Figure 2 The relationship between the three concepts ‘quality
factors’, ‘patient satisfaction’ and ‘goals’ in the QSP model.
The QSP model
The QSP model originates from a method used to assess
customer satisfaction. The model has been developed and
adapted for assessing patient satisfaction. Studies have been
performed in eye clinics and dermatological clinics [13]. The
QSP model consists of three parts that are later combined
to give one score for each respondent:
(i) Patient satisfaction is rated on a global, standardized
scale and assessed by means of three questions. This
forms the Patient Satisfaction Index’ (PSI) which is
estimated from the weighted average of the three
questions.
(ii) Experienced quality of different dimensions or factors of quality are a priori regarded as important in
explaining patient satisfaction. Examples of such
factors are information, waiting-time and satisfaction
with medical care. Three or four questions were
formulated to cover each factor.
(iii) Experienced quality of factors is defined as the
clinic’s goals. Examples of these goals are increased
confidence and willingness to come back.
An example of a question from the QSP questionnaire is
given in Table 2. All questions were judged on a 10-point
scale.
The statistical QSP model consists of two parts:
(i) Part I. A structural model describes how the latent
variables patient satisfaction and the different quality
factors and goals are related to each other. The
relationship between the three concepts quality factors, patient satisfaction, and goals is described in
Figure 2. This model is used for the statistical
analysis.
(ii) Part II. Each concept in the model is measured with
a latent variable built up as a factor related to several
questions and this constitutes the measurement part
of the model. The relationships between the different
quality factors and patient satisfaction are expressed
as equations. This means that a regression coefficient
is calculated describing the importance of each factor
on the PSI. The regression factor can be interpreted
in terms of the importance of the factor studied
while at the same time controlling for the other
factors, i.e. keeping the rest of the factors constant.
The results can be presented in a coordinate system
with the factor under study on the y-axis and its
impact on the PSI shown on the x-axis.
259
J. Nathorst-Böös et al.
The statistical method used is a multivariate analysis based
on the principles used in the Partial Least Squares (PLS)
method [14].
The study was conducted at the Department of Obstetrics
and Gynecology at Karolinska Hospital, a university outpatient clinic with approximately 4000 visits each year. The
catchment area of the clinic is the northern part of Stockholm,
serving approximately 800 000 inhabitants. Apart from the
problems often observed at gynecological clinics, such as
bleeding disorders, Pap smear tests and HRT treatment, most
of the clinic patients are healthy.
Table 3 The eight domains and 37 questions used in the
QPP and QSP questionnaires
Domain
Medical care
Waiting time
Treatment by the doctor
Main measures
The first step was to perform a pilot study with open-ended
questions in order to see what issues were regarded as
important by the patients. A simple questionnaire form was
given to 100 patients who were asked to indicate three things
they considered good and three things they considered bad.
From this assessment the most important domains and
questions were identified. Suitable questions were then selected from the QPP questionnaire. This process was carried
out in conjunction with the nurses and doctors at the clinic.
It was considered important to make the questionnaire as
simple as possible and to exclude questions which seemed
inappropriate to the outpatient setting. The questions were
formulated in a similar way and arranged in the same order
as in the QPP and QSP questionnaires. The 37 questions
finally chosen covered eight different factors. The questions
and factors are presented in Table 3.
An additional 19 questions from the QSP form covered
Goals and General satisfaction. The questionnaires were
handed out by the receptionist to all women attending the
outpatient clinic and were alternated among the patients.
Every other patient received a QPP questionnaire and the
remaining patients received a QSP questionnaire. The women
were requested to fill in the forms during their visit but those
who for various reasons (lack of time being the most frequent
excuse) chose to fill in the forms after their visit received an
stamped addressed envelope so that they could return their
completed form by post. The number of forms distributed
to each group was 460. Two hundred and sixty (63%) of the
QSP questionnaires and 247 (54%) QPP questionnaires were
returned. After sorting out the forms with incomplete answers,
it was found that 60% of the QSP questionnaires and 52%
of the QPP questionnaires had been returned satisfactorily.
Statistical analysis
The PQI was calculated for the QPP from the formula
described earlier. The QSP was analysed using the PLS
method. Furthermore, a Structural Equation Modelling (SEM)
was performed for both methods using STREAMS 1.7 (Structural Equation Modelling Made Simple) [15]. By use of the
SEM technique, different features of the application of the
two methods could be evaluated. The aspects brought up in
this study were: (i) confirmatory one-factor models for the
quality factors were tested for goodness-of-fit and (ii) reliability estimates for the sum-index of variables for each
260
Treatment by the nurse
Information
Participation
Environment
Accessibility
Question area
Examination
Medical treatment
Doctor’s skill
Nurse’s skill
On the telephone
During the appointment
Showed you respect
Gave you sincere answers
Treated you in a positive way
Listened to you
Showed commitment and
care
Showed you respect
Gave you sincere answers
Treated you in a positive way
Listened to you
Showed commitment and
care
Examinations
Blood sampling
Treatment
Drugs
Results of tests
Where to turn with questions
Choose time for appointment
Choose doctor
Discuss alternative treatments
Choose if students should be
present or not during
examination
Finding your way to the clinic
Tidiness at clinic
Cosiness at clinic
Comfort at clinic
Atmosphere
Possibility of speaking with
the doctor in private
Telephone hours for making
an appointment
Reaching the nurse for an
appointment
Doctor’s telephone hours
Reaching the doctor by
phone
the clinic’s opening hours
quality factor were calculated [3]. Regression coefficients for
the relationships between the quality factors and ‘patient
satisfaction’ were estimated with standard errors taking
measurement errors in the 56 questions underlying the seven
quality factors into account.
An evaluation of QSP and QPP: two methods measuring patient satisfaction
Table 4 Background data of the QSP and QPP questionnaire respondents
QSP
QPP
Respondents
n=260
n=247
.......................................................................................................................................................................
Median age (age range)
42 (14–80)
41 (11–80)
Swedish as native language
86%
90%
What level of education do you have?
Elementary school, primary school or similar
22%
20%
Middle school, junior-high or similar
24%
23%
High school
21%
20%
University or college
34%
37%
Was this your first visit to the clinic?
Yes
26%
20%
No, I have been here before occasionally
21%
30%
No, I have been here several times before
53%
50%
How would you rate your present state of health?
Very good
30%
27%
Rather good
48%
52%
Neither good nor bad
16%
14%
Rather bad
5%
6%
Very bad
1%
1%
Results
Patients’ background data including age, native language,
education, present state of health and previous visits to the
clinic are presented in Table 4. The women’s ratings of the
different factors in the QPP and QSP questionnaires are
presented in Table 5. In the QPP model, the results regarding
perceived reality, subjective importance, personal quality index
are presented using both the PLS and the SEM-analysis
techniques. Treatment by the doctor and medical care received
the highest scores in the perceived reality domain. Accessibility
received the lowest score. The PQI was calculated according
to the formula given above, and the scores ranged between
3.1 and 14.1. The patients placed highest value on ‘treatment
by the doctor’ and ‘medical care’ but ‘accessibility’ and
‘participation’ received low scores.
In the QSP, the total PSI received a relatively high score
of 81.8 out of a possible 100. Also in the QSP model
‘treatment by the doctor’ and ‘medical care’ received high
scores while ‘participation’ and ‘accessibility’ received low
scores. As previously, a calculation was performed for every
factor studied in order to explain the variance in the PSI.
This means that a high value indicates high importance on
the PSI and vice versa. As a comparison, the regression
coefficient was also calculated for the QSP data using the
SEM-technique (Table 5).
In Figure 3 the mean values of the factors are plotted on
the y-axis and their impact on the PSI is shown on the xaxis as described above. The results can be illustrated in a
coordinate system with the ranking value on the y-axis and
its impact on the total PSI on the x-axis. Factors with high
scores and large impact on the total PSI are placed in quadrant
I, while factors with high scores but low impact are placed
in quadrant IV. In the same way, factors with low scores but
large impact are placed in quadrant II, while factors with low
scores and low impact are placed in quadrant III. ‘Treatment
by the doctor’ and ‘medical care’ were placed in quadrant I,
which consequently means that these factors had both high
impact and ranking. Also ‘environment’ turned out to have
a relatively high impact on the PSI while its ranking was
moderate. Accessibility was ranked as the lowest factor but
had a significant impact on the PSI. The three remaining
factors displayed no significant impact on the PSI. The
variance in the PSI explained by the seven quality factors
was as high as 79%. This indicates that about 62% of the
PSI variance was captured by the QSP questionnaire. The
alternative, SEM method was a two-step procedure: (i) testing
and adjusting a SEM model to the QSP data, and (ii)
estimation of regression coefficients with the so called Maximum Likelihood Method (MLM) taking measurement errors
into account [16]. The results are presented in Table 5 and
can be compared with the PLS regression coefficients. As
seen from the figures, results were found to be similar to
the PLS data.
Conclusion
Measuring patient satisfaction has become increasingly popular. It is important to discuss how these results can be
integrated into the health care process in order to increase
patient satisfaction. Early in the planning of a study it is
therefore important to integrate in what context the results
will be used. This should, if possible, include the headings:
‘goals’, ‘relevant questions’, ‘interpretation’ and ‘intervention’.
The aim of the present study was to perform an evaluation
of the two different methods with respect to the proposed
261
262
n.s., not significant.
Medical care
Treatment by the doctor
Treatment by the nurse
Information
Participation
Environment
Accessibility
3.64
3.68
3.45
3.33
2.68
3.23
2.53
(3.59–3.69)
(3.60–3.76)
(3.40–3.50)
(3.27–3.39)
(2.62–2.74)
(3.17–3.29)
(2.49–2.57)
3.76
3.83
3.51
3.72
3.27
3.20
3.23
(3.73–3.79)
(3.78–3.88)
(3.47–3.55)
(3.69–3.75)
(3.23–3.31)
(3.14–3.26)
(3.19–3.27)
13.0
13.5
11.8
11.7
7.8
10.4
6.0
8.81
9.03
8.87
8.49
7.44
8.03
6.26
(8.73–8.89)
(8.92–9.14)
(8.79–8.95)
(8.40–8.58)
(7.34–7.54)
(7.93–8.13)
(6.16–6.36)
(mean±95% CI)
2.5 (1.3–3.7)
3.2 (1.8–4.6)
0.5± n.s.
0.5± n.s.
0.6 (0–1.4)
2.7 (1.7–3.7)
1.4 (0.6–2.2)
(mean±95% CI)
4.5 (1.7–7.3)
1.8 (0.4–3.2)
0.4 (0–1.2)
0.8±n.s.
0.9 (0–3.1)
2.4 (1.4–4.4)
1.1 (0.5–1.7)
QSP
SEM
QPP
QPP
QSP
Importance
Importance
Perceived reality
Subjective importance
QPP
Experienced quality
PLS regression
SEM regression
Quality
factors
(mean±95%
CI)
(mean±95%
CI)
PQI
(mean±95%
CI)
coefficient
coefficient
................................................................................................................................................................................................................................................................................................................
Table 5 The women’s judgement of the different factors in the QPP and QSP questionnaires
J. Nathorst-Böös et al.
An evaluation of QSP and QPP: two methods measuring patient satisfaction
Figure 3 The mean value of the different quality factors in
the QSP model and their impact on the PSI.
field of application, and the methods’ strengths and weaknesses. Sixty-three per cent of the QSP and 54% of the QPP
forms were returned indicating that the QPP form was
somewhat more time consuming to complete, resulting in a
higher dropout rate. Regarding the SEM analysis of the
dimensions of quality; that is of the seven quality factors, the
two methods generated the same factor structure. As expected, a lower reliability index was found for the four-point
QPP scale than for the 10-point QSP scale. This means that
for this group of women patients a better measure is achieved
if the more differentiated 10-point scale is used. For other
patients, for example elderly people, the four-point response
scale may be preferred, as it is simpler and quicker to answer
and generates the same answer pattern, although it may have
a somewhat higher measurement error rate.
The ease of interpreting the results and developing a basis
of action is different for each model. In the QSP, this is
incorporated directly into the method of questioning which
is built upon a structural equation model that shows how
quality factors, patient satisfaction and the clinic’s goals are
related to each other. The equation results in a series of
coefficients of importance indicating the different factors’
relative impact on patient satisfaction. These relationships can
subsequently be presented graphically and the presentation
describes the mutual relationship of the factors involved and
gives a good indication for how intervention can be made.
The high level of explained variance in the PSI indicates that
the modelling of the QSP data has given reliable results.
In this study the same questions concerning experienced
reality were used in both the QSP and the QPP. The QPP
form also included the subjective judgement of importance.
The QPP model has its basis in orally conducted interviews
aimed at investigating what issues in health care are considered
important by patients. This process has resulted in a question
bank where questions are grouped into specific categories
and from which suitable questions can be selected. This firststep procedure is important; it addresses the issue of validating
the questionnaire and is not included in the QSP model. In
the QPP model, a PQI is directly calculated on the basis
of the patient’s estimation of subjective importance and
experienced reality. The properties of the formula resulting
in this index are, however, inconsistent and the same index
can be reached by means of several combinations of importance and reality factors, which makes interpretation difficult.
The index gives no information on how patient satisfaction
changes if quality improvement is made. As a contrast between
these two methods, analysis of these additional data specific
to the QPP was set as a secondary priority, mainly because
this method showed a tendency towards estimating most
subjective importance factors as ‘very important’ or ‘important’.
In summary, the advantages of the QPP model are that it
has a comprehensive and solid question bank. This consists
of 56 questions extracted by means of multiple regression
analysis from 900 indicators. The QSP model’s advantage is
its immediate usefulness and its clear graphic presentation.
The importance of the different factors are grounded in the
patients’ values but can be extracted from the model. An
integration and further development of these two approaches
could prove useful.
References
1. Carr-Hill R. The measurement of patient satisfaction. J Public
Health Med 1992; 14: 236–249.
2. Raphael W. Do we know what patients think? A survey comparing the views of patients, staff and committee members. Int
J Nursing Studies 1967; 4: 209–223.
3. Locker D, Dunt D. Theoretical and methodological issues in
sociological studies of consumer satisfaction with medical care.
Soc Sci Med 1978; 12: 283–292.
4. French K. Methodological considerations in hospital patient
opinion surveys. Int J Nursing Studies 1981; 18: 7–32.
5. Carr-Hill R, Dixon P, Thompson A. Too simple for words.
Health Serv J 1989; 99: 728–729.
6. Meredith P, Wood C. Aspects of patient satisfaction with
communication in surgical care: confirming qualitative feedback
through quantitative methods. Int J Qual Health Care 1996; 8:
253–264.
7. Ware JE, Sherbourne CD. The MOS 36-Item short form health
survey (SF-36): 1 Conceptual framework and item selection.
Med Care 1992; 30: 473–483.
8. Hays RD, Larson C, Nelson EC, Batalden PB. Hospital quality
trends: a short form patient based measure. Med Care 1991; 29:
661–663.
9. Tarlov A, Ware JE, Greenfield S et al. The Medical Outcomes
Study: An application of methods for monitoring the results of
medical care. J Am Med Assoc 1989; 262: 925–939.
10. Wilde B, Starrin B, Larsson G, Larsson M. Quality of care from
263
J. Nathorst-Böös et al.
a patient perspective – a grounded theory study. Scand J Caring
Sci 1993; 7: 113–120.
11. Scott A, Smith RD. Keeping the customer satisfied: Issues in
the interpretation and use of patient satisfaction surveys. Int J
Qual Health Care 1994; 16: 353–359.
12. Wilde B, Larsson G, Larsson M, Starrin B. Quality of care.
Development of a patient-centered questionnaire based on a
grounded theory model. Scand J Caring Sci 1994; 8: 39–48.
13. Eckerlund I, Jönson B, Tambour M, Westlund AH. Change
oriented patient questionnaires – testing a new method at three
264
departments of ophthalmology. Int J Health Care Quality Assurance
1997; 10: 254–259.
14. Fornell C, Cha J. Partial Least Squares. In Bagozzi RP, ed.,
Advanced Methods of Marketing Research. Cambridge: Blackwell,
1994: pp.52–78.
15. Gustaffson HE, Stahl PA. STREAMS User’s Guide Version 2.5.
Gothenburg: Multivariate Ware, 1999.
16. Hox JJ, Becger TM. An Introduction to Structural Equation
Modelling. Family Science Review 1998; 11: 354–372.
Accepted for publication 23 February 2001