Download Effect of computerisation on Australian general practice: does it

Quality in Primary Care 2010;18:33–47
# 2010 Radcliffe Publishing
Research paper
Effect of computerisation on Australian
general practice: does it improve the
quality of care?
Joan Henderson BAppSc (HIM) (Hons) PhD (Med)
Senior Research Fellow
Graeme Miller MBBS PhD FRACGP
Associate Professor, Medical Director
Helena Britt BA PhD
Associate Professor, Director
Ying Pan BMed MCH
Senior Analyst
Family Medicine Research Centre, Discipline of General Practice, School of Public Health, University of
Sydney, Australia
ABSTRACT
Background There is an assumption expressed in
literature that computer use for clinical activity will
improve the quality of general practice care, but
there is little evidence to support or refute this
assumption.
Aim This study compares general practitioners
(GPs) who use a computer to prescribe, order tests
or keep patient records, with GPs who do not, using
a set of validated quality indicators.
Methods BEACH (Bettering the Evaluation and
Care of Health) is a continuous national crosssectional survey of general practice activity in
Australia. A sub-sample of 1257 BEACH participants between November 2003 and March 2005
were grouped according to their computer use for
test ordering, prescribing and/or medical records.
Linear and logistic regression analysis was used
to compare the two groups on a set of 34 quality
indicators.
Results Univariate analyses showed that computerised GPs managed more problems; provided fewer
medications; ordered more pathology; performed
more Pap smear tests; provided more immunisations; ordered more HbA1c tests and provided
more referrals to ophthalmologists and allied health
workers for diabetes patients; provided less lifestyle
counselling, and had fewer consultations with
Health Care Card (HCC) holders. After adjustment,
differences attributable solely to computer use were
prescribed medication rates, lifestyle counselling,
HCC holders and referrals to ophthalmologists.
Three other differences emerged – computerised
GPs provided fewer referrals to allied health workers
and detected fewer new cases of depression, and
fewer of them prescribed anti-depressants. Twentythree measures failed to discriminate before or after
adjustment.
Conclusion Deciding on ‘best quality’ is subjective.
While literature and guidelines provide clear parameters for many measures, others are difficult to
judge. Overall, there was little difference between
these two groups. This study has found little evidence to support the claim that computerisation of
general practice in Australia has improved the
quality of care provided to patients.
Keywords: clinical computer use, family practice,
quality indicators, quality of health care
34
J Henderson, G Miller, H Britt et al
How this fits in with quality in primary care
What do we know?
There is an assumption that using a computer will improve the provision of care in many areas of the health
system, including general practice. To date there is little evidence that this is the case, and US studies have
produced little support for this claim.
What does this paper add?
This paper applies a set of quality indicators to national representative general practice (GP) activity data to
compare the practice behaviour of GPs who incorporate a computer in their clinical activity with those who
do not. The results from the Australian setting support those so far reported from the USA – that to date, the
use of a computer for clinical activity has done little to improve the quality of primary care.
Introduction
There is an underlying premise apparent from the
literature of the past three decades that using a computer will improve the provision of care in many areas
of the health system, including general practice. Current claims reference previous work, which paper
trails often show to be based on suppositions made
some 15 to 20 years earlier; for example Garrido et al
(2005) state that ‘Electronic health records reduce
uncertainty by providing greater accessibility, accuracy and completeness of clinical information than
their paper counterparts’, referencing a 1991 General
Accounting Office (GAO) report.1 The GAO report
(p. 25) actually concluded that ‘automated systems
show promise’, that speed of record transfer and
accuracy of information ‘should improve the quality
of care’ and adds that ‘no fully automated medical
record system exists, so the strengths and weaknesses
of such a system have not been documented and are
not clearly understood’.2
In Australia, practice computerisation has been
encouraged since the 1990s through incentives and
accreditation processes, and a variety of clinical software products is available. Computers may be used
for a range of functions from use for administrative
purposes only, to being fully incorporated into all
levels of clinical activity.3,4 The use of computers (at
all), and the level of use for clinical activity, is entirely
discretionary both between and within practices.
However, in 1999 Richards et al reported that they
had found little hard evidence that the general use of
computers in Australia improves efficiency at individual practice level or benefits the health sector generally, or that improving outcomes was an aim when
designing information systems.5 This is not just a local
trend. Healthfield et al proposed that decision makers
in the UK and the USA may be being ‘swayed by the
general presumption that technology is of benefit to
health care and should be wholeheartedly embraced’
while the evidence to either support or oppose this
supposition is still scarce.6
Mitchell and Sullivan (2001) undertook a systematic review of world literature on primary care computing from 1980 to 1997.7 Most studies identified
some positive effects of computerisation in selected
areas, but they found only 17 assessing the impact of
computers on patient outcomes, a number they concluded insufficient to measure the real benefits for
patients.7 While there is some evidence that computer
use is associated with individual improvements to the
quality of care,8–10 there is also emerging evidence that
the computer, while solving problems in some areas, is
causing or accentuating problems in others.11–14
Recently, there has been increasing demand for
information on health care quality by health economists,
policy makers, health professionals and consumers.15
While this is an international trend, the approach to
quality measurement and the capacity to validly assess
quality varies widely between countries.16,17 The use of
quality indicators has become accepted as a reasonable
approach for assessing quality. The focus has shifted
in recent times, from process measures which reflect
what was done, to outcome measures, which show the
effect of what was done.18
Over the past 15 years, computer use by Australian
GPs has increased such that over 97% have a computer
available at their practice,19 and it is therefore timely to
investigate how the incorporation of the computer
into clinical activity affects the quality of care provided
by GPs. In a previous study we reported the extent
and utilisation of computer use in Australian general
practice.3 This study aims to compare GPs who use a
computer in their clinical activity with those who do
not, on a range of quality indicators developed for use
with primary care data, to determine whether the use
of the computer has improved the quality of care
provided to patients.
Effect of computerisation on Australian general practice: does it improve the quality of care?
Methods
This study is an analysis of data from the national
BEACH (Bettering the Evaluation And Care of Health)
program. The BEACH methods are reported in detail
elsewhere, but in summary BEACH is a continuous,
national, paper-based, cross-sectional survey of general
practice activity in Australia. Approximately 1000 GPs
participate annually, recruited from a national rolling
random sample drawn by the Australian Government
Department of Health and Ageing (DoHA). Participating GPs provide demographic information about
themselves and their practices, including questions
about their computer use, on a GP profile questionnaire. They also provide patient demographics and
encounter information for 100 consecutive, consenting, unidentified patients. The age–sex distribution of
patients at BEACH encounters is compared with that
of all GP encounters claimed through Medicare,
Australia’s universal healthcare scheme, and shows
excellent precision.20
The 1319 GP participants who completed the
BEACH survey between November 2003 and March
2005 were divided into two groups as follows:
1 Clinical computer users Defined as those who use
a computer for clinical functions, e.g. prescribing
and/or test ordering and/or medical records, with
or without internet and/or email.
2 Non-clinical computer users Defined as those
who use a computer for administrative functions
and/or internet and/or email only, without use of
clinical components available in the medical software (prescribing, test ordering, medical records).
Those GPs who did not use a computer at all were also
included in the latter group. Following univariate
analysis, the extent to which resulting differences
between the two groups were explained by other
variables was identified through a series of adjustments using logistic and multiple regression.
Quality indicators
In the absence of an evidence-based model for
determining how computers would alter behaviour
and affect quality, we approached the problem from
the perspective of ‘best quality’ and compared clinical
computer users and non-clinical computer users to see
which group performed ‘best’. To make this assessment, we measured their behaviour against a set of
quality indicators applicable in a primary care setting.
A set of 36 quality indicators validated in a previous
study using BEACH data21 were used to compare the
practice behaviour of GPs assigned to the two groups.
35
Hypotheses
Based on the assumption that the use of computers
will improve health outcomes, the overall hypothesis
was that clinical computer users would provide a
‘better’ standard of care. The individual hypothesis
and rationale for each domain of care was also based
on this assumption. Arrows in Tables 1(a) and 1(b)
specify the direction hypothesised as ‘better’ quality
for each indicator.
The average length of consultation in minutes was
calculated from recorded start and finish times for a
sub-sample of patient consultations with GPs in each
group. Encounters were designated as either long or
prolonged based on their Medicare Benefits item
number.22 Problems managed by GPs were classified
according to the International Classification of Primary Care, Version 2.23 Medications were classified
using an in-house system called the Coding Atlas for
Pharmaceutical Substances (CAPS).
Statistical analysis
Conventional simple random sample methods were
used for the GP-based statistical analyses. Results are
reported as proportions when describing events that
can occur only once per GP or per patient encounter,
but as rates per 100 encounters where events can occur
more than once per consultation. As the patient
encounters were a cluster-based sample, we adjusted
the 95% confidence intervals and P values for the
single-stage clustered study design using procedures
in SAS version 8.224 and STATA version 8.0.25
We made univariate comparisons of characteristics
of the GPs in each group (listed in Box 1), eliminated
those highly correlated with others, and used simple
logistic regression to identify those associated
(P < 0.10) with clinical computer use. We used stepwise procedures in logistic regression analysis to
identify characteristics independently related to clinical computer use (P < 0.05). A series of models were
built on a hierarchical basis with predictors fitted
depending on the outcome of interest. Predictors
included GP and practice characteristic outcomes,
patient, morbidity and treatment outcomes. Models
used for outcomes are specified in the footnotes to
Tables 1(a) and 1(b). Logistic regression was used to
analyse categorical outcomes, and linear regression for
continuous and ordinal outcomes, after adjusting for
potential confounding variables.
Test ordering
The denominator for clinical computer users included
GPs who used a computer for any clinical purpose, but
there were a number of GPs in this group who did not
use the test ordering function of their clinical software.
J Henderson, G Miller, H Britt et al
36
Box 1 GP and practice characteristics compared in simple logistic regression analysis and
then used in step-wise logistic regression analyses
GP characteristics
Practice characteristics
Age (<45, 45–54, 55+ years)
*
{
Sex
*
{
Place of graduation (Australia/
other)
FRACGP status (yes/no)
Years in general practice (<10, 10–
19, 20+)
*
{
*
{
{
§
Years since graduation (<20,20–
29, 30+)
Sessions per week (<6, 6–10, 11+)
{
§
{
Direct patient care hours per week
(<31, 31–40, 41–50, 51+)
Work in past 4 weeks –
in residential aged care facility
(yes/no)
as a locum (yes/no)
*
as salaried/session hospital
medical officer (yes/no)
in a deputising service (yes/no)
*
Whether all patients are bulkbilled (yes/no)
Any consultations in language
other than English (yes/no)
Registered with Department of
Veterans’ Affairs (yes/no)
Registrar status (in GP training)
(yes/no)
Size of practice (solo, 2–4, 5–10,
11+ GPs)
Practice location by RRMA1
(metropolitan/rural)
Practice location by ASGC2
(major city/not major city)
Practice location by State
Socio-economic status by SEIFA3
(disadvantaged <4 SEIFA/less
disadvantaged SEIFA 4–11)
*
{
*
{
Practice accreditation status (yes/
no)
Practice nurse at major practice
address (yes/no)
After-hours patient care arrangements (own or cooperative/
deputising service)
*
{
*
{
{
{
{
Status as a teaching practice for
undergraduates or registrars
*
*
{
*
{
{
{ Variables that were found to be highly correlated with other variables and were therefore not retained in the modelling process for
clinical computer use
* Variables that showed some association (P<0.10) with use of a computer for clinical purposes, and were therefore included in the
logistic regression analysis for clinical computer use
§ Variables that were found to be highly correlated with other variables and were therefore not retained in the modelling process for
computer use for test ordering
{ Variables that showed some association (P<0.10) with use of a computer for clinical purposes, and were therefore included in the
logistic regression analysis for computer use for test ordering
Note: FRACGP = Fellowship of the Royal Australian College of General Practitioners
1
Australian Government Department of Health and Ageing. Rural, Remote and Metropolitan Areas (RRMA) classification.
www.health.gov.au/internet/wcms/publishing.nsf/content/work-bmp-where-rrma
2
Australian Bureau of Statistics (ABS). Australian Standard Geographical Classification (ASGCT). Canberra: Australian Bureau of
Statistics, 2004.
3
Australian Bureau of Statistics. Census of population and housing: Socio-Economic Indexes for Areas (SEIFA), Australia. Canberra:
Australian Bureau of Statistics, 2001.
Effect of computerisation on Australian general practice: does it improve the quality of care?
Table1(a) Univariate and multivariate analysis of quality indicators
Quality
indicator
Consultation
length (in
minutes)
GPs using a
computer for
clinical purposes
GPs not using a
computer for
clinical purposes
Unadjusted*
Adjusted(a)
n
n
Regression P value
coefficient
Regression P value
coefficient
34 633
n
Mean
15.0
6084
Rate per n
100 of
(n)
Mean
15.0
0.05
0.90
Rate per Regression P value
100 of
coefficient
(n)
–0.38
0.40
Regression P value
coefficient
Long consultations per 100
encounters (")
99 153
12.2
17 478
10.7
1.50
0.14
–0.41(e)
0.70
Prolonged
consultations
per 100
encounters (")
99 153
1.0
17 478
1.1
1.37
0.24
–0.37(e)
0.76
Reasons for
encounter per
100 encounters
(")
106 900
150.7
18 800
150.1
0.54
0.81
0.59(d)
0.82
Problems
managed per
100 encounters
(")
106 900
150.5
18 800
144.1
6.42
0.003
3.44(d)
0.12
Clinical
treatments per
100 encounters
(")
106 900
39.7
18 800
40.1
–0.40
0.88
–2.72(e)
0.32
Procedural
treatments per
100 encounters
(")
106 900
17.6
18 800
18.4
–0.82
0.57
–1.26(e)
0.31
Prescribed
106 900
medications per
100 encounters
(#)
81.9
18 800
89.8
–7.96
0.01
–6.54(e)
0.02
Allied health
106 900
referrals per 100
encounters (")
3.0
18 800
2.7
0.28
0.29
–0.55(e)
0.03
Hospital
106 900
referrals per 100
encounters (#)
0.6
18 800
0.7
–0.17
0.23
–0.14(e)
0.47
37
38
J Henderson, G Miller, H Britt et al
Table1(a) Continued
Specialist
106 900
referrals per 100
encounters (#)
8.3
18 800
7.5
0.83
0.06
–0.01(e)
0.98
Total
investigations
per 100
encounters (#)
106 900
51.3
18 800
41.7
9.6
<0.001
–0.60 (e)
0.82
Pathology test
orders per 100
encounters (#)
106 900
41.6
18 800
32.6
8.96
<0.001
–0.11(e)
0.96
Imaging test
orders per 100
encounters (#)
106 900
8.6
18 800
8.2
0.44
0.38
–0.53(e)
0.35
Other
investigations
per 100
encounters (#)
106 900
1.1
18 800
0.9
0.22
0.05
0.04(e)
0.78
Pap smear per
100 encounters
with females
aged 15–70 yrs
(")
43 090
5.7
7095
4.1
1.58
0.045
–0.16(c)
0.82
All immunisations per 100
encounters with
patients < 5
years old (")
6740
20.5
868
15.2
5.24
0.036
3.50(b)
0.34
106 900
7.2
18 800
8.9
–1.70
0.03
–1.72(d)
0.03
PSA tests per
100 screening
contacts with
males > 50 years
old (#)
1674
9.8
214
13.1
–3.29
0.19
–4.85(b)
0.08
HbA1c per 100
contacts with
diabetes (")
3432
25.1
688
17.6
7.53
0.001
3.10(b)
0.24
Referrals to
ophthalmologist
or allied health
per 100 contacts
with diabetes (")
3432
7.1
688
3.6
3.50
<0.001
2.94(b)
0.002
Lifestyle
counselling per
100 encounters
(")
Effect of computerisation on Australian general practice: does it improve the quality of care?
Table1(a) Continued
n
Rate per n
100 of
(n)
Rate per Regression P value
100 of
coefficient
(n)
Regression P value
coefficient
ACE inhibitors 5838
per 100 contacts
with LVF, IHD,
diabetes or
cerebrovascular
disease (")
5.9
1075
4.5
1.48
0.07
0.16(b)
0.86
Aspirin or
clopidogrel per
100 contacts
with LVF, IHD,
diabetes or
cerebrovascular
disease (")
5838
8.7
1075
9.6
–0.90
0.46
–1.93(b)
0.14
Warfarin per
100 contacts
with atrial
fibrillation (")
906
35.4
145
40.0
–4.57
0.42
–5.23(b)
0.42
Imaging per 100 5036
contacts with
lower back pain
or strain/sprain
(#)
14.8
917
16.3
–1.48
0.37
–2.73(b)
0.15
NSAIDs per 100 2347
contacts with
arthritis (all
types) and >65
(#)
38.0
394
39.6
–1.59
0.66
–1.18(b)
0.77
Analgesics (non 2347
NSAID) per 100
contacts with
arthritis and >65
27.2
394
29.7
–2.51
0.41
–3.51(b)
0.38
Antibiotics
5072
prescriptions per
100 contacts
with URTI (#)
34.7
912
41.2
–6.49
0.08
2.66(b)
0.54
Antibiotics
3841
prescriptions per
100 contacts
with new URTI
(#)
36.9
714
41.7
–4.82
0.24
3.65(b)
0.44
Antibiotics
1122
prescriptions per
100 contacts
with URTI in
children aged <5
(#)
20.4
154
24.7
–4.27
0.42
0.60(b)
0.92
39
40
J Henderson, G Miller, H Britt et al
Table1(a) Continued
106 900
0.7
18 800
0.8
–0.07
0.39
–0.21(d)
0.043
Counselling per
100 contacts
with depression
(")
4342
13.5
716
12.0
1.53
0.41
1.87(b)
0.39
Antidepressants
per 100 contacts
with depression
(#)
4342
61.3
716
66.6
–5.31
0.07
–7.57(b)
0.02
Benzodiazepine
per 100 contacts
with insomnia
(#)
1719
57.6
284
60.6
–2.97
0.53
–0.16(b)
0.97
New diagnosis
of depression
per 100
encounters (")
*Missing data removed.
(a)
Adjusted using one of the following models:
(b)
Model controlling for GP age; GP sex; FRACGP status; work in deputising service in preceding 4 weeks; bulk-billing for all
patients; practice accreditation status; presence of a practice nurse at the major practice address
(c)
Model controlling for all variables in (b) plus patient age
(d)
Model controlling for patient age; patient sex; Commonwealth Health Care Benefits Cardholder status; Veterans’ Affairs card
holder status; NESB status; Aboriginal or Torres Strait Islander status; ‘new patient’ status; GP and practice characteristics included
(b)
in
(e)
Model controlling for the presence or absence of problems managed by ICPC-2 Chapter at the encounter; the GP, practice and
patient characteristics included in (d)
Note: (") and (#) denotes the direction hypothesised as ‘better’ quality for each indicator; bold = statistically significant difference;
PSA = prostate specific antigen; LVF = left ventricular failure; IHD = ischaemic heart disease; HbA1c = haemoglobin, type A1c; ACE
= angiotensin converting enzyme; URTI = upper respiratory tract infection; NSAID = non-steroidal anti-inflammatory drug
Therefore, we compared test ordering behaviour for
the set of all clinical computer users and their counterparts in the first instance, and then repeated the
investigation for the eight test ordering quality indicators, with the GPs grouped according to their use of
the test ordering function of their software.
8.821. The result for a two-sample comparison of
proportions was a power of 0.8002 (80%) to detect a
3.3% difference between estimates, and of 0.8987
(90%) to detect a 3.8% difference between estimates.
Results
Compared with their counterparts, GPs who used a
computer for clinical activity were significantly more
likely to: be female (P = 0.001); younger (P < 0.001);
have had fewer years in general practice (P < 0.001);
have trained for their primary medical degree in
Australia rather than overseas (P = 0.001); be Fellows
of the RACGP (P < 0.001); work in larger practices
(with five or more GPs; P < 0.001); work in accredited
practices (P < 0.001); and have a practice nurse at their
major practice address (P < 0.001). They were significantly less likely to: bulk-bill Medicare for all their
patients (P < 0.001); work in solo practices (P < 0.001);
or work in major cities or in other metropolitan areas
(P = 0.0002).
Individual computer use was determined for 1257 of
the 1319 GPs. There were 1069 GPs in the clinical
computer use group (106 900 patient encounters) and
188 in the comparison group (18 800 encounters).
There were 901 GPs who reported using computers for
test ordering, and 356 who did not. The sub-sets of
consultations with start and finish times recorded
included 34 633 consultations with clinical computer
users and 6084 consultations with non-clinical computer users. Using the sample sizes for 106 900 and
18 800 encounters, the intra-cluster correlation was
calculated as 0.079 with a variance inflation factor of
GP and practice characteristics
Effect of computerisation on Australian general practice: does it improve the quality of care?
Table 1(b) Univariate and multivariate analysis of quality indicators
GPs using a
computer for test
ordering
GPs not using a
computer for test
ordering
Adjusted(a)
Unadjusted*
n
Rate per n
100 of
(n)
Rate per Regression P value
100 of
coefficient
(n)
Regression P value
coefficient
Pathology test
orders per 100
encounters (#)
90 100
42.6
35 600
34.3
8.25
<0.001
1.28(e)
0.41
Imaging test
orders per 100
encounters (#)
90 100
8.6
35 600
8.4
0.19
0.62
–0.59(e)
0.15
Other
investigations
per 100
encounters (#)
90 100
1.1
35 600
1.0
0.18
0.046
0.04(e)
0.69
Total
investigations
per 100
encounters (#)
90 100
52.3
35 600
43.7
8.62
<0.001
0.73(e)
0.68
Pap smear per
100 encounters
with females
aged 15–70 yrs
(")
36 751
5.9
13 434
4.3
1.57
0.006
–0.09(c)
0.85
PSA tests per
100 screening
contacts with
males > 50 years
old (#)
1408
9.7
480
11.5
–1.73
0.34
–2.22(b)
0.27
HbA1c per 100
contacts with
diabetes (")
2838
26.3
1282
18.6
7.69
<0.001
4.72(b)
0.015
Imaging per 100
contacts with
lower back pain
or strain/sprain
(#)
4182
14.8
1771
15.4
–0.59
0.64
–1.32(b)
0.34
* Missing data removed
(a)
Adjusted using one of the following models:
(b)
Model controlling for GP age; GP sex; FRACGP status; work in deputising service in preceding 4 weeks; bulk-billing for all
patients; practice accreditation status; presence of a practice nurse at the major practice address
(c)
Model controlling for all variables in (b) plus patient age
(d)
Model controlling for patient age; patient sex; Commonwealth Health Care Benefits Cardholder status; Veterans’ Affairs card
holder status; NESB status; Aboriginal or Torres Strait Islander status; ‘new patient’ status; GP and practice characteristics included
in (b)
(e)
Model controlling for the presence or absence of problems managed by ICPC-2 Chapter at the encounter; the GP, practice and
patient characteristics included in (d)
Note: (") and (#) denotes the direction hypothesised as ‘better’ quality for each indicator; bold = statistically significant difference;
PSA = prostate specific antigen; LVF = left ventricular failure; IHD = ischaemic heart disease; HbA1c = haemoglobin, type A1c;
ACE = angiotensin converting enzyme; URTI = upper respiratory tract infection; NSAID = non-steroidal anti-inflammatory drug
41
42
J Henderson, G Miller, H Britt et al
Quality indicators
Results of the univariate and multivariate analyses for
the quality indicators are shown in Table 1(a) for
clinical computer users and for non-users. Table 1(b)
shows the indicators reanalysed according to GPs’
computer use for test ordering.
In total the GPs who used a computer in their
clinical activity differed from GPs who did not on
only seven indicators. The unadjusted regression coefficients showed almost twice as many differences,
but for many of these results, the adjusted regression
coefficients showed that the differences were explained
by influences other than the GP’s use of a computer.
Significant differences attributable to clinical computer
use included: consultations with Commonwealth Health
Care Benefits Card holders (adjusted odds ratio 0.83; P
= 0.035, results not tabulated); overall prescribing
rate; antidepressant prescribing; detecting new cases
of depression; referrals to ophthalmologists for diabetes
patients; referrals to allied health professionals and
provision of lifestyle counselling.
Tables 2(a) and 2(b) provide an overview of all the
quality indicators examined, including the hypothesis
for each. It shows the indicators that did not discriminate at either univariate or multivariate levels of
analysis, or both (marked with a single X). For other
indicators, the use of a tick (3) shows where differentiation occurred between clinical computer users
and their counterparts, by showing that the indicator
discriminated and the hypothesis was accepted in
either the unadjusted results or after statistical adjustments were made, or both. For some indicators,
the hypothesis was accepted at the univariate level
(as indicated with a tick (3)), but ultimately rejected
following adjustment (marked with a single X). Where
the hypothesis was rejected, and the outcome was a
reversal of the hypothesis, the result is marked with a
double X (XX).
Discussion
On balance these results suggest that the use of a
computer has had little effect on the quality of care
provided by the GPs to their patients. After adjustment for other characteristics, clinical computer users
performed ‘better’ on three of 34 quality indicators,
and ‘worse’ on four. There was no difference in their
performance over the remaining 29. Where the indicators were used to compare test ordering behaviour
through the computer, only one difference emerged;
in this instance, those ordering tests through their
software performed ‘better’ than their counterparts.
In total, from 44 indicators, clinical computer users
performed ‘better’ on four and ‘worse’ on four, while
no differences were discernible for the remaining 36.
What was different?
Why the groups differed on these particular indicators
and not others is not readily apparent. One explanation for the lower overall prescribing rate of clinical
computer users is that some clinical software in
Australia defaults to the maximum number of repeats
allowed under Pharmaceutical Benefits Scheme rules
when a prescription is written.12 Unless the default is
manually overridden, patients would be given the
maximum number of medication repeats allowable,
and would not need to return for prescriptions as
frequently. However, why these GPs prescribe fewer
antidepressant medications for patients with depression, but do not differ on rates for other medications, is unclear.
Decision support tools may have influenced the
computer users to provide more referrals to ophthalmologists for patients with diabetes, yet the referral
rate to allied health professionals overall was lower for
computerised GPs, and it would seem unlikely that
these tools would single out ophthalmologists over
other healthcare providers. Added to their higher
ordering of HbA1c tests, it could be inferred that the
clinical use of a computer results in a GP providing
better care for diabetic patients. Electronic reminders
are effective in modifying physician behaviour26 and it
might follow that GPs who are exposed to electronic
reminders for diabetic patients in their software respond and therefore act differently. However, GPs
who do not use the test ordering function of their
software would still be exposed to these reminders, so
electronic flags alone are unlikely to have caused this
difference in test ordering behaviour.
We hypothesised that clinical computer users would
detect more cases of depression, yet they detected
fewer new cases – a reversal of the hypothesis. However, their overall management rate of depression did
not differ; neither did their rate of counselling of
patients with depression. Depression is an illness
which is not easily detected, particularly in situations
where the patient may have difficulty disclosing the
full extent of their symptoms.27 Managing a problem
once it has been diagnosed and making a new diagnosis are two different scenarios and in this case
perhaps the division of consultation time between
patient and computer, and the diversion of attention
from the patient, reduces the opportunity to detect the
unspoken signals which GPs often rely on in these
situations.
Computer proficiency should also be considered
with regard to the length of consultation. The GP
groups were identical on this indicator, suggesting no
Effect of computerisation on Australian general practice: does it improve the quality of care?
Table 2(a) Summary of results for quality of care indicators – clinical vs non-clinical
computer use
Domain of care
Descriptive
result
Adjusted
result
1 Have a longer mean consultation length
(in minutes)
X
X
2 Have a greater proportion of long
consultations
X
X
3 Have a greater proportion of prolonged
consultations
X
X
4 Report more patient reasons for encounter
X
X
5 Manage more problems per encounter
3
X
Non-pharmacological
management
6 Provide more clinical treatments
X
X
7 Perform more procedural work
X
X
Pharmacological
management
8 Prescribe fewer medications overall
3
3
Referrals
9 Refer more often to allied health services
X
XX
10 Refer less often to hospitals
X
X
11 Refer less to specialists
X
X
12 Order fewer investigations in total
XX
X
13 Order fewer pathology tests
XX
X
14 Order fewer imaging tests
X
X
15 Order fewer other investigations
X
X
16 Have relatively more consultations with
indigenous persons
X
X
17 Have relatively more consultations with
holders of a Commonwealth Health Care
Benefits card
XX
XX
18 Perform relatively more Papanicolau tests
for 15–70-yr-old women
3
X
19 Perform more immunisations for
children < 5 yrs of age
3
X
20 Provide lifestyle counselling more often
XX
XX
Inappropriate
preventive care
21 Order relatively fewer PSA tests for males
aged > 50 yrs old
X
X
Diabetes management
22 Order relatively more HbA1c tests for
patients with diabetes
3
X
23 Refer patients with diabetes more often to
ophthalmologists and allied health services
3
3
Consultation length
and complexity
Tests and
investigations
Social disadvantage
Appropriate
preventive care
Measure of quality hypothesis: when compared
to GPs not using computers clinically, clinical
computer users will:
43
44
J Henderson, G Miller, H Britt et al
Table 2(a) Continued
Cardiovascular disease 24 Prescribe ACE inhibitors for IHD, heart
management
failure, diabetes and cardiovascular disease
at a higher rate
X
X
25 Prescribe aspirin or clopidogrel for patients
with heart failure, IHD, diabetes or
cerebrovascular disease at a higher rate
X
X
26 Prescribe warfarin for atrial fibrillation at a
relatively higher rate
X
X
27 Order relatively less imaging for low back
pain and sprains/strains (any site)
X
X
28 Prescribe relatively fewer NSAIDs for
arthritis patients > 65 yrs old
X
X
29 Prescribe more simple and compound
analgesics for these patients
X
X
X
X
31 Prescribe antibiotics less often for new
cases of URTI
X
X
32 Prescribe antibiotics less often for URTI in
children <5 yrs old
X
X
Psychological problem 33 Detect new cases of depression more often
management
34 Have higher counselling rates for
depression
X
XX
X
X
35 Have lower antidepressant prescribing
rates for depression
X
3
36 Have lower benzodiazepine prescribing
rates for depression
X
X
Musculoskeletal
disease
Infection management 30 Prescribe antibiotics less often for upper
respiratory tract infection
Note: 3 – hypothesis accepted; X – hypothesis rejected, there being no significant differences between the groups; XX – hypothesis
rejected, result reversed from that hypothesised
difference in the level of quality given to the patients,
but this may also mean that the extra time involved in
dealing with the computer means less ‘quality’ time
spent with the patient over the same duration by the
less computer proficient GPs.
Similar studies
A similar cross-sectional study in the USA examined
the association of electronic health record (EHR) use
with 17 ambulatory care quality indicators, with similar
results.28 For 14 of the 17 indicators, there was no
difference in performance between visits with or without
the use of an EHR. On two indicators, the clinicians
using EHRs performed ‘better’ and on one indicator
they performed ‘worse’. Other US studies examining
the relationship between EHR use and quality of care
also found no assocation.29,30
Strengths and limitations
This study employs a method for collection of
nationally representative data, which has proved to
be a valid and reliable approach to providing an
accurate picture of the behaviour of Australian GPs.31
The large number of observations allows good statistical power for most outcomes. We have reported no
difference between the GP groups for some of the
variables measured but acknowledge that where differences are very small, there may have been too few
Effect of computerisation on Australian general practice: does it improve the quality of care?
45
Table 2(b) Summary of results for quality of care indicators – computerised vs noncomputerised test ordering
Domain of care
Measure of quality hypothesis: when compared
to GPs not using computers to order tests, GPs
ordering tests via computer will:
Descriptive
result
Adjusted
result
Tests and
investigations
1
Order fewer investigations in total
XX
X
2
Order fewer pathology tests
XX
X
3
Order fewer imaging tests
X
X
4
Order fewer other investigations
XX
X
Appropriate
preventive care
5
Perform relatively more Papanicolau tests
for 15–70 yr old women
3
X
Inappropriate
preventive care
6
Order relatively fewer PSA tests for males
aged > 50 yrs old
X
X
Diabetes management
7
Order relatively more HbA1c tests for
patients with diabetes
3
3
Musculoskeletal
disease
8
Order relatively less imaging for low back
pain and sprains/strains (any site)
X
X
Note: 3 – hypothesis accepted; X – hypothesis rejected, there being no significant differences between the groups; XX – hypothesis
rejected, result reversed from that hypothesised
cases to be able to make a reliable acceptance of a null
hypothesis. For example, the rate of other investigations (see Table 1(a)) compared 1201 cases in the
clinical computer user group to 169 cases in the group
of their counterparts. These investigations occurred in
each group at the comparatively low rate of only 11 in
every 1000, and nine in every 1000 patient encounters
respectively.
Computer use was self-reported in this survey, and
some GPs may have inaccurately reported their level
of usage, through recall bias or perceived desirable
responding. However, the questions about computer
use were incorporated within a larger set of questions
about a variety of GP characteristics and we have no
reason to assume that their responses were inaccurate
to a degree that may have compromised this research.
As an entity, quality is difficult to measure. The use
of quality indicators is an inexact science at best, and
the incorrect application of inappropriate quality
indicators cannot produce a valid or reliable result.32
However, the set of indicators used in this study were
designed originally in consultation with the Royal
Australian College of General Practitioners (RACGP)
National Manager, Quality Care and Research and the
RACGP National Standing Committee, Research,
drawing from Australian and international guidelines
for preventive activities. These included the RACGP
‘Red Book’, the Canadian guide to clinical preventive
health care and guidelines for National Health Priority
areas such as the National Heart Foundation cardiovascular disease guidelines.21 The quality indicators
were validated in previous work done for the RACGP21
and are suitable for use with the BEACH data source
used in this study.
Future implications
One of the difficulties in clearly assessing the relationship between the inclusion of computers in the clinical
process and quality of care is that GPs are not using the
computer to its full potential. In many instances
Australian GPs do use the computer to print prescriptions and order tests or referral letters, but do not use
the electronic health record function available through
their software, for a variety of reasons. Many are still
heavily reliant on paper records.3,4 The situation
appears similar in the USA.33
We were able to examine GP practice behaviour
where computers had been included in the clinical
process, but within the computer use group there was
wide variation in usage levels. It may simply be that
computer use has so far made little difference to the
quality of care because the computer is not used by
many individuals to its full capacity. The cross-sectional
data available via the BEACH method, while applicable to the process measures utilised in this study,
46
J Henderson, G Miller, H Britt et al
cannot provide individual patient outcomes. Complete, longitudinal data would be needed to allow the
application of indicators that could provide outcome
measures – ironically, information that might become
available once GPs use their computers exclusively
and comprehensively. At such a time, this type of
investigation could be repeated, but given the improbability of finding a comparison group of non-clinical
computer users, other methods will need to be devised.
ACKNOWLEDGEMENTS
The GPs and patients who contributed to this study
participated voluntarily without remuneration and
the authors wish to thank them for their generosity.
REFERENCES
1 Garrido T, Jamieson L, Zhou Y, Wiesenthal A and Liang
L. Effect of electronic health records in ambulatory care:
retrospective, serial, cross sectional study. BMJ 2005;
330:581.
2 United States General Accounting Office. Medical ADP
Systems: automated medical records hold promise to
improve patient care. 1991. GAO/IMTEC-91–5, 1–30.
Washington, US GAO. 30–6–2007.
3 Henderson J, Britt H and Miller G. Extent and utilisation
of computerisation in Australian general practice. Medical Journal of Australia 2006;185:84–7.
4 McInnes DK, Saltman DC and Kidd MR. General practitioners’ use of computers for prescribing and electronic health records: results from a national survey.
Medical Journal of Australia 2006;185:88–91.
5 Richards B, Bolten P, Veale B and Quinlan F. Information Technology in General Practice. Canberra: Commonwealth Department of Health and Aged Care,1999.
6 Heathfield H, Pitty D and Hanka R. Evaluating information technology in health care: barriers and challenges. BMJ 1998;316:1959–61.
7 Mitchell E and Sullivan F. A descriptive feast but an
evaluative famine: systematic review of published articles
on primary care computing during 1980–1997. BMJ
2001;322:279–82.
8 Institute for Safe Medication Practices. A Call to Action:
eliminate handwritten prescriptions within three years!
www.ismp.org/msaarticles/WhitepaperPrint.htm
(accessed 4 April 2005).
9 Wyatt J and Walton R. Computer based prescribing.
BMJ 1995;311:1181–2.
10 Teich JM, Merchia PR, Schmiz JL, Kuperman GJ, Spurr
CD and Bates DW. Effects of computerized physician
order entry on prescribing practices. Archives of Internal
Medicine 2000;60:2741–7.
11 National Prescribing Service Limited. Antibiotic Prescribing in General Practice. //nps.org.au/Doc/pdfs/
PPR21_Antibiotics.pdf
12 Newby DA, Fryer JL and Henry DA. Effect of
computerised prescribing on use of antibiotics. Medical
Journal of Australia 2003;178:210–13.
13 Koppel R, Metlay JP, Cohen A et al. Role of computerized physician order entry systems in facilitating
medication errors. Journal of the American Medical
Association 2005;293:1197–203.
14 Han YY, Carcillo JA, Venkataraman ST et al. Unexpected increased mortality after implementation of a commercially sold computerized physician order entry
system. Pediatrics 2005;116:1506–12.
15 Pont LG, Denig P, van der Molen T, van der Veen WJ
and Haaijer-Ruskamp FM. Validity of performance
indicators for assessing prescribing quality: the case of
asthma. European Journal of Clinical Pharmacology 2004;
59:833–40.
16 Schoen C, Osborn R, Huynh PT, Doty M, Peugh J and
Zapert K. On the front lines of care: primary care
doctors’ office systems, experiences, and views in seven
countries. Health Affairs (Millwood) 2006;25:w555–
w571.
17 Blendon RJ, Schoen C, Donelan K et al. Physicians’ views
on quality of care: a five-country comparison. Health
Affairs (Millwood) 2001;20:233–43.
18 Davies HT and Lampel J. Trust in performance indicators? Quality in Health Care 1998;7:159–62.
19 Britt H, Miller G, Charles J et al. General Practice Activity
in Australia 2006–2007. General Practice Series No. 21.
Cat. no. GEP 21. Canberra: Australian Institute of Health
and Welfare, 2008.
20 Britt H, Miller GC, Charles J et al. General Practice
Activity in Australia 2005–2006. General Practice Series
No. 19. Cat. no. GEP 19. Canberra: Australian Institute
of Health and Welfare, 2007.
21 Miller G, Britt H, Pan Y and Knox S. Relationship
between general practitioner certification and characteristics of care. Medical Care 2004;42:770–8.
22 Commonwealth Department of Health and Ageing.
Medicare Benefits Schedule Book. Canberra: DoHA, 2003.
23 Classification Committee of the World Organization of
Family Doctors (WONCA). ICPC-2: International
Classification of Primary Care (2e). Oxford: Oxford
University Press,1998.
24 SAS Proprietary Software Release 8.2. Cary: SAS Institute
Inc., 2001.
25 Stata Statistical Software: release 8.0. College Station, TX:
Stata Corporation, 2003.
26 Bennett JW and Glasziou PP. Computerised reminders
and feedback in medication management: a systematic
review of randomised controlled trials. Medical Journal
of Australia 2003;178:217–22.
27 Cooper-Patrick L, Powe NR, Jenckes MW, Gonzales JJ,
Levine DM and Ford DE. Identification of patient
attitudes and preferences regarding treatment of depression. Journal of General Internal Medicine 1997;
12:431–8.
28 Linder JA, Ma J, Bates DW, Middleton B and Stafford
RS. Electronic health record use and the quality of
ambulatory care in the United States. Archives of Internal
Medicine 2007;167:1400–5.
29 Zhou L, Soran CS, Jenter CA et al. The relationship
between electronic health record use and quality of care
over time. Journal of the American Medical Informatics
Association 2009;16:457–64.
Effect of computerisation on Australian general practice: does it improve the quality of care?
30 Keyhani S, Hebert PL, Ross JS, Federman A, Zhu CW
and Siu AL. Electronic health record components and
the quality of care. Medical Care 2008;46:1267–72.
31 Britt H, Miller GC, Charles J et al. General Practice
Activity in Australia 2007–2008. General Practice Series
No. 22. Cat. no. GEP 22. Canberra: Australian Institute
of Health and Welfare, 2008.
32 Campbell SM, Braspenning J, Hutchinson A and
Marshall MN. Research methods used in developing
and applying quality indicators in primary care. BMJ
2003;326:816–19.
33 Simon SR, Soran CS, Kaushal R et al. Physicians’ use of
key functions in electronic health records from 2005 to
2007: a statewide survey. Journal of the American Medical
Informatics Association 2009;16:465–70.
FUNDING
The BEACH program is funded by a consortium of
Australian government agencies, government funded
quality of care agencies and pharmaceutical manufacturers. During the data collection period for this substudy, the BEACH program was funded by the
Australian Government Department of Health and
Ageing, the National Prescribing Service, Abbott
Australasia, AstraZeneca (Australia) Pty Ltd, JanssenCilag Pty Ltd, Merck Sharp and Dohme (Aust) Pty
Ltd, Pfizer Australia Pty Ltd, Sanofi-Aventis Australia
Pty Ltd, the Office of the Australian Safety and
Compensation Council (Australian Government Department of Employment and Workplace Relations)
and the Australian Government Department of Veterans’ Affairs.
ETHICAL APPROVAL
The BEACH program was approved by the Human
Research Ethics Committee of the University of Sydney
47
(Reference No. 7185) and the Ethics Committee of the
Australian Institute of Health and Welfare.
PEER REVIEW
Not commissioned; externally peer reviewed.
CONFLICTS OF INTEREST
Organisations fund BEACH through individual research agreements with the University of Sydney
which provides complete research autonomy for the
team conducting the BEACH program. Funders provide input into the research design development
through an Advisory Board, however, the final decisions regarding research design, data collection and
analysis and reporting of findings remain with the
principal investigators under the ethical supervision
of the University of Sydney and the Australian Institute of Health and Welfare (AIHW). The funding
organisations supporting BEACH have no editorial
control over any aspect of the resulting papers including the presentation of results. No financial support
for production of this paper is accepted from the
funding organisations. None of the authors of this
paper have a financial conflict of interest.
ADDRESS FOR CORRESPONDENCE
Dr Joan Henderson, Family Medicine Research
Centre, University of Sydney, Acacia House, Westmead
Hospital, PO Box 533, Wentworthville, NSW 2145,
Australia.Tel: +61 2 9354 0618; fax: +61 2 9845 8155;
email: [email protected]
Received 1 September 2009
Accepted 6 December 2009