Download Lee Harvey - Quality Research International

Seventh Quality in Higher Education
Seminar
Transforming Quality
30–31 October 2002
Melbourne
Transforming quality
Deliberate ambiguity:
can quality transform or do we
need to transform quality?
Themes
• Is quality in higher education about
transforming students?
• How might (external) quality
monitoring be transformed to help
improve the quality of the student
experience and of the learning?
• What has been the transformative
impact of external quality
monitoring?
Theme 1
Is quality in higher education
about transforming students?
Quality learning
• What constitutes a high quality
learning process and outcomes?
• To what extent is there a need to
reconceptualise how higher
education engages with the key
issues of
—access
—employability
—funding?
External evaluation
Object
Focus
Rationale
Approach
provider
governanace
& regulation
accountability
accreditation
medium of
delivery
curriculum
design, admin
control
audit
output
learning
experience
compliance
assessment
learner
qualification
improvement
standards
monitoring
national
regional
international
External evaluation
Object
Focus
Rationale
Approach
provider
governanace
& regulation
accountability
accreditation
medium of
delivery
curriculum
design, admin
control
audit
output
learning
experience
compliance
assessment
learner
qualification
improvement
standards
monitoring
national
regional
international
Interlinked elements
funding
learning
quality
access
employability
Interlinked elements
funding
value for money
transformation
learning
quality
fitness for purpose
access
excellence
employability
Interlinked elements
funding
value for money
institutions
transformation
academic staff
learning
quality
students
employers
fitness for purpose
access
excellence
employability
Theme 2
How might (external) quality
monitoring be transformed to
help improve the quality of
the student experience and of
the learning?
External evaluation
Object
Focus
Rationale
Approach
provider
and what
is
provided
static
elements:
regulation,
curriculum,
outputs
accountability
and
conformance
accrediting,
assessing
and checking
learner
learning
experience
improvement
improvement
audit
External evaluation
Object
Focus
Rationale
Approach
provider
and what
is
provided
static
elements:
regulation,
curriculum,
outputs
accountability
and
conformance
accrediting,
assessing
and checking
learner
learning
experience
improvement
improvement
audit
External evaluation
Object
Focus
Rationale
Approach
provider
and what
is
provided
static
elements:
regulation,
curriculum,
outputs
accountability
and
conformance
accrediting,
assessing
and checking
learner
learning
experience
improvement
improvement
audit
External evaluation
Object
Focus
Rationale
Approach
transformation of the
learner
learner
learning
experience
improvement
improvement
audit
Transformative learning
• A continuous process of assimilation,
reflection, synthesis and critique.
• Questioning absolutes,
preconceptions and taken-for
granteds —others and ones own.
• Deconstructing knowledge and
building alternative understandings.
Transformative learning
Accepting
Rote
Engaging/
Questioning
Understanding
Transformative
learning
Reconceptualising
Transformative learning
Enhancing students’ abilities and
knowledge
Empowering students to be active
learners
Enhancing learners
Enhancing students as transformative
learners means:
• providing students with access to a
body of knowledge;
• enabling students to develop a range
of intellectual and other attributes
through which they can engage and
develop knowledge.
Attributes
intellect
knowledge
analysis, synthesis, critique
willingness to
continue learning
ability to find
things out
communication
team working
interpersonal skills
self skills
risk taking
flexibility and adaptability
Empowering learners
Empowering students as
transformative learners means:
• treating students as intellectual
performers rather than as passive
recipients of teaching;
• encouraging critical engagement
with a body of knowledge.
Quality monitoring?
To what extent can external quality
processes assure transformative
learning.
Approaches to date are not strong on
learning at all —tend to conservatism.
Quality monitoring?
External processes are not the primary
mechanism by which transformative
quality improvement in higher
education is assured.
Day-to-day quality assurance is through
internal academic processes.
External processes should articulate
with, and augment, internal procedures.
Theme 3
What has been the
transformative impact of
external quality monitoring?
Impact
• What impact has EQM had and on
what?
• Does it go beyond the level of
rhetoric?
• Does it lead to short-term response
or does it lead to permanent
cultural changes?
• If so, does this permeate all levels
of the institution or is it a
management preoccupation?
External quality monitoring
leads to bureaucratisation and inflexibility
is amateurish, burdensome and inefficient
is concerned with accountability not
improvement
leads to ‘game playing’ and ‘performance’
short-term response not cultural changes
has no real impact on student learning
Bureaucracy
• Any form of EQM would involve
some level of ‘bureaucracy’.
• Key issue is not the existence of a
bureaucracy or of bureaucratic
processes but the nature of the
bureaucracy and its processes.
• Bureaucracy must meets needs of
external and internal stakeholders,
not be self-perpetuating.
Quality bureaucracies
Three main roles:
• ensure integrity of HE
• act as a catalyst for improvement
• act as a conduit for useful
information
Amateurism
• Dominant approach —selfassessment, peer review,
statistical data — not necessarily
seen as the best approach.
• Burdensome.
• Most benefit to the peer assessors
not the assessed.
Efficiency
• Doubts about the efficiency of most
EQM.
• Cost (of agency and and to the institution)
outweighs the value gained.
• Periodic ‘events’ do not help inform
change management.
• EQM inhibits innovation through its
conservative or rigid evaluation
criteria.
Improvement
• Temporary impact.
• EQM must interact with internal
quality systems — often not the case.
• Changes in culture
—slow
—commitment.
• Event or continuous process:
performance and game playing.
Performance & game playing
• Engagement mediated by the
perceived, short-term affect.
• ‘Game playing’ and compliance.
• ‘Performance’ to ensure maximum
return.
• Obscures the reality.
• No surprise: ‘natural’ outcome of
accountability-oriented processes.
• Game playing taking up resources
for very little real return.
Self-assessment
• Main value of EQM is the internal
self-reflection.
• But ‘two sets of books’.
• Fear of revealing weaknesses.
Longevity of process
• Improvement potential decreases
as process becomes more elaborate
and routine?
• Emphasis shifts to procedural
elements rather than innovative
process.
• Need for constant reflection on and
change in EQM, more trust and
collaboration.
• Periodic change in purposes and in
the agencies themselves.
Longevity
“
Without periodic change, there is
the danger of ending up with a
British-style, QAA-type, system: a
rolling ‘juggernaut’, that is not sure
what it is looking for, but which
ensures compliance and minimizes
innovation and risk-taking. British
institutions continue to comply,
even if the return on the investment
is derisory, because of the fear of
loss of income.
”
Impact on learning
• Extreme sceptical that EQM had any
impact on programme quality or
student learning.
• No evidence of clear impact on
learning
—available research suggests that
other factors outweigh the impact of
EQM.
• Structure and organisation of EQM
is not compatible with empowering
staff and students to enhance the
learning situation.
Impact on learning?
‘
I still haven’t seen a study that
directly links external evaluation to
improved student learning. (NOR)
’
Impact on learning?
‘
We still know almost nothing about
the outcome of this concern for
quality in terms of improvements in
student learning. (SWE)
’
Impact on learning?
‘
I can detect no improvement in the
learning situation of students —
perhaps, in fact, even the opposite.
Because we spend so much time
trying to lay ‘paper trails’ for audit,
and trying to ensure good RAE
ratings, time to devote to students
(certainly for informal interaction
with them) is at a premium. (UK)
’
A Quality Manifesto
“
Academics and students of
the world unite and reclaim
the quality agenda…..
”
Thank you