Download Policy Board 2002 – report notes

Evidence-based e-learning: what really works? – a report from the ALT Policy Board
2002
ALT Policy Board 2002 was held at Aston University, Birmingham, on 3 July 2002. Attending,
and contributing to the presentations and discussions, were 60 representatives of ALT’s
institutional and corporate members. These included representatives of universities and
further education institutions, as well as from commercial organisations and other bodies in
the further and higher education sectors.
ALT Policy Board 2002 brought together representatives of further and higher education
institutions and corporate members to discuss evidence concerning the need for and
effectiveness of e-learning, and the implications for strategic developments within FE and HE
institutions.
The aim was to consider evidence that would underpin the use of both digitised learning
resources and electronic communication environments, and explore the implications for the
future. Many of the presentations gave strong evidence for the growing effectiveness of
e-learning. However, at the time of the meeting there were few large-scale studies or
implementations of development and delivery of e-learning. However, amongst the plethora of
small-scale, often case-based evaluative studies of e-learning, common themes pointed to
similar conclusions.
The evidence from one type of learner and/or learning situation may not be applicable to
others and a lot of debate concerned specific transferability issues. One of the other major
items that arose in discussions was under what circumstances the principles and evidence
from one type of learner and/or learning situation necessarily apply to others. A ‘one size – or
model – fits all’ is unlikely to be appropriate. We also still need more work defining the kind of
evidence that can be accepted.
The research agenda is beginning to move from case studies to larger studies, or
meta-analyses of case studies and this will have important long term consequences as it
becomes more focussed and relevant. This would be taken forward by ALT as a theme for a
new committee ALT-RP.
There are currently many methods by which institutions seek to collect operational or
research-based evidence for the effectiveness of their e-learning development and
implementations, and work was needed to identify which are the most appropriate
methodologies for evidence collection in different circumstances and the most appropriate
models in which to frame the evidence. There is a current lack of understanding of the
complexity of some of the issues but clear thinking was beginning to emerge. Differences in
institution, mode, technology, learner type, and level all served to add complexity to data.
The extent to which governmental policy is evidence-based was debated with the majority
view being that it was more driven by short term policy considerations in some areas. On the
other hand, if one waits for conclusive evidence but at the same time denies that most
evidence has any value, this becomes a recipe for permanent inaction. Thus, whilst it was not
necessary to have a “smoking gun”, firmer evidence bases would diminish the probability of
waste of public funds allowing more money to be spent on research which could then be
decried as part of the process.
The costs of developing e-learning given varied very widely, but the differences often reflected
significant differences between the types of learner, and the extent of e-learning delivery
within their total educational experience. The value of central production of courses and
materials that can be widely used was contrasted with the often local, relatively small-scale,
development of e-learning materials in some institutions.
Diana Laurillard’s presentation (The Open University) provided evidence from the OU’s
rigorous systems of iterative testing and evaluation of learning materials. Her presentation,
sensitively and ably delivered by a colleague, emphasised the need to build on good practice,
with iterative testing and sharing of best practice. The Open University’s experience is also
that the use of many forms of ICT within education work best when it is both compulsory and
assessed. DianaLaurillard.ppt – 1.5MB
Anna Rosetti, describing the experience of the Land-Based Colleges Consortium (LBCC), and
Joe Wilson, from the Scottish Further Education Unit, showed how the involvement of
teaching staff and the use of their expertise in all stages of development of materials is key to
appropriateness, quality and usability of materials. Another lesson the LBCC has learnt is that
demonstrable benefits to the learner (rather than use of technology ‘because we can’) must be
at the forefront of plans for the development of materials. LBCNC.ppt – 90kB JoeWilson.ppt –
320kB
The description of many developments of e-learning in the recent past as ‘postmodern praxis’
(Chris O’Hagan, University of Derby) shows that, in a time when there has been a lack of
evidence on the best approaches, this has often been the only method for deciding on the
approach to be used. ChrisOHagan.ppt – 50kB
Robin Mason (OU) spoke as acting Director of Research (part-time) at the eUniversity. She
addressed the agenda for the UkeU research centre and the extent to which it could effectively
meld pure research with data gleaned from the myriad of students planned by UKeU.
RobinMason.ppt – 250kB
Seb Schmoller from Sheffield College described developments and critical success factors in
FE deployment. The course for being an online tutor developed by his team on behalf of a
consortium of colleges is widely accepted and deployed. SebSchmoller.ppt – 60kB
Bob Banks and David Kay from FD learning described and reflected on their varied
experiences across the sector. The contrasts between the costs of large operations such as
the UfI and small college operations was emphasised. FDL.ppt – 300kB
Many other issues for research emerged and these will be remitted to ALT-RP to prioritise and
take forward, in collaboration with others. These include:
 The need for quality assessment of, and research on, learning objects and their
effectiveness
 The need for research into the possible danger of atomisation of content through the use
of learning objects
 Whether an institution should rely on a single VLE or MLE.

Whether and how online collaboration can be most effectively deployed to enhance the
learning experience in a flexible and scalable fashion

The extent to which students appreciate supported, highly interactive learning, with
community and communication being more important than slick presentation.

The desirability of a mix of different media that were complementary

The dangers of over-assessment and inappropriate assessment
Several issues emerged in discussion of what was needed next. These included:





The need to move beyond case studies to a more co-ordinated agenda.
The need to put things in an overall framework.
The need for research and development in the pedagogy of e-learning to tie better into
discipline based deployment of VLEs.
The need for pedagogy and technology to interrelate and be “appropriate”
The need for a coordinated, high profile research agenda
Overall a good day for the collection of issues. It remains to be seen whether ALT-RP can
build on these to help provide best practice and solutions.
By John Slater (Chair of ALT’s Research and Policy Executive), with input from Seb Schmoller (ALT Executive Secretary), based
on a document originally written by Peter Murray.
February 2003