Download The Value and Impact of Research Data Infrastructure

The Value and Impact of Research
Data Infrastructure
Neil Beagrie
(Charles Beagrie Ltd)
PASIG, Karlsruhe, September 2014
Value + Impact Analysis of
Three UK Data Preservation
Services
John Houghton (Victoria University) + Charles Beagrie Ltd
Methods applied to:
Economic & Social Data Service
Archaeology Data Service
British Atmospheric Data Centre
studies funded by ESRC, Jisc, Jisc/NERC respectively
Presentation Overview
• Introduction - Measuring Value and Impact of Research
Infrastructure
• UK Value and Impact Studies
– ADS results
• Concluding Remarks
• UK Focus but the research and lessons are international
and for digital preservation more generally
UK Research and Innovation
• Why Research matters
– £3.5 billion a year currently spent on publicly
funded (RCs) research generates an additional
annual output of £45 billion in UK companies
(Haskel and Willis 2010)
– Wider non-economic value to policy, heritage, and
knowledge
• Where disciplinary data centres and services exist
they represent approx 1.4-1.5% of total research
expenditure (OSI PC WG)
• Why does preservation of digital data matter for
research?
Astronomy
• Data archives
– “Central to astronomy today”
– HST, 2MASS, and SDSS archival research is major contributor to scientific
productivity
c/o R. White (STScI) and pp. 5-11, 5-12 of NWNHAA
Robert Hanisch
5 May 2011
Research Infrastructures
and Impact
Data
Intellectual Capital
Training & Skills
Human Capital
Buildings / Equipment
Grid / ICT Networks
Staff
Technical / Organisational
Environment
Organisational Capital
Professional
Networks
Relationship Capital
Previous Work
Big Science and Innovation Study for BIS July 2013
• Desk review of c. 100 studies internationally;
• 3 studies highlighted to BIS as being particularly
good examples of ‘good practice’ in the
measurement of economic impacts:
– Berkeley Lab 2010
– Human Genome Project 2011
– Economic and Social Data Service 2012 (authors)
[ Two further studies of BADC and ADS published
later – we think/hope even better!]
Best Practice from ESDS study
Investment
& Use Value
(Direct)
Contingent
Value
(Stated)
Efficiency
Impact
(Estimated)
Return on
Investment
in the Data
(Estimated)
User
Community
Wider
Impacts
(Not Directly
Measured)
Society
User
Community
User
Community
Investment
Value
Amount spent on
producing the
good or service
Use Value
Amount spent by
users to obtain the
good or service
•
•
•
•
•
•
Willingness to Pay
Maximum amount
user would be willing
to pay
Consumer Surplus
Total willingness to
pay minus the cost
of obtaining
Net Economic
Value
Consumer surplus
minus the cost of
supplying
Willingness
to Accept
Minimum amount
user would be
willing to accept
to forego
good or service
User Community
Estimated value of
efficiency gains due
to using the
good or service
Range of Time Savings
(from time spent with
data from the centre
to overall work time)
Increased
Return on
Investment
in the Data
Estimated increase
in return on
investment
in data creation
arising from the
additional use
facilitated by the
data centre
?
Applies range of methods;
Includes counter-factual;
Data collection tailored to different stakeholders: depositors, users, research,
teaching;
Data weighting - survey value responses weighted to reflect the overall pattern of use
from weblogs;
Case studies/ KRDS benefits illustrate benefits and impact pathways;
Research- partnership with Service: adaptation to community and knowledge transfer.
Approaches and Methods
•
We combined quantitative and qualitative approaches, to
quantify the value and impacts of the data services and
explore other, non-economic impacts.
•
The combination is important in capturing and presenting
the full range and dimensions of value.
•
All three studies combined:
• Desk-based analysis of existing literature and reports, looking at
both methods and findings;
• Existing management and internal data collected by the data
services; and
• Original data collection in the form of online surveys of users and
depositors, together with semi-structured interviews.
The Value and Impact of the
Archaeology Data Service
Economic
Key Findings - Economic
Series of different economic approaches used:
• The investment value of ADS is around £1.2 million per
annum;
• ADS impact on research, teaching and studying efficiency:
worth at least £13 million per annum;
• Contingent Valuation - What ADS data and services are worth
to its users:
– Willingness to pay around £1.1 million per annum.
– Willing to accept around £7.4 million per annum.
• RoI scenarios -additional use over 30 years - a 2-fold to 8-fold
return on investment.
Investment
& Use Value
(Direct)
Contingent
Value
(Stated)
Efficiency
Impact
(Estimates)
ADS Regular
User Community
ADS Website
User Community
ADS Website
User Community
Investment
Value
£1.2m
per annum
Willingness
to Pay
£1.1m
per annum
Willingness
to Accept
£7.4m
per annum
User Community
Efficiency Gain
[ADS data use]
£13m
per annum
Consumer Surplus?
(Could be up to £6m per annum)
(on a Willingness-to-Accept basis)
(More than double)
(ADS operational budget)
Net Economic Value?
(Could be up to £5m per annum)
(on a Willingness-to-Accept basis)
Wider ADS
Community
Wider
Impacts
(Not Directly
Measured)
Society
Increased
Return on
Investment
(Severely constrained)
(by capacity to pay)
Use Value
£1.4m
per annum
Return on
Investment
(Scenarios)
User Community
Efficiency Gain
[All activity time]
£58m
per annum
[Additional Use]
£2.4m - £9.7m
(2.1 to 8.3-fold RoI)
Increased
Return on
Investment
[Non Recreate]
£1.5m - £5.9m
(1.3 to 5.1-fold RoI)
-BUT-
Additional re-creation
costs of up to £1m
Based on Deposit &
Download Counts
(2 to 6-fold RoI)
Based on Download Count
Based on User Count
Based on Data Spend
?
Stakeholder Perceptions
Key Findings – Perceptions
• Qualitative analysis - interviews and survey comments reveal
strong support for the ADS - many aware of the value of the
services;
Workshop feedback on quantitative + qualitative interim
findings:
• Stakeholders aware of value to them – less aware of value to
others;
• Stakeholders were positive about seeing value expressed in
economic terms - not previously considered or seen
presented - but they also felt it was important not to dwell
exclusively on economic measures of value;
• The study shows the benefits of integrating a range of
approaches to measuring the value (and for its
dissemination).
Perceptions
AHRC Impact Study
Impact Studies - conclusions
• Economic benefits exceed the operational costs
• A mix of qualitative and quantitative methods is
important to capture and present the value of the Data
Services
• The studies are increasing recognition of the value of
the Data Services and digital preservation and data
sharing generally
• Helping stakeholders see value to them and to others
• Need to extend work into other disciplines and types of
research infrastructure
Further Information
Synthesis of the three studies:
http://repository.jisc.ac.uk/5568/1/iDF308__Digital_Infrastructure_Directions_Report%2C_Jan1
4_v1-04.pdf