Download ToRExtEvAfrStorybookJuly20162

Terms of Reference
External evaluation of the first four years of the
implementation of Saide’s African Storybook initiative
Saide is seeking the services of an evaluator experienced in accountability evaluations to
conduct a final evaluation of the first four years of its African Storybook initiative, funded by
Comic Relief.
Background
The African Storybook initiative aims to address the shortage of contextually appropriate
books for early reading in the languages of Africa. On the African Storybook website,
www.africanstorybook.org , users can find, create, translate or adapt stories for early
reading. They can download and copy the stories and/or illustrations without having to ask
for permission or pay a fee. The stories can be read online or offline or printed from the
website. To encourage and learn from use, the project is working with teachers, librarians,
teacher educators and literacy development organisations mainly in South Africa and East
Africa. For more information, please see http://www.saide.org.za/african-storybookinitiative
Evaluation purpose, audience and questions
The purpose of this evaluation is to provide an external review of what ASb has achieved
against the outcomes and indicators agreed with the funder on inception, as well as of the
Theory of Change.
The agreed outcomes follow.
•
Outcome 1: Literacy development organisations (LDOs) and educators working in African
countries use the website successfully to find, create, interact with and translate /adapt
stories for use in their contexts. Through the website, sufficient stories are available for
regular reading practice.
•
Outcome 2: Literacy development organisations and educators working in African
countries use the stories in a variety of ways (pedagogic and technical) for early literacy
in their contexts
•
Outcome 3: Growing recognition that openly licensed stories available for versioning for
particular contexts and languages have a significant role to play in supporting early
literacy development, particularly for very young African children
•
Outcome 4: Saide’s change management and project management processes become
more rigorous.
In reviewing the outcomes, the evaluators need also to comment on

The relevance of the initiative: Is the initiative meeting a need for affordable access for
educators and the children in their care to enjoyable storybooks in the languages of
Africa for early reading? Is it meeting a need for the support and development of local
initiative in the creation of early reading materials that are culturally and linguistically
appropriate, and in contexts in which access to technology and education have been
severely limited in the past? To what extent is the initiative being recognised and taken
up internationally as well as in the pilot countries?

The design of the initiative: Are the activities designed to meet these needs in
technologically and pedagogically innovative and theoretically sound ways? Are the
changes that have been made/are being made to the project design appropriate? What
improvements could be made?

The efficiency and effectiveness of the initiative: How well is it being run? Is it able to
adapt to the evolving needs and conditions, both local and global?

The sustainability of the initiative: Is the initiative paying sufficient attention to the
sustainability of its results?
The audience for the external evaluation is Comic Relief (the current funder of ASb), other
potential funders, the Saide Board of Trustees and management, as well as the African
Storybook team.
Scope
The following components are envisaged:

Review of key ASb documents.

Engagement with external mid term formative evaluation.

Engagement with a hub site in each of East Africa and South Africa/Lesotho.

Engagement with a government agency working with ASb stories and website in
East and South Africa.

Engagement with one teacher college/faculty working with African Storybook.

Interviews with Saide international advisors.

Interviews with members of the Saide/ASb team.
Governance/accountability
The external reviewer will be contracted by Saide, and the consultancy fee paid from the
Comic Relief grant.
The external evaluation will be managed by Saide’s ASb Project Leader.
Professional qualifications and experience
The successful applicant will
• Be an evaluation practitioner with a relevant Masters degree
• Have a background in education (preferably also in literacy), as well as
• Have experience in accountability evaluations in Sub-Saharan African
contexts.
2
Experience in evaluating digital innovations in education would be an advantage.
Deliverables and schedule
Draft evaluation report – 30 November 2016
Presentation to Saide’s ASb team – December 2016
Final report – 31 January 2017
Applications submitted in response to these ToR
Budget amount
Maximum for all components R250 000, excluding VAT and travel costs which Saide will pay
directly.
Deadline for submission of applications
8h00, Central African Time, 2 August 2016
Format
By email PDF document containing the following:
•
Elaboration of scope with approach and methodology
•
Plan for evaluation
•
Elaboration of proposed budget
•
CV of applicant
•
Contact details of three referees
3
Annexure 1: ASb outcomes and indicators
Outcome 1: Literacy development organisations (LDOs) and educators working in African
countries use the website successfully to find, create, interact with and translate /adapt
stories for use in their contexts. [Through the website, sufficient stories are available for
regular reading practice.]1
Indicators in Learnings Plan

Number of openly licensed unique stories uploaded to the website

Number of openly licensed unique illustrations uploaded to the website

Number of stories translated into African languages on the website using ASb
templates

Number of stories versioned on the website using ASb templates

Number of stories downloaded from the website

Number of website visitors

Educators use the website successfully to create, translate and adapt stories for
early literacy

Partners (working in schools, colleges of education, and non-governmental
organisations) use the website to find, create, interact with and translate/adapt
stories independently for use in their own contexts
From the Annual Data Form
Educators use the website successfully to find, create, and translate/adapt stories.
1

Number of pilot site educators

Number of partners in pilot countries

Number of other educators/users
This sentence appears in the Learnings Plan, but not in the list of outcomes in the Annual report.
4
Outcome 2: Literacy development organisations and educators working in African
countries use the stories in a variety of ways (pedagogic and technical) for early literacy
in their contexts
Indicators in Learnings Plan

Number of story development workshops held2

Stories generated through ASb used in pilot sites and partner literacy
development organisations are used in diverse pedagogical and technical ways.

Pilot sites initiating new reading activities and opportunities using stories
generated through ASb in a variety of African contexts
From annual data form
Story are used for different pedagogical/early literacy purposes by educators in partner
literacy organisations

Use by pilot sites

Independent (spontaneous)3 use (other partners, as well as general
website users)
Children’s reading practices in the pilot sites are enriched by the use of the website and its
stories at their site: -
2

Children reached in pilot sites

Children reached through partner literacy organisations.4
This indicator should really have been in Outcome 1.
3
The phrasing in the 2015 annual report was ‘spontaneous uptake’. However, we would like to modify this to independent
uptake. Most people who use our website do so as a result of the web of relationships that we have built up. So ‘spontaneous’
does not capture this correctly.
4
In early 2014, we decided not to pursue children’s research (children as direct beneficiaries), but to focus on educators.
5
Outcome 3: Growing recognition that openly licensed stories available for versioning for
particular contexts and languages have a significant role to play in supporting early
literacy development, particularly for very young African children
Indicators in Learnings Plan

Extent of provided stories requiring major revision before placing in ASb
templates5

Support partners providing feedback on the website6

Literacy activists aware of potential for Open Educational Resources to increase
access to stories for early literacy in Africa

Partners actively contributing to implementation of ASb in a variety of ways on a
mutual benefit basis

Saide engages with government department/province/district to explore cost
effective ways for the provision of openly licensed local language early literacy
materials for early literacy in the schooling system

Expanding network of advocacy partners participating in project implementation

The usefulness of the website as well as the stories as a resource is recognised by
teacher educators, as well as in large scale reading programmes.
From the annual data form
Partners listed on the site as partners/supports, authors/academics, and people who
attend the annual conferences recognise the role
[on website, on partner grid, conference attendees, pilot site advocacy, mailing list]
Reporting for 2015 was under the following headings:

Extent of alignment with the international literacy development community

Government support for systemic implementation

Willingness to donate illustrated stories under an open licence

Take up by teacher educators

Take up by other innovators
5
The logic behind this indicator is that if donated stories require major revision or major work before being uploaded onto the
website, it might not be worth spending time sourcing them.
6
This was included as an indicator for outcome 3, but it seems to fit better with outcome 1 – the nature of the feedback will
indicate whether or not people had difficulty accessing and using the site.
6
Outcome 4: Saide’s change management and project management processes become
more rigorous.
Three sub-outcomes:
4a: Saide has a better understanding of changes required by engagement with this
project.
4b: Saide’s ability to manage a growing number of collaborative relationships improves.
4c: Saide’s project management processes become more rigorous.
7