Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Terms of Reference External evaluation of the first four years of the implementation of Saide’s African Storybook initiative Saide is seeking the services of an evaluator experienced in accountability evaluations to conduct a final evaluation of the first four years of its African Storybook initiative, funded by Comic Relief. Background The African Storybook initiative aims to address the shortage of contextually appropriate books for early reading in the languages of Africa. On the African Storybook website, www.africanstorybook.org , users can find, create, translate or adapt stories for early reading. They can download and copy the stories and/or illustrations without having to ask for permission or pay a fee. The stories can be read online or offline or printed from the website. To encourage and learn from use, the project is working with teachers, librarians, teacher educators and literacy development organisations mainly in South Africa and East Africa. For more information, please see http://www.saide.org.za/african-storybookinitiative Evaluation purpose, audience and questions The purpose of this evaluation is to provide an external review of what ASb has achieved against the outcomes and indicators agreed with the funder on inception, as well as of the Theory of Change. The agreed outcomes follow. • Outcome 1: Literacy development organisations (LDOs) and educators working in African countries use the website successfully to find, create, interact with and translate /adapt stories for use in their contexts. Through the website, sufficient stories are available for regular reading practice. • Outcome 2: Literacy development organisations and educators working in African countries use the stories in a variety of ways (pedagogic and technical) for early literacy in their contexts • Outcome 3: Growing recognition that openly licensed stories available for versioning for particular contexts and languages have a significant role to play in supporting early literacy development, particularly for very young African children • Outcome 4: Saide’s change management and project management processes become more rigorous. In reviewing the outcomes, the evaluators need also to comment on The relevance of the initiative: Is the initiative meeting a need for affordable access for educators and the children in their care to enjoyable storybooks in the languages of Africa for early reading? Is it meeting a need for the support and development of local initiative in the creation of early reading materials that are culturally and linguistically appropriate, and in contexts in which access to technology and education have been severely limited in the past? To what extent is the initiative being recognised and taken up internationally as well as in the pilot countries? The design of the initiative: Are the activities designed to meet these needs in technologically and pedagogically innovative and theoretically sound ways? Are the changes that have been made/are being made to the project design appropriate? What improvements could be made? The efficiency and effectiveness of the initiative: How well is it being run? Is it able to adapt to the evolving needs and conditions, both local and global? The sustainability of the initiative: Is the initiative paying sufficient attention to the sustainability of its results? The audience for the external evaluation is Comic Relief (the current funder of ASb), other potential funders, the Saide Board of Trustees and management, as well as the African Storybook team. Scope The following components are envisaged: Review of key ASb documents. Engagement with external mid term formative evaluation. Engagement with a hub site in each of East Africa and South Africa/Lesotho. Engagement with a government agency working with ASb stories and website in East and South Africa. Engagement with one teacher college/faculty working with African Storybook. Interviews with Saide international advisors. Interviews with members of the Saide/ASb team. Governance/accountability The external reviewer will be contracted by Saide, and the consultancy fee paid from the Comic Relief grant. The external evaluation will be managed by Saide’s ASb Project Leader. Professional qualifications and experience The successful applicant will • Be an evaluation practitioner with a relevant Masters degree • Have a background in education (preferably also in literacy), as well as • Have experience in accountability evaluations in Sub-Saharan African contexts. 2 Experience in evaluating digital innovations in education would be an advantage. Deliverables and schedule Draft evaluation report – 30 November 2016 Presentation to Saide’s ASb team – December 2016 Final report – 31 January 2017 Applications submitted in response to these ToR Budget amount Maximum for all components R250 000, excluding VAT and travel costs which Saide will pay directly. Deadline for submission of applications 8h00, Central African Time, 2 August 2016 Format By email PDF document containing the following: • Elaboration of scope with approach and methodology • Plan for evaluation • Elaboration of proposed budget • CV of applicant • Contact details of three referees 3 Annexure 1: ASb outcomes and indicators Outcome 1: Literacy development organisations (LDOs) and educators working in African countries use the website successfully to find, create, interact with and translate /adapt stories for use in their contexts. [Through the website, sufficient stories are available for regular reading practice.]1 Indicators in Learnings Plan Number of openly licensed unique stories uploaded to the website Number of openly licensed unique illustrations uploaded to the website Number of stories translated into African languages on the website using ASb templates Number of stories versioned on the website using ASb templates Number of stories downloaded from the website Number of website visitors Educators use the website successfully to create, translate and adapt stories for early literacy Partners (working in schools, colleges of education, and non-governmental organisations) use the website to find, create, interact with and translate/adapt stories independently for use in their own contexts From the Annual Data Form Educators use the website successfully to find, create, and translate/adapt stories. 1 Number of pilot site educators Number of partners in pilot countries Number of other educators/users This sentence appears in the Learnings Plan, but not in the list of outcomes in the Annual report. 4 Outcome 2: Literacy development organisations and educators working in African countries use the stories in a variety of ways (pedagogic and technical) for early literacy in their contexts Indicators in Learnings Plan Number of story development workshops held2 Stories generated through ASb used in pilot sites and partner literacy development organisations are used in diverse pedagogical and technical ways. Pilot sites initiating new reading activities and opportunities using stories generated through ASb in a variety of African contexts From annual data form Story are used for different pedagogical/early literacy purposes by educators in partner literacy organisations Use by pilot sites Independent (spontaneous)3 use (other partners, as well as general website users) Children’s reading practices in the pilot sites are enriched by the use of the website and its stories at their site: - 2 Children reached in pilot sites Children reached through partner literacy organisations.4 This indicator should really have been in Outcome 1. 3 The phrasing in the 2015 annual report was ‘spontaneous uptake’. However, we would like to modify this to independent uptake. Most people who use our website do so as a result of the web of relationships that we have built up. So ‘spontaneous’ does not capture this correctly. 4 In early 2014, we decided not to pursue children’s research (children as direct beneficiaries), but to focus on educators. 5 Outcome 3: Growing recognition that openly licensed stories available for versioning for particular contexts and languages have a significant role to play in supporting early literacy development, particularly for very young African children Indicators in Learnings Plan Extent of provided stories requiring major revision before placing in ASb templates5 Support partners providing feedback on the website6 Literacy activists aware of potential for Open Educational Resources to increase access to stories for early literacy in Africa Partners actively contributing to implementation of ASb in a variety of ways on a mutual benefit basis Saide engages with government department/province/district to explore cost effective ways for the provision of openly licensed local language early literacy materials for early literacy in the schooling system Expanding network of advocacy partners participating in project implementation The usefulness of the website as well as the stories as a resource is recognised by teacher educators, as well as in large scale reading programmes. From the annual data form Partners listed on the site as partners/supports, authors/academics, and people who attend the annual conferences recognise the role [on website, on partner grid, conference attendees, pilot site advocacy, mailing list] Reporting for 2015 was under the following headings: Extent of alignment with the international literacy development community Government support for systemic implementation Willingness to donate illustrated stories under an open licence Take up by teacher educators Take up by other innovators 5 The logic behind this indicator is that if donated stories require major revision or major work before being uploaded onto the website, it might not be worth spending time sourcing them. 6 This was included as an indicator for outcome 3, but it seems to fit better with outcome 1 – the nature of the feedback will indicate whether or not people had difficulty accessing and using the site. 6 Outcome 4: Saide’s change management and project management processes become more rigorous. Three sub-outcomes: 4a: Saide has a better understanding of changes required by engagement with this project. 4b: Saide’s ability to manage a growing number of collaborative relationships improves. 4c: Saide’s project management processes become more rigorous. 7