Summary

Background / Terms of Reference

Key Questions

• What is the impact on student achievement of introducing ICTs in an educational setting in developing countries?
• How does this compare with other types of interventions?
• How should this impact be measured, and what are the related issues, especially as they relate to Education For All and other Millennium Development Goals?

Background

Despite over ten years of investment in ICTs to benefit teaching and learning in many developing countries, little is known about their substantive impact on learning processes and outcomes:  

• The impact of ICT use on learning outcomes is unclear, and open to much debate.
• There is an absence of widely accepted standard methodologies and indicators to assess impact of ICTs in education.
• There is a disconnect between the rationales most often put forward to advance the use of ICTs in education (to introduce new teaching and learning practices and to foster 21st century thinking and learning skills) and their actual implementation (predominantly for use in computer literacy and dissemination of learning materials).

There is very little reliable and comparable hard and soft data about the impact (positive, negative and inconclusive) of ICT use in education in developing countries and, while a few notable international comparative studies have been done, all such studies cite (and lament) the lack of rigorous attention to this area.  The vast majority of studies that claim positive impact on educational outcomes from ICT in education interventions suffer from a basic methodological problem – the absence of appropriate control groups – and, because they often rely to a great extent on self-reporting (among other reasons), they may be subject to important biases.  While best practices exist in educational assessment in general, there is a great need for a handbook outlining best practices, guidelines and issues related to impact evaluation of ICT use in education, especially in the context of Education For All (EFA) and the education-related MDGs, and specifically relevant to the particular resource constraints and challenges faced by countries at most risk of not meeting MDG targets by 2015. 

There is a particular need for guidance explicitly relevant and useful to policymakers and donor staff in countries eligible to participate in the Fast Track Initiative, many if not most of which (a) suffer from a lack of qualified and quality personnel able to conduct rigorous and independent impact assessments; (b) are ill-equipped to evaluate claims made by vendors of ICT-related products and services targeted at the education sector; (c) have little funding for such activities; and (d) lack supporting institutions and information channels to help enable the work of such evaluation activities.  Issues related to cost and cost effectiveness of ICT in education initiatives, as well as of educational assessments of this nature in general, are particularly acute in these countries.  There is also a pressing need for additional work related to performance indicators to monitor the use and effects of ICTs.