Search

Report/Evaluation Type:Content Type:
Displaying 1 - 10 of 2991

Addressing the ‘Infodemic’ to cope with the COVID-19 (Coronavirus) pandemic

Web Resource
Coronavirus infodemia concept illustration. Sad Woman standing with mobile phone full of news and warnings about economy crisis and COVID 19 outbreak
In addition to tackling the COVID-19 pandemic, governments and professionals responsible for monitoring the progress of policies and evaluating their impact face a phenomenon that has particularly intensified in this crisis: that of an infodemic — a flood of information, some accurate and some not. As governments and average citizens face difficult decisions in the trade-off between protecting Show MoreIn addition to tackling the COVID-19 pandemic, governments and professionals responsible for monitoring the progress of policies and evaluating their impact face a phenomenon that has particularly intensified in this crisis: that of an infodemic — a flood of information, some accurate and some not. As governments and average citizens face difficult decisions in the trade-off between protecting public health while preserving the economy and livelihoods, this flood of information makes it hard to find trustworthy sources and reliable guidance when they are needed the most. To help address this challenge, Clear Lusophone Africa and Brazil (LAB) officially launched the Monitor de Evidências Covid-19 (COVID-19 Evidence Monitor) during gLocal Evaluation Week 2020 that wrapped up earlier this month. Our aim is to provide our partners in government and the evaluation community, along with the various segments of society in the countries where we work, with a source of reliable information to support evidence-based decision making. Misinformation can lead to a great deal of harm, and many organizations, such as the UN, are now officially fighting it. On the brighter side, however, knowledge-production and information-sharing has skyrocketed. Technology giants have signed the Open COVID Pledge, making their intellectual property available free-of-charge under open license. Major scientific publications such as Elsevier and Springer have given free access to research related to the new coronavirus. Many relevant magazines and newspaper globally have also taken down their paywalls for their pandemic coverage - a decision that has often led to new subscribers. The  main role of the Monitor de Evidências Covid-19 is to help our audience navigate the infodemic associated with the new coronavirus crisis by curating  qualified, relevant, and evidence-based content from around the world about various topics, such as the economy, labor market, social policy, education, health, and early childhood, among others. The next step is to organize this data-gathering from multilateral bodies, governments, think tanks, and academic institutions into an online repository and make it available to our audience — public leaders, policy makers, academics, students, and so on, in Lusophone Africa and Brazil — through our social media channels and a brand new weekly newsletter, which is now available for subscriptions. For the next few months, the Monitor de Evidências Covid-19 will also add regular reviews of the evidence-based, curated content to the online repository. All communications products provided by the Monitor will be delivered in Portuguese, making it easier for our community to access quality knowledge and use it in their everyday work in the policy arena.  By focusing our efforts on filtering trustworthy sources and information, we expect to provide reliable guidance to professionals and the public at large seeking evidence to make decisions in these uncertain times of the COVID-19 pandemic and contribute to make them less susceptible to the toxic aspects of the infodemic phenomenon. This initiative is also part of the  commitments and efforts of the CLEAR global network to promote the planning and implementation of evidence-based public policies. It is an assertive way of strengthening systems and capacities for monitoring and evaluation, which will be a key element to produce the most effective and efficient responses to the new challenges posed by the current crisis.

Independent Evaluation Group and École nationale d'administration publique formalize collaboration towards building a new global partnership

Web Resource
Independent Evaluation Group and École nationale d'administration publique formalize collaboration towards building a new global partnership
The World Bank's Independent Evaluation Group (IEG) and the École nationale d'administration publique (ENAP) have signed a Memorandum of Understanding (MOU) to pool resources and expertise to meet the global demand for stronger monitoring and evaluation capacity.  Working together, they will be in a much stronger position to meet this demand in key regions, including Francophone Africa, Show MoreThe World Bank's Independent Evaluation Group (IEG) and the École nationale d'administration publique (ENAP) have signed a Memorandum of Understanding (MOU) to pool resources and expertise to meet the global demand for stronger monitoring and evaluation capacity.  Working together, they will be in a much stronger position to meet this demand in key regions, including Francophone Africa, North America and the Caribbean, and the Middle East and North African regions.   The agreement comes at a time when there is a great need for evaluation capacity building around the world, which no single organization can meet on its own. Formalizing the collaboration between IEG and ENAP will enable the two institutions to work together to share knowledge and lessons learned, coordinated their efforts and expertise, and capitalize on their respective networks to advance evaluation capacity development in key areas of the world. The collaboration will help expand the delivery of ENAP's Programme International de formation en évaluation du développement (PIFED) to geographic and linguistic spaces that remain currently under-served. Watch IEG Director-General Alison Evans and ENAP Director-General Guy LaForest introduce this new collaboration for global evaluation capacity development. {"preview_thumbnail":"/sites/default/files/Data/styles/video_embed_wysiwyg_preview/public/video_thumbnails/oi7W5IK2YkQ.jpg?itok=GtdFWLJh","video_url":"https://youtu.be/oi7W5IK2YkQ","settings":{"responsive":0,"width":"854","height":"480","autoplay":0},"settings_summary":["Embedded Video (854x480)."]}  "This agreement will allow ENAP and IEG to go much farther in building and developing evaluation capacity in developing countries," said Guy Laforest, the Executive Director of ENAP. “I am pleased that our school's expertise in evaluation is now available on a wider scale, especially in this year of the 10th anniversary of the PIFED.” “IEG looks forward to strengthening its partnership with ENAP in order to generate synergies and thus extend the impact of the monitoring and evaluation support programs of our two institutions,” said Alison Evans, the Director General of IEG. “This partnership will enable us to support more governments and institutions in strengthening their systems and capacity to support data-based decision-making and results-based information, and thus accelerate their progress towards the sustainable development goals (SDGs).” As a step towards signing the MoU, ENAP hosted an event during the 2020 gLOCAL Evaluation Week, which took place from June 1 to 5, and was organized by IEG and the CLEAR Initiative. ENAP held a joint panel with the Francophone Evaluation Network of Canada and International Program for Development Evaluation Training (IPDET) on the theme "The Online Transition of Pandemic Assessment Capacity Building Practices: Challenges, Opportunities and Limits" in which hundreds of people participated. Note: This is a translated version of the original news story in French. Sign up to receive updates about the growing global partnership to close the gap in monitoring and evaluation capacity worldwide.  

The World Bank and global collaboration: Lessons for the COVID-19 (coronavirus) response

Web Resource
The World Bank and global collaboration: Lessons for the COVID-19 (coronavirus) response
The coronavirus pandemic (COVID-19) calls for an international response. But when and why does collective action work best? We studied the World Bank Group’s global collaborations to find out. The global health ecosystem is complex, with many actors, initiatives and competing priorities. The World Bank has been navigating this landscape and supporting a wide spectrum of collaborations for Show MoreThe coronavirus pandemic (COVID-19) calls for an international response. But when and why does collective action work best? We studied the World Bank Group’s global collaborations to find out. The global health ecosystem is complex, with many actors, initiatives and competing priorities. The World Bank has been navigating this landscape and supporting a wide spectrum of collaborations for decades. In the early 2000s, the Bank helped create the Global Fund and GAVI, the Vaccine Alliance. Since then, the Bank has found itself contributing to important global public health issues, including responding to many epidemics that are now household names – SARS, MERS, Ebola, Avian Flu, and others. Unlike these past epidemics, the COVID-19 pandemic has rapidly morphed into a global health and economic crisis. Addressing the multiple impacts of the pandemic will require collective action on a greater scale and bringing together, or convening, multiple actors to draw on their respective comparative advantages. Lessons from Evaluating “the World’s bank” The Independent Evaluation Group recently finished a major evaluation of the World Bank Group’s global convening. Though the report was wrapped up just before the COVID-19 outbreak, some of its key findings and recommendations are relevant to the global response: We found high demand for the World Bank Group’s global convening. The high demand is because partners see the need to come together to develop joint solutions to pressing challenges and trust the Bank Group to do a good job. Trust is always important in a crisis. Now, more than ever, people look for experts and organizations that they can trust to lead. Several factors drive effectiveness. Our evaluation found that the Bank Group’s convening is more likely to be effective when global partners share a common understanding and sense of urgency that collective action is needed; internal capacities are strong; and initiatives have clear objectives, links to country programs, and sustained engagement. These conditions are all present for the World Bank Group’s COVID-19 response. There is value in focus and continuity. A clear sense of the specific goals of the Bank Group’s convening and the scope of its engagements are essential foundations for effective global work.  If an organization, any organization, tries to do everything, it does nothing well. If debt relief for the poorest countries is the key goal, stick to that goal for some time. Don’t introduce too many competing goals, and don’t abandon the goal before it is within sight. Set goals and track progress. The Bank Group often does not give itself enough credit for the results of its global work. When there often are no clear goals for the global work, and no tracking of progress, reporting the results becomes impossible. It would help the World Bank Group to better track, assess, and report the results of its global work. Manage tensions between – and within – organizations. There is some degree of tension and competition over roles and mandates in the global community. Tension may arise with other organizations as both they and the World Bank Group seek, or are perceived to seek, pieces of the COVID-19 agenda and the organizational prestige that comes from being at the forefront of the crisis response. Also, inside the World Bank, units may compete to stake out their piece of the action. Tensions are not necessarily a bad thing, but they do need to be managed. An understanding of when collective action works best and why, with the same focus on results applied to global and other work, will help lay the foundations for even more effective joint efforts to address a host of global challenges. A recent IEG evaluation traces the successes of international collaboration in tackling global challenges.   Laying the groundwork for global teamwork Digging down into the initiatives that are at the forefront to help developing countries cope with COVID-19, reveals many of the footprints of the World Bank Group’s global work. These build on years of concerted effort setting up partnerships, cooperation platforms, data exchanges, and financial mechanisms that allow countries and organizations to join forces on shared problems. Recent examples are the Global Preparedness Monitoring Board, which was co-convened by the Bank and WHO to ensure preparedness for global health crises, and the Coalition for Epidemic Preparedness Innovations (CEPI), a global mechanism to finance and co-ordinate vaccine development for which the World Bank is a trustee. While these were launched in 2018 and 2017 respectively, they are in fact the fruits of the World Bank’s focused and sustained global engagements in health.  The G20’s push for bilateral debt relief for poor countries, to free up resources to cope with the impacts of the coronavirus, is another example of current global convening. This can be traced back to the World Bank Group’s and the International Monetary Fund’s convening of G20 member countries over many years, working on debt relief, crisis responses and diverse initiatives in many sectors. The global effort to address the COVID-19 pandemic will face many more challenges, and drawing on lessons from past experiences of convening can help individual actors navigate the complex terrain of collective action. The World Bank plays a large convening role in global health issues. Learn more in Appendix F of The World’s Bank: An Evaluation of the World Bank Group’s Global Convening. Image Credit: adapted from shutterstock/ GoodStudio and shutterstock/ Marish

IEG Work Program and Budget (FY21) and Indicative Plan (FY22-23)

PDF file
IEG will build on the strategic framework it adopted in FY 20, centering its work program around 6 work streams on i) Gender, ii) Fragility, Conflict, and Violence (FCV), iii) Climate Change and Environmental Sustainability, iv) Mobilizing Finance for Development (MFD), v) Human Capital, and vi) Jobs, Growth, and Shared Prosperity, and 2 cross cutting themes on i) Governance and Institutions, and Show MoreIEG will build on the strategic framework it adopted in FY 20, centering its work program around 6 work streams on i) Gender, ii) Fragility, Conflict, and Violence (FCV), iii) Climate Change and Environmental Sustainability, iv) Mobilizing Finance for Development (MFD), v) Human Capital, and vi) Jobs, Growth, and Shared Prosperity, and 2 cross cutting themes on i) Governance and Institutions, and ii) the WBG’s Corporate Effectiveness. IEG will also maintain an increased and balanced focus on country level outcomes. To contribute meaningfully to the WBG response to the COVID-19 crisis, in the near term, IEG will update its pipeline evaluations to contextualize findings and lessons where relevant. IEG will also respond to WBG management requests for just in time notes that synthesize evidence and lessons from past evaluations to inform the crisis response, and IEG will provide on-demand M&E advice to WBG operational teams working on crisis related programs and play an active role in sharing relevant evaluative insights and lessons drawn from past crises. In the short to medium term, IEG will also conduct early stage evaluations of the WBG’s response to the crisis, intended to offer evidence useful to enhancing implementation effectiveness. In the medium to longer term, IEG will undertake ex post evaluations of the impact of the Bank Group’s response and lessons to inform future crisis response.

Addressing administrative data gaps in India to fight COVID-19 (coronavirus) and speed recovery

Web Resource
Mumbai, Maharashtra, India,2020. Installing Aarogya Setu app on mobile phone under home quarantine. Launched by government of India for tracking prevention & testing of Coronavirus (Covid-19) pandemic
The authors work with CLEAR South Asia, hosted by Abdul Latif Jameel Poverty Action Lab (J-PAL), South Asia As the world moves to ease restrictions imposed for COVID-19, it is increasingly clear that we will need to learn to live with the virus for some time and make significant behavioral changes. Governments have a difficult role to play in balancing public health concerns with economic needs Show MoreThe authors work with CLEAR South Asia, hosted by Abdul Latif Jameel Poverty Action Lab (J-PAL), South Asia As the world moves to ease restrictions imposed for COVID-19, it is increasingly clear that we will need to learn to live with the virus for some time and make significant behavioral changes. Governments have a difficult role to play in balancing public health concerns with economic needs and are grappling with questions related to how and what to open. Apart from these immediate challenges there is a need to hasten the economic recovery and build greater resilience to fight future pandemics. India has made significant progress in the use of data to guide its strategies for coping with the impacts of the pandemic, a tool that will prove as useful for navigating the reopening. In view of its critical importance, CLEAR South Asia has been working with state governments in India to build capacity to collect and evaluate data better. The COVID-19 pandemic, more than ever, has demonstrated the value of data in addressing public health crises. Researchers in the US analyzed anonymized cell phone location data to understand which enterprises can be ‘super spreaders’. India Observatory has developed a GIS-enabled dashboard to show the movement of migrants in real time and to identify relief centers on their routes. Moreover, governments are using data in different forms for contract tracing and isolation. Good quality data, particularly administrative data, can become a vital tool for governments to plan reopening and rebuilding the economy.              Enlarge and download this infographic Administrative data, or admin data, which is data collected during routine transactions, can be a rich and inexpensive source of information to generate useful research, especially when data from different sources is combined (e.g. mobile signal data and incidence of disease) for easy use. However, in developing countries like India, there are significant challenges with regard to accessibility and usability of admin data, especially when government is the data provider, which limits its use. Access to admin data for research use in India is largely driven by individual champions rather than a comprehensive legal framework or protocols to govern access, storage, transfer and use in a transparent manner. In addition, there are constraints due to capacity and knowledge in making data available to researchers in a secure manner. For instance, ensuring a secure means of data transfer, determining sensitivity level of data fields, meta-data documentation, ability to anonymize data at source are a challenge for data providers, especially governments. Another barrier, particularly for government generated admin data, is its usability in terms of appropriate formats, standardization of collection process and quality. Digitized data in PDF formats or at an aggregated level are not very useful for research purposes. Often, codes are not standardized even for basic geographical units (such as districts, villages) across datasets in India which makes combining datasets difficult. Another challenge is ascertaining data quality in regard to its reliability and accuracy. Given the growing importance of data, the Indian government is increasingly aware of these gaps. A data protection bill is under consideration by the Parliament. The government released data protocols to address privacy concerns under its contract tracing application Arogya Setu. Moreover, NITI Aayog, a premier think tank of the Government of India, has recently launched its vision for the National Data and Analytics Platform (NADP) to address some issues around data usability. While most of these efforts are concentrated at the national level, there is also scope and demand for capacity building at the state level (federal units in India), where implementation takes place. CLEAR South Asia has been working actively with state governments in India to address issues related to access and usability of government data (both primary and admin). We have conducted customized trainings and hands-on workshops for government staff on conducting independent data audits and quality checks. We have also provided advisory support to our government partners on their data policy, for transitioning to digital data collection and improving the data collection/recording process. In one such partnership, the CLEAR/ J-PAL South Asia team provided advisory inputs into the design of a new data collection system that the department was looking to transition to for recording complaints received on their women’s helpline. Our team also helped the government with more standardized formats for recording information in the interim. Going forward, as CLEAR South Asia center, we plan to intensify our engagement with state governments and other data providers to demonstrate the potential of data use and in the process strengthen the access and usability of their admin data systems. Our team will ramp up efforts to provide customized capacity building workshops, and/or advisory services on how to make data more accessible for research, strengthening data collection processes, instituting systematic data quality checks as well as strengthening its use for decision-making to address specific needs and help in course correction. Through these efforts, we hope to support the government’s increasing use of data to inform their decision-making process and as a foundation for rigorous evaluations. We hope that through these sustained efforts of CLEAR SA and our government stakeholders we can build better data systems to help recover and fight the next pandemic.   Pictured above: Mumbai, Maharashtra, India,2020. Installing Aarogya Setu app on mobile phone under home quarantine. Launched by government of India for tracking prevention & testing of Coronavirus (Covid-19) pandemic.  Image credit: Shutterstock/ PhotographerIncognito

The International Finance Corporation’s and Multilateral Investment Guarantee Agency’s Support for Private Investment in Fragile and Conflict-Affected Situations (Approach Paper)

PDF file
In countries affected by fragility, conflict, and violence (FCV), the private sector can play a critical role in providing jobs and income. Inclusive and sustainable economic growth led by private investment can help heal grievances stemming from economic exclusion. Although the private sector in fragile environments and in conflict is often informal, constrained, and distorted and may involve Show MoreIn countries affected by fragility, conflict, and violence (FCV), the private sector can play a critical role in providing jobs and income. Inclusive and sustainable economic growth led by private investment can help heal grievances stemming from economic exclusion. Although the private sector in fragile environments and in conflict is often informal, constrained, and distorted and may involve entities that are parties to conflict, it is essential for providing livelihoods, income, and services to people. This evaluation seeks to inform the implementation of the Bank Group FCV strategy and IFC’s and MIGA’s commitments to scale up investments in FCS. As the Bank Group is launching its 2020–25 FCV strategy, this evaluation will inform its implementation. The report will help gauge the effectiveness of and develop lessons from efforts to enhance the range of IFC and MIGA initiatives to scale up and improve sustainable private investments in FCS under the Capital Increase Package and IFC’s and MIGA’s strategies.

Evaluation Headlines: Adapting Evaluation in the time of COVID-19

Web Resource
Evaluation Headlines: Adapting Evaluation in the time of COVID-19
In this edition of Evaluation HEAD-lines, Independent Evaluation Group Director-General Alison Evans and Independent Evaluation at ADB Director General Marvin Taylor-Dormond discuss how their respective departments are adapting their workplan to support their institutions respond effectively to the COVID-19 crisis.In this edition of Evaluation HEAD-lines, Independent Evaluation Group Director-General Alison Evans and Independent Evaluation at ADB Director General Marvin Taylor-Dormond discuss how their respective departments are adapting their workplan to support their institutions respond effectively to the COVID-19 crisis.

Rewiring Evaluation Approaches at the Intersection of Data Science and Evaluation

Web Resource
rewiring evaluation event
These sessions will feature a combination of evaluators and data science users and practitioners to explore lessons, opportunities and challenges in rewiring evaluation. These sessions will feature a combination of evaluators and data science users and practitioners to explore lessons, opportunities and challenges in rewiring evaluation. 

gLOCAL Evaluation Week 2020: A Global Knowledge Movement to Advance Monitoring and Evaluation Knowledge Sharing

Web Resource
With the ten-year countdown to the Sustainable Development Goals (SDGs) underway, it is more important than ever for countries to have the systems and capacities in place to track the progress of their national development strategies, evaluate their impact, and learn from evidence. The new circumstances posed by the Covid-19 pandemic have highlighted even more the crucial role played by Show MoreWith the ten-year countdown to the Sustainable Development Goals (SDGs) underway, it is more important than ever for countries to have the systems and capacities in place to track the progress of their national development strategies, evaluate their impact, and learn from evidence. The new circumstances posed by the Covid-19 pandemic have highlighted even more the crucial role played by monitoring and evaluation (M&E) in designing and developing adequate policy responses, the importance of learning from evidence, and sharing this knowledge across the board. With the objective of fostering a culture of M&E learning and knowledge-sharing and sustaining the momentum of global efforts to promote M&E capacity, the CLEAR Initiative will convene the second annual gLOCAL Evaluation Week from June 1st to 5th.   gLOCAL Evaluation Week 2020 is fully virtual and will feature more than 250 events around the world organized by local public, private, and academic institutions and organizations that produce, use, or promote evaluations to strengthen the impact of development programs. In addition to the traditional thematic areas such as Evaluation Capacity Development, Evaluation Communication and Use, and Evaluation Methods, the special theme, ‘Evaluation 2030’, intends to encourage discussions about the role of M&E in supporting the achievement of long-term development outcomes such as the 2030 Development Agenda and the Sustainable Development Goals. There will be several events exploring M&E themes in various sectors, such as gender, climate change, health, among others.  We also have several events that explore the implications of and for M&E in times of Covid-19. {"preview_thumbnail":"/sites/default/files/Data/styles/video_embed_wysiwyg_preview/public/video_thumbnails/MquEulFC5XE.jpg?itok=6HyjLW3K","video_url":"https://youtu.be/MquEulFC5XE","settings":{"responsive":0,"width":"854","height":"480","autoplay":0},"settings_summary":["Embedded Video (854x480)."]} Sharing local and global M&E knowledge and experiences are essential to strengthen capacity, foster accountability, promote evidence-based decision making, and learn from results. These goals are central to the CLEAR Initiative’s mission and the inspiration for gLOCAL Evaluation Week. -Sophie Sirtaine, Director of Strategy and Operations, Independent Evaluation Group of the World Bank Group, and CEO of the CLEAR Initiative.  Last year, 177 institutions around the world supported the 2019 inaugural gLOCAL Evaluation Week during which over 270 events were organized in 38 countries. An estimated 22,000 participants convened to learn and share M&E knowledge. This year, along with the CLEAR Centers, our partners and collaborators are rising to the challenges posed by Covid-19 with a great deal of innovation and adaptability by making optimal use of technology to bring knowledge and experience sharing opportunities closer to you than ever before. The virtual nature of gLOCAL 2020 will allow organizers to reach wider audiences and enable M&E professionals and organizations to engage in a larger and continuous dialogue across time-zones. This is aligned with CLEAR’s commitment to support the exchange of M&E knowledge and experiences to promote evaluation capacity development, support evidence-based decision making, and strengthen development outcomes at local and global levels. As we get ready for gLOCAL Evaluation Week 2020 next week, this is how you can be involved in gLOCAL 2020: Promote gLOCAL Evaluation Week through your networks and channels. Share news and social media updates about gLOCAL on your social media and other platforms. More information is available at the CLEAR Initiative. Participate in gLOCAL Evaluation Week and encourage everyone to participate. Go to gLOCAL Evaluation Week to see the 2020 Calendar of Events. Share with us and others your experience of gLOCAL 2020. Write to us and follow us on social media. For more information and to see the calendar of events, please visit gLOCAL Evaluation Week 2020 Calendar of events and the CLEAR Initiative, follow the CLEAR Initiative (on twitter @theCLEARInitiative), or email clear@worldbank.org    

What will it take to learn fast to save lives during COVID-19 (coronavirus)?

Web Resource
Medical professionals assigned to the 531st Hospital Center conduct an after-action review following the mass casualty scenario outside the operation room of the field hospital at Sierra Army Depot, California, on Oct. 28, 2019. (image credit: Spc. ShaTyra Reed/Army) Note the appearance of U.S. Department of Defense (DoD) visual information does not imply or constitute DoD endorsement.
In an emergency room, much as in the response to a global crisis such as the coronavirus pandemic, saving lives requires teams to learn fast, to shift their thinking, and to adapt their practices in real time. It requires the application of simple, nimble practices for evaluating, learning, and reacting—not the creation of detailed planning or checklists—but processes for gathering and sharing Show MoreIn an emergency room, much as in the response to a global crisis such as the coronavirus pandemic, saving lives requires teams to learn fast, to shift their thinking, and to adapt their practices in real time. It requires the application of simple, nimble practices for evaluating, learning, and reacting—not the creation of detailed planning or checklists—but processes for gathering and sharing knowledge about what is working, what is failing, and why. Luckily, there are a suite of nimble tools, linked to a practice called Emergent Learning that can easily be adapted by teams working on the  World Bank’s crisis response. Tools like the After Action Review (AAR), first developed by the United States military to extract lessons and shift tactics between battle, can be applied at pivot points in project operational cycles. The Harvard Business Review Article, Learning in the Thick of it, describes how the army, and many fast-moving, consumer goods companies, use the tool to generate and distill raw data from the front line, and to feed this back into the implementation cycle. An AAR can be conducted anywhere, often, and is most effective when carried out at key decision points throughout an implementation cycle. It is also most effective when all team members are engaged in the process. The tool is used—and is increasingly gaining traction—amongst humanitarian agencies, beginning with the Tsunami response. IEGs 2014 evaluation, How the Bank Learns, called on the World Bank to “to become better at learning from lending”—a need all the more dire in the times of COVID-19.  The evaluation challenged the World Bank to integrate lessons from active experience—from both effective and failed efforts—into ongoing operations, even if that meant changing a course of action. This is the point of emergent learning: it promotes contestation and iteration. It encourages the organization and its operations to make learning a part of all activities, and part of everyday work, and less an isolated chore. With increasingly complex development challenges, this becomes ever more important. There has been a wide call for organizations, including the World Bank, to think differently—to integrate complexity into development practice—and to make learning an integral part of the way that they think and act. At the World Bank, Michael Woolcock makes the case for iterative and adaptive work. In his work with the British government’s Department of International Development (DFID), Ramalingam navigates the “wicked development problem”: the gap between what is trying to be achieved, and the methods being used to achieve it. Flexible, adaptable approaches were happening despite DFID’s corporate processes, not because of them. Oxfam’s Duncan Green provides hope and optimism about how change can happen. As Green puts it, quoting Milton Friedman much before the COVID crisis, "only a crisis produces real change”. Or, we can take a cue from Benedict Carey, in his book, How We Learn. Did you know that taking a test on a subject before you know anything about it improves subsequent learning? The premise seems absurd, but it reminds us that stimulating our awareness of context and refining the parameters of a problem are helpful in identifying and formulating solutions. In addition to AARs, the field of emergent learning offers four essential tools for making the approach to the current pandemic more effective: The ‘Framing Question' is built with the simple phase: “What will it take to”….[deliver healthcare to the most vulnerable COVID-19 patients?]. It creates a focus for collective learning by asking what it will take to achieve a desired outcome. It is a forward-focused, action-oriented challenge to the group to ensure alignment around an agreed premise or idea. When done well, the framing question is at once a mechanism to coalesce around a desired outcome and a means to define individual contributions. A good framing question has the ability to "train a group’s attention forward, in a collective inquiry that leads to action”. Emergent Learning Tables needed to surface and capture data, insights, formulate hypotheses and identify opportunities for further action. A way to “bring the team around the table”—the tool encourages teams to reflect by asking ‘what do we know so far?” Importantly, the process helps the team make a deliberate connection between the past and the future to ensure that previous lessons inform subsequent actions. ‘Before Action Review’ is an opportunity to discuss in detail what success will look like, and to establish intended results and identify anticipated pitfalls. The review sets the team on a learning pathway—linked to iterative, and often held, 'After Action Review' (see below). ‘After Action Review’ enables real time reflection on how the actual results compare with the intended ones. What caused the results and what will sustain or improve them? View a quick animated presentation about these 4 emergent learning tools {"preview_thumbnail":"/sites/default/files/Data/styles/video_embed_wysiwyg_preview/public/video_thumbnails/-20Ooj8KsSU.jpg?itok=lnvlacbg","video_url":"https://youtu.be/-20Ooj8KsSU","settings":{"responsive":0,"width":"850","height":"480","autoplay":0},"settings_summary":["Embedded Video (850x480)."]}   As countries and organizations like the World Bank Group move quickly to respond to the COVID-19 crisis, nimble tools are needed to shine a light on and assess what is working and what is not. Traditional monitoring and evaluation approaches—built on established theories and project baselines—are of limited use in a fast-moving and constantly changing crisis. Project teams should be encouraged and supported to reach for a wider set of emergent learning tools for enhanced contestability, and to create an environment of constant learning and where mistakes are acknowledged—and where both contribute to improving the impact of interventions. The stakes are high: during the times of COVID-19, learning fast can help protect the most vulnerable and ultimately save lives.   pictured above: Medical professionals assigned to the 531st Hospital Center conduct an after-action review following the mass casualty scenario outside the operation room of the field hospital at Sierra Army Depot, California, on Oct. 28, 2019. (image credit: Spc. ShaTyra Reed/Army) Note the appearance of U.S. Department of Defense (DoD) visual information does not imply or constitute DoD endorsement.