The relevance of digital tools for development projects is widely recognized, yet staff capacity often lags behind donor and organizational requirements. Helvetas is in the middle of an initiative to bring projects in our more than 30 partner countries into the digital world through the use of KoboToolbox (“Kobo”) for data collection and PowerBI to create dashboards. The aim is to improve not just monitoring and evaluation (M&E), but learning and accountability as well.
“Going digital” is an opportunity to improve M&E processes, including how and when data is collected and used. In addition to enhancing project management, going digital is also expected to reduce administrative burdens: Currently, country offices fill out an annual spreadsheet detailing progress against performance indicators, but the goal is to replace those spreadsheets with dashboards that update automatically throughout the year as new data is uploaded.
Here are the five key lessons learned from the first year of Helvetas’ efforts to expand the use of digital tools by Helvetas staff around the world.
1. Invest in staff resources
After running a pilot with projects in five countries to test best methods for the dissemination and use of digital tools, Helvetas determined there was a need to ramp up digitalization efforts. To that end, a Digital M&E Coordinator was hired with the mandate to provide training to country offices and assist them with the transition.
Similarly, for nine months select staff from nine countries attended a Digital M&E Academy that was developed and facilitated by the Digital M&E Coordinator, in order to not just improve their own M&E skills and learn digital tools but to become country and regional champions who can further assist with scaling up the roll-out. Country directors and project managers invested their staff’s time — eight hours in class and 10-20 hours for assignments for each module — for the benefit of not just their own countries but Helvetas overall. In the Academy, participants learned and improved specific skills, such as using the digital tools, then applied them to their own projects immediately; they also learned leadership skills relevant for Digital M&E Champions.
Helvetas cannot scale-up digital M&E tools just through staff at the head office. Employees in offices around the world need to lead digital efforts, be sources of expertise and promote the transition both in their countries as well as in their regions or language.
2. Be ambitious
Helvetas created a plan that provides support for countries at whatever level they fall on the digital spectrum, while still setting expectations for every country to go digital.
- Working with senior staff, a goal was set for at least 75% of countries (if not all) to be using KoboToolbox and PowerBI by the end of 2024. Countries can select their own goals. For countries with little or no digital experience, Track 1 sets minimum requirements (using Kobo for two projects, making one PowerBI dashboard). For other countries who have already been using digital tools for years or who want to fast-track progress, Track 2 has more large-scale goals (such as using Kobo for all projects; making dashboards for partners, funders or other stakeholders). We’re currently on track for all countries to be “digital” by the end of 2024.
- The Digital M&E Academy developed regional champions who will now facilitate Regional Academies beginning in September 2023. The Regional Academies will be held in English, French and Spanish — the three official languages of Helvetas.
- Because PowerBI is more challenging to learn, Helvetas is paying for professional PowerBI training for a few staff in every country, as well as technical assistance while they develop their first country-based dashboards. Country directors and other managers thus have the opportunity to learn the tool and develop a plan for creating project and country dashboards.
- Supporting resources, including webinars, blogs and a Digital M&E Manual, all provide additional assistance to staff around the world. Materials are centrally located on Helvetas’ intranet. The newest offering was launched in July 2023: Community of Practice (CoP) drop-in sessions in English, French and Spanish that provide opportunities for staff to discuss their digital M&E issues, receive peer-to-peer support about those challenges, and share how their projects or countries have solved common issues.
- Going digital is more than just implementing Kobo and PowerBI. Rather, it presents an opportunity to improve M&E systems overall. This can include updating forms before putting them in Kobo (e.g., removing outdated questions or answer options; adding in skip logic and data validation rules), identifying ways Kobo can improve data collection (e.g., collecting GIS data, creating surveys that primary stakeholders can fill out via Smartphones), and developing systems to reflect on the data more frequently (e.g., reviewing dashboards regularly, identifying learnings, digging deeper into the data to determine root causes of successes or failures).
Emir Cholukov, Digital M&E Academy graduate, Helvetas Kyrgyzstan
3. Plan for agility as well as reality
Country experiences with digital tools varies widely. A number of countries, including Madagascar and multiple Eastern European countries, are experienced in using digital surveys for projects. The Guatemala, Honduras and Nepal offices have also been building complex dashboards for a number of years. But, on the other end of the spectrum, a few offices have used paper forms until recently. We therefore work with each country to determine their needs.
To help countries progress in their digital M&E journey, the Academy participants developed a roll-out plan for all projects in their countries. They reviewed the plans with their country directors, who made any necessary adjustments before signing off. One Academy graduate noted that she, her colleague and their country director have changed their roll-out plan after learning from experience that going digital takes longer than anticipated. Indeed, it is country-based staff who determine not only the pace of the rollout but the approach as well — Bolivia is working on moving all projects forward step by step together, while many countries select key projects to start with and then expand to additional projects.
Learning has been integral to this project, beginning with the pilot that tested how to roll out digital tools. An adaptive management approach has meant we have been able to shift targets and methods based on actual needs and situations in countries. For example, the initial goal two years ago was for data to flow immediately from projects to country level to organization-wide performance indicators. But the different needs of countries have shown that a slower approach is necessary — hence the more modest goal for 2024 that gives countries time to implement the digital tools at their own pace.
The past year has highlighted the need for more standardization — a significant change in an organization that prides itself on its decentralized management, where country offices run with great independence. In order for data to flow from project to organization level, project-level performance indicators need to become more standardized; in addition, the data collection forms (called surveys in Kobo) need to ask questions using the same terminology so that project-level data can be aggregated at the country and organizational levels, as well as by thematic working field (e.g., WASH, private sector development, civic engagement). Thanks to the extensive digital M&E outreach in the past year, country directors, country staff and head office staff have come to recognize the need for more uniformity in order to reduce administrative burdens and leverage these useful technologies.
3c. Data architecture
We are also still wrestling with how our data architecture should be structured behind the scenes. An inter-unit data architecture team developed a plan to move data from Kobo to a custom-built user interface that would make data cleaning easier, faster and more systematic; the clean data would then be sent to a data lake or data warehouse where it would be available for use in PowerBI dashboards. But we have realized that we first need to standardize performance indicators so the data warehouse is structured appropriately. In addition, we need to embed this project within the larger context of digital transformation at Helvetas.
4. Use the great ideas of others
Ideas from both head office and country staff have furthered the Digital M&E initiative. Mozambique was one of the first two countries to receive PowerBI training. When the team met afterward to plan their dashboards, they tackled the problem from the bottom up. First, they aligned all logframe and Helvetas performance indicators with questions in Kobo surveys, including determining how the indicators will be calculated. They then began building project dashboards with the details, followed by program dashboards with more big-picture summary graphs.
Based on insights from the Mozambique team, the Digital M&E Manual was developed. Further case studies and tips will be added using lessons learned by other countries while they develop their own digital programs.
Separately, upon hearing about a thorny technical issue, a head office staff person suggested Helvetas participate in a social hackathon. The Hack for Social Good, run by technical colleges in Switzerland, provided digital expertise and solutions that will be used for the data architecture.
5. Build communities
The Digital M&E Academy has enabled staff to develop warm relationships with peers in other countries. Some Academy participants even attended webinars that others gave (an Academy assignment to develop their leadership skills), with one person commenting, “We have to support our colleagues.” They will continue to support each other going forward, and the new cohorts of Regional Academy participants will expand the circle to include champions from all countries.
Additional communities are developing. The four Latin American countries are talking now about standardizing indicators — and thus dashboards — in their region. Likewise, the new CoP drop-in sessions are bringing together staff from all countries; multiple people have stepped forward to help organize the sessions.
What began as a modest project – rolling out digital M&E tools in all countries – has provided insights into the challenges of digitizing core systems and processes as well as the diverse needs and factors to be addressed. Helvetas’ culture is changing at the same time that tools – software, platforms, apps – and systems are improving to enable us to better meet the needs of the people we serve in more than 30 low- and middle-income countries.
About the Author
Rebecca Laes-Kushner is the Digital M&E Coordinator at Helvetas, helping countries along their path to “going digital” for M&E. She has worked on databases and outcome/performance measurement for social service programs in the United States and Switzerland for almost 20 years.