How we measure our impact

An incredibly tricky aspect of development work is how to determine its impact. How do you measure girls’ empowerment? How do you know whether students’ improving performance in school is due to your programs and not because of an unknown third factor? Although it’s difficult for non-profits to say with hundred percent certainty that a positive change was the result of their work, there are some ways through which they can indicate that it is very, very likely that their programs were responsible for the progress. This happens with the use of a monitoring and evaluation framework. What does this look like for GHEI?

To understand that, let’s delve into one of GHEI’s biggest programs: the Mother Mentor Program. This is a relatively young program, as it was piloted in March 2016 - a little over two years ago. The program pairs first-time pregnant women with trusted mothers in the community who serve as “Mother Mentors”, working with mothers and families from pregnancy through the first two years of the child’s life to maximize the child’s health and development.

First, the program’s outcomes have to be clearly established. The pursued outcomes of the Mother Mentor program can be summarized as follows:

1. Improve nutrition for pregnant women and children.

2. Reduce infections in pregnant women and children.

3. Reduce stress in pregnant women and mothers.

4. Promote development of children.

Mother Mentor Alice visiting one of the mothers in the program.

Monitoring

To establish whether the Mother Mentor Program achieves its outcomes, several outcome indicators are developed. The term is quite self-explanatory: what indicates that the outcomes were accomplished? In the case of our Mother Mentor Program, what indicates that a child is healthy? What indicates that the mother knows what constitutes a nutritious diet during pregnancy? What indicates that she knows how to feed her baby and when it is necessary to take her infant to the hospital?

A long, long list of outcome indicators has been set up for the Mother Mentor program. During their (bi)weekly visits, the Mother Mentors not only discuss best practices with the mothers, they also assess how the mother and the baby have been doing since the last time they met. The mentors collect data such as the number of times the baby had diarrhea, the number of times the baby ate solid food, the number of times that an adult engaged in playtime with the baby, the number of times a mother exclusively breastfed her baby - and the list goes on.

These numbers are all compiled into a database, where they are compared to our so-called baseline numbers. Our baseline numbers reflect the same outcome indicators, only then among those mothers and babies in the community that are not involved in GHEI’s programs. The baseline group is a much larger group. The baseline outcome indicators tell us what the situation among mothers and babies is like if there is no intervention. In other words, how often do those children who do not receive support from GHEI through the Mother Mentor program have diarrhea? And how often do children who are part of the program have it? Is the difference big enough to assume the program had an effect?

The GHEI health team analyzing data from the Mother Mentor program.

Evaluation

If there is a big difference and it can be explained by the logic of the program, then GHEI could state that, yes, our Mother Mentor program has a positive impact on the well-being of mothers and children enrolled in the program. For instance, our outcome indicators from the first quarter of 2018 show that 2,8% of the children in our program experienced diarrhea in the two weeks before the data was collected. Among other members of the community, this percentage was 29%. It is quite safe to say that the Mother Mentor program played an important role in reducing diarrhea among children.

Improving our programs

But if the percentages are not as expected, then GHEI would have to take another look at the program to see why the intended result was not achieved. For instance, in 2017, one of the intended results of the program was that 80% of the mothers would take iron pills on a daily basis. However, our monitoring efforts revealed that the actual percentage (60%) was way lower. Consequently, GHEI’s health team evaluated why this was the case. Did the mentors not provide sufficient information about the importance of iron? Or was there something else going on?

It turned out that in the local pharmacies and health facilities, iron pills were very low in stock and it was difficult for the mothers to get their hands on them. Based on this information, GHEI adjusted the program: GHEI now purchases the iron pills and gives these to the mothers. This is how we continually aim to improve our programs.

It’s not yet possible to tell what the long term impact of the program will be. The goal is that, through a trickle-down effect, the mothers in our Mother Mentor program will share the knowledge they acquired through the program with their family members and their children. However, this will require a whole different set of outcome indicators and it will take quite some patience before these long term effects could be assessed. For now, we are happy that our intended results on a short-term level are achieved.