Impact Evaluation of Shorebird Investments

For over a decade, there have been calls for a greater focus on the rigorous evaluation of investments for environmental conservation. Many have advocated using impact evaluations and counterfactual frameworks. Impact evaluation goes beyond program monitoring; instead, it attempts to measure the causal effect of an intervention using a credible counterfactual scenario and seeks to understand the conditions under which any effect arises. The environmental sector has been slower to widely adopt impact evaluation approaches compared with others.

ACS and colleagues recently published an impact evaluation focused on conservation investments targeting long-distance migratory species. We evaluated the impact of a philanthropic program investing in the conservation of shorebirds and their wintering habitat along the Pacific Americas Flyway. The flyway spans >16,000 km of coastline between Alaska and Chile. Millions of shorebirds use it biannually, moving between breeding and nonbreeding grounds and back again.

Using a quasi-experimental, mixed methods approach, we estimated what would have happened to shorebird populations at 17 wintering sites without the sustained and additional investment they received. We modeled shorebird populations across the entire flyway and at sites with and without investment. Combining shorebird abundance estimates with a land-cover classification model, we used the synthetic control method to create counterfactuals for shorebird trends at the treatment sites. We found no evidence of an overall effect across three outcome variables. Species- and site-level treatment effects were heterogeneous, with a few cases showing evidence of a positive effect, including a site with a high level of overall investment. Results suggest six shorebirds declined across the entire flyway, including at many Latin American sites. However, the percentage of flyway populations present at the sites remained stable, and the percentage at the treatment sites was higher (i.e., investment sites) than at control sites.

Multiple mechanisms behind our results are possible, including that investments have yet to mitigate impacts and negative impacts at other sites are driving declines at the treatment sites. A limitation of our evaluation is the sole focus on shorebird abundance and the lack of data that prohibits the inclusion of other outcome variables. Monitoring infrastructure is now in place to design a more robust and a priori shorebird evaluation framework across the entire flyway. With this framework, it will prove easier to prioritize limited dollars to result in the most positive conservation outcomes.

Rate of change of relative abundance (2010–2019) for six shorebird species at treatment (received foundation funding) (n = 17) and donor pool sites (no foundation funding; n = 59, sites that are potentially selected to create counterfactuals; black lines, regression with a general additive model).