Advances in Nature-based Carbon Removal Solutions

The early weeks of 2023 have been awash with news about the challenges in the voluntary carbon market. Most prominent among the reporting on the carbon market was an article published by The Guardian in January, which claimed that 94% of REDD+ carbon credits are worthless. Not surprisingly, The Guardian story kicked up a lot of interest in exactly how firms buying, selling, developing and advising on carbon credits define carbon removal (or offset) quality.

While recent reports have gotten the attention of investors, there is a path to quality carbon dioxide removal (CDR) with forestry credits such as REDD+, improved forestry management (IFM), and afforestation, reforestation, and restoration (ARR). So what should carbon credit buyers be looking for?

FURTHER READING: Guardian reporting details significant risks with avoided deforestation offsets

What is causal inference and why is it important?

One of the most exciting scientific advances in the voluntary carbon market over the last year has been the industry's embrace of causal inference methods to quantify carbon removal. Most forestry projects are credited against a counterfactual baseline (what would happen on the project land if the project was not undertaken), and understanding what makes a “good” baseline has historically been a vexing problem for forestry projects. Causal inference methods can help the voluntary carbon market identify “good” baselines by using statistical procedures that mimic experimental design.

The most common use of these methods in the voluntary carbon market is the use of matching algorithms to create artificial control groups that are similar to observations receiving the treatment. The groups are then compared to calculate the impact of the treatment on the outcome. These methods have become particularly popular for REDD+ projects. In a REDD+ scenario, an area inside of the project area is matched to similar areas outside. The matched areas outside of the project then create the baseline, and carbon fluxes are calculated as differences between the project area and the matched baseline. In the right settings and when done with care, matching methods can create plausible baselines that can increase the quality of carbon credits.

As a scholar, I’ve spent the last decade using these methods, and enthusiastically cheer the industry for adopting them. Along with a global group of collaborators, I have applied these techniques to a host of land use issues including, protected areas, forest protection laws, war, drug legalization, zoning, sacred forests, wildfire and of course climate change policies. In fact, I’ve found these methods so useful for quantifying causal impacts that I co-authored a peer-reviewed manuscript on how to apply these techniques, complete with sample datasets and code.

Over the last year, a number of companies have publicly shared their methodologies for using causal inference to gauge carbon credit quality while others have started to use causal inference methods to develop registry approved methodologies. Despite the adaptation of these methods within the voluntary carbon market, the quality of assessments using these methods varies and results from causal inference methods can be contested. Indeed, both supporters and critics of The Guardian article used causal inference techniques to evaluate projects, even though they came to different conclusions.

How can causal inference methodologies be improved?

While the tools of causal inference are firmly rooted in statistics and mathematics, their successful application in applied settings is part scientific method, part specialized scientific knowledge, and part practical experience. The quality of evidence uncovered by these models depends on a number of assumptions being met, and often these assumptions cannot be empirically tested. This means you need not just sound statistical techniques, but a nuanced understanding of the process that drives land use change and carbon fluxes in the first place.

This is part of the reason why both the evidence for and against The Guardian’s reporting can claim to be using appropriate methods but come to different conclusions. In this case, some results were sensitive to the inclusion or exclusion of just a few variables and it takes system-wide knowledge as well as methodological skill to make the right section—an approach The Guardian did not take. Embracing causal inference is a big step forward, but there is work yet to be done.

As I look at the state of causal inference in the voluntary carbon market, there are three areas where the techniques used today could be improved.

1. Go beyond matching. Unless matching is combined with other statistical methods, it generally fails to control for trends in forest change that are happening before the project is established. If these trends differ between matched project and non-project observations, results based on matching metrics alone can be biased. This may be the case where project forests, which by definition have little past deforestation, are compared to control forests that may have already experienced some deforestation. In this case, matched observations may face different deforestation pressures and comparison may lead to biased results.
   
   Fortunately, this bias can be eliminated by integrating data from time periods before the project start date (Jones and Lewis 2015). Two well known methods that do this are difference-in-difference models (Nolte et al. 2017) and synthetic controls (West et al. 2020). These techniques can be enhanced by using matching as pre-processing (Ho et al 2006) and have proven robust in many settings.

   
   

2. Account for unobservable factors. Matching does not control for unobservable factors that can determine forest change (Greenstone and Gayer 2009). That is, matching is based solely on variables that can be quantified by the researcher. However, we know that there are many factors that determine forest change but for which data simply does not exist. For example, land use history can have a profound influence on current forest health and structure and therefore impact future harvest. Yet in most parts of the world, maps of land use history (for example historical wildfires, long ago forest clearings or settlements) are not available and therefore cannot be included in matching. When an unobserved variable differentially affects project and non-project areas, results based solely on matching can be biased (Jones and Lewis 2015).
   
   Luckily, there are statistical techniques that can help control for the presence of factors that are not quantifiable to the researcher. Common techniques rely on the use of panel data (i.e., data where there are repeated observations over multiple time periods) and the inclusion of “fixed effects.” Most project analysis in the voluntary carbon market already make use of panel data, and are therefore strong candidates to adopt these classes of models in conjunction with, or as an alternative to, matching.

   
   

3. Recognize the limitations of the method. Causal inference models can increase the quality of the voluntary carbon market. Yet, I am struck by the over-precision claimed by some parties applying these techniques. All models have uncertainty which can come from uncertainty in the underlying data or in the model structure itself. Thus far, I’ve not seen this uncertainty properly accounted for and communicated in most applications within the voluntary carbon market. We should be doubtful of claims regarding a precise amount of carbon removal that do not also include measures of uncertainty as well.
   
   Causal inference methods are powerful, but their results should be interpreted with humility. Uncertainty abounds in these models and as scientists we should be sure that this uncertainty is understood by the communities we serve. While uncertainty exists, it is not a reason for inaction. We cannot let uncertainty become a reason for inertia, rather we must work to properly understand how to integrate uncertainty into decision making.
   
   

The science (and art) of causal inference

At Carbon Direct, we take an approach that relies on multiple lines of evidence. Rather than relying on set methodologies for all projects of the same type, we draw from our extensive modeling toolkit to choose the best set of models on a project-by-project basis. We do not rely on matching statistics alone, but also integrate difference-in-difference models, fixed effects panel regressions, regression discontinuity design, Heckman sample selection models, and other modeling techniques to understand carbon credit quality.

Which models ultimately produce the strongest evidence, and assure our clients that recommended projects are high quality, will depend on the location and type of project, data availability, whether the project is still in development or if credits are offered ex-post. Our project-by-project approach, combined with our deep experience with the voluntary carbon market and expertise in applying these methods to real world situations means our clients can feel confident in making decisions based on our work.

Overall, the application of causal inference to the voluntary carbon market is already having a positive impact on the market by bringing a new rigor to calculating carbon credits from counterfactual baselines. Yet there is still work to be done to improve our methods and bring greater integrity to the voluntary carbon market.