The Problem with Scoring

Numbers, trends and graphs are not always what they seem, but they are very appealing in a fact-minded culture. Scoring systems used in nonprofit institutions, while they do work, they blind us to the many opportunities that exist within our constituency. The problem with scoring is that it introduces bias. It may be biased based on wealth, it may be based on professional expertise, etc. Most scores on the market currently are based on a formula, and because data is always open to interpretation from humans, formulas and scores will inherently use data to support human bias.

Take, for example, a major donor score. Let's say that score is based on a formula that weights wealth at 33% of the score. It then inherently says that 1/3rd of the score is determined by wealth and that the bias of the decision maker who weighted the score is pre-set. The bigger issue is not the bias built into the formula but that your score itself, once fully produced, then creates a bias amongst the people who have to use it. The very bias we expect is then confirmed when we use these scores because it is important to note that the top-scored people of any formula will always be good prospects. They are good prospects because that is how the formula was built. However, these scores ignore opportunities and stunt the growth of opportunities at the bottom of the pipeline.

Scoring is useful because development officers must prioritize activities to meet goals, and scores are one way to help with that. Lower-ranked people require cultivation and time. This means that cultivation is often ranked lower on their priority list, and the cultivation of the lower-ranked priorities may naturally be ignored. A cycle of ignoring opportunities can develop when we think that we understand what a major donor looks like because we start to believe the biases that are built into the score. There becomes a tendency to not cultivate the people who do not fit into our version of what we think a major donor is. If a student gets D’s or F’s throughout school, they are labelled as a poor student, and therefore, their value is often determined by a score that does not holistically look at their potential.

Should they allow other people’s opinions to determine their sense of worth, then we would miss out on some of the greatest talents the world has ever seen, and we do. Scoring major gift prospects based on formulas ingrained with our own biases inherently rules people out in the same way that grading children can in school. It is not only who a donor is based on society's classifications, but what they are doing and what they care about that will be a better indicator of their propensity to give. Before recently, this kind of behavioural constituent data was unavailable because the platforms that generate it did not exist for nonprofits.

The proliferation of data has made it possible to see people in ways we didn't before. Traditionally, the data available for nonprofits to build their prospect lists was very limited and, with rare exceptions, restricted to wealth, donation, and demographic information. For decades, although there have been many studies on open rates of direct mail, we didn't actually know whether a donor read our direct mail letter, but we may surmise that they did if we got a reply card. Now, however, even if they don't donate online, we know if they opened an email, clicked on a particular story, or performed a variety of other actions.

This data needs to be generated, captured and then factored into list building in order to create a more accurate profile of what a major donor actually looks like vs what we think that they look like. Machine learning can then use this data to make predictions on which constituents are likely to take or not take a certain action, be it making a major gift, upgrading their gift, lapsing on giving, repeating their gift, etc. The power of behavioural data is already evidenced by the scale and influence of giants like LinkedIn, Facebook, Amazon, Netflix, and Google. What they all have in common is that their harnessing of data has enabled their extreme growth and allowed them to experience the profitable benefits of recurring revenue.