Search Term Friday: Type 1 and Type 2 Errors and Deciding Who Is Going to Hell

I think a lot of people are surprised to find out that I don't teach theology. Nor do I teach any classes in our College of Biblical Studies.

I am an experimental psychologist. Which means that I mainly teach undergraduate and graduate statistics classes. And from time to time on this blog I've used statistics to illustrate various theological issues or ideas.

One such attempt was using what is called Type 1 and Type 2 errors to illustrate how we make decisions about who is or is not going to heaven or hell. Every week search terms about "type 1 and type 2 errors" link people to that playful but serious post:

As I said, a large part of my day job is teaching statistics. Still, I often let theological issues emerge in my statistical lectures. Worlds collide was it were. For example, consider how Type 1 and Type 2 errors can help us think about who is going to heaven or hell.

What are Type 1 and Type 2 errors? When researchers look at trends in data sets they have to make a decision about if the trend they are looking at is real or illusory. By "real" I mean that the trend is due to some underlying causal mechanism. However, trends can emerge in data by mere chance. Think of the constellations in the sky. We see patterns up there--the figures of the Zodiac--but the patterns are the product of random forces.

In short, given our knack for seeing order in randomness researchers need tools to determine if a given trend is real or illusory. In the social sciences this tool is called Null Hypothesis testing. The Null Hypothesis is the assumption that the trend you are seeing, despite appearances, is illusory. It is "due to chance." Generally, an Alternative Hypothesis is pitted against the Null Hypothesis. The Alternative Hypothesis is the assumption that the trend is real, due to some systematic relationship between the two variables.

In Null Hypothesis Testing you assess the viability of the Null (by using probabilities) to see if the Null or the Alternative Hypothesis is the best explanation for what you are seeing. Starting with the Null the researcher faces one of two choices: "Reject" or "Fail to Reject" the Null. If you reject the Null you think the Alternative hypothesis is the best explanation for the data: You think the trend you are seeing is real. If you Fail to Reject the Null you stick with the Null and conclude that the trend you are seeing is illusory, likely due to chance.

Now what is interesting about all this is that the whole process is error-prone. You could reject the Null (think the trend is real) when, in fact, the trend was illusory. Or, you could think the trend is illusory (fail to reject the null) when, in fact, the trend is real.

In short, there are two kinds of mistakes you can make. The first is called a Type 1 error when you reject the Null (consider the trend to be real) when the Null is true (the trend is actually illusory). The second is a Type 2 error when you fail to reject the Null (consider the trend illusory) when the Null is false (the trend is actually real).

Personally, I find this language confusing. An easier way to think of it is to note that Type 1 errors are "false positives": You make a positive claim but are wrong (i.e., you claim the trend is real but it is not). A Type 1 error is like crying wolf. You reject the Null and cry "Eureka!" But you are wrong. You didn't make the discovery you thought you made. Conversely, Type 2 error is a "false negative." Instead of crying wolf a Type 2 error is a Trojan Horse kind of mistake. You didn't raise the alarm, but you should have. You missed the threat (or that trend in the data set).

To illustrate all this I draw the following table on the board a couple of times a year:
Again, that table isn't very easy to understand so I use the following illustration to help my students get the logic of Type 1 and Type 2 errors. Imagine, I say, a guy who has just started dating a girl. He remembers that on their first date she mentioned that her birthday was coming up this week. But the guy can't remember the exact day. It might be today. Or maybe not. Embarrassed to admit to her that he didn't remember he decides to make a guess. He has two choices. When he sees her today he can say "Happy Birthday!" Or he can say nothing, hoping that today isn't her birthday. The reality behind the situation is pretty simple: Either today is her birthday or it isn't.

The four possible outcomes--guesses plotted against reality--are given below:
Saying "Happy Birthday!" when it is not her birthday is like a Type 1 error. It is a false positive: I'm saying it is your birthday when, in fact, it isn't.

Conversely, staying quiet when today is her birthday is like a Type 2 error. It is a false negative: Today is my birthday and you said nothing to me, you missed it.

Now at this point you are probably wondering, what does any of this have to do with theology? Well, the interesting thing in all this is that Type 1 and Type 2 errors are pretty much everywhere. And they often occur when you have to make decisions about people.

Think about hiring practices or college admissions. On the front end you have to make a choice: Will they thrive or fail? After the selection (the hire or admission) the reality unfolds. They either do a good job or they don't. They either graduate or they don't. In short, whenever we make decisions about people we often make Type 1 and Type 2 errors. We hire people who flake out on us. Or admit people who can't make the grade. These are Type 1 errors, "false positives." But we also pass on people who would have made great hires. Or we deny admission to students who would have graduated. These are Type 2 errors, the "false negatives."

And here is where the theological application emerges. Christians often divide the world into two groups, the saints and the sinners. The saved and the lost. The elect and the unregenerate. The church and the world. Think of this as a kind of "admissions decision." Are you "accepted" into the "church" or not?

And these are not trivial considerations. How does the church define where its borders will be? Where does the right hand of fellowship begin and end? How inclusive or exclusive should we be? Who needs to be evangelized?

Who is going to heaven and who is going to hell?

Importantly, Type 1 and Type 2 errors will be involved in this process. For example, there might be people we fellowship who, in the eyes of God, stand under judgment. These would be Type 1 errors. These are the errors conservatives think liberals are making. Liberals are extending the label "Christian" to people who are actually damned. In the eyes of conservatives liberals falsely extend grace to people who stand under God's judgment.

By contrast, liberals condemn conservatives for making Type 2 errors, damning and excluding people God loves.

What makes these disagreements tense is that this side of heaven no one knows who is right or who is wrong. Which is frustrating for all parties.

But I think something more can be said about the matter.

Specifically, although we don't know in a given case if we are making a Type 1 or Type 2 error we can choose the kinds of errors we will generally make. And this raises a very interesting question: Knowing we are going to make errors, what kind of errors should we make?

What should be our theology of error?

Let me explain what this looks like.

As I said above, Christians, while looking at individuals and groups, make judgments regarding the status of other people. Are these other people "Christians" or not? Likely, these judgments are a mix of doctrinal and moral observations. What do these people believe and what do they do with their lives? Plot these judgments on a horizontal axis, from sinners to saints.

While we are making these judgments God is making God's own judgments. Plot God's judgments on a vertical axis, also going from sinners to saints.

With both axes plotted we have something like the following, with the red dots representing people or communities:
A couple of observations about this diagram. What I'm trying to show here is that, while there is some agreement between God and the judging person, there is also some discrepancy between human and divine judgments. That is, when humans see someone as virtuous or wicked God, for the most part, agrees. However, the association isn't perfect. We might see someone a very virtuous while God knows him to be hypocrite. Or, we might label someone a "sinner" where God sees this person as a saint. That is, generally vice is vice and virtue is virtue, in heaven and on earth. But there are locations of disagreement. Our perceptions of morality don't always align with God's perspective.

Now, in light of these judgments Christians have to make a choice, grouping people into the categories of Saved or Lost. In statistical language, the point at which this judgment is made is called the selection ratio:
You can think of the selection ratio as a kind of "cut off" score where a decision is made. Like a minimum SAT or ACT score for a college admission. Scores above the cut off are "selected," admitted to the college. Scores below the selection ratio are denied admission.

But the selection ratio isn't the only thing in play. In heaven God has God's own opinion about who is Lost or Saved. And God's decision is going to define the ultimate outcome. In statistical language God's decision is called the base rate. God's base rate is represented below:
With these ideas in hand we can now see how the church can make Type 1 and Type 2 errors. Specifically, using some sort of selection ratio the church carves up the world into the Saved and the Lost. The former are called "the church" and the latter are objects of judgment and evangelism. But behind all this is God's base rate, God's own judgment about who will be saved and who will not be saved. And given that human and divine opinions are not in 100% agreement (due to our human limitations) we have the possibility for error. That is, we might see someone as Lost who, in the end, God will Save. Or, we can see someone as Saved who, in the the end, God will damn. These errors can be seen when we plot the selection ratio and the base rate together:
These errors are just like the Type 1 and Type 2 errors encountered above:
Okay, we are nearing the theological payoff of these observations. Here's the important point: By adjusting the selection ratio we can affect the kinds of errors we will make. We have two choices here.

One thing we can do is adjust the selection ratio upward, moving it rightward. This is what we call "raising standards." As the selection ratio moves toward the right fewer and fewer people are selected because the standards are going up. Think again of college admissions. If we raise the SAT/ACT scores we move the selection ratio to the right. This reduces our overall admissions as we become more "selective."

The advantages and disadvantages of increasing the selection ratio are obvious. On the downside we are reducing quantity. But on the upside we are improving quality. And by improving quality we are reducing a kind of error. Specifically, we are reducing our Type 1 error rate. We are making fewer "false positives." By increasing quality standards we aren't admitting people who can't make the cut.

However, and this is the key insight, in decreasing our Type 1 errors we have increased our Type 2 errors. By becoming more selective we've reduced our false positives but have increased our false negatives. That is, we are turning away more and more students from the school when many of these students would have been successful if we had admitted them.

This is, I would argue, a model for how conservative churches are handling the judgments of Saved and Lost. Conservative churches have increased the selection ratio, they raise holiness and/or doctrinal standards. And they do this to be "conservative," to reduce false positives. They want to be sure to fellowship only the true Christians. They don't want to waste fellowship on the Lost. This strategy is shown below:
By increasing their standards of fellowship the conservative church has become more selective, judgmental and discriminating.

The trouble, as we see in the figure above, is that when you move the selection ratio rightward you are, indeed, decreasing false positives. However, while you are reducing false positives you are increasing your number of false negative (Type 2) errors. That is, by raising standards you are disfellowshiping people whom God loves. You think a lot of people on earth are going to hell when, in fact, they are going to be with you in heaven. In short, heaven is going to be a lot more crowded than your selection ratio led you to believe.

So what happens when you lower standards, becoming more inclusive as we see in many liberal churches? This is what happens when you move the selection ratio leftward:
As you can see, when you move the selection ratio to the left, fellowshiping more and more people, you systematically reduce your false negative errors. By casting such a broad net of fellowship you "catch" all the Saved. But your inclusiveness also catches a lot of the Lost. That is, in reducing your false negatives you've increased your false positives. Just the opposite situation we saw in the conservative church.

And here--finally!--we can get to the issue regarding our "theology of error." That is, although we can never know when and where we are making errors (i.e., we don't know God's base rate) we can, generally speaking, choose the kinds of errors we will make. Specifically, we can increase or decrease our selection ratio. We can become more exclusive or inclusive. More sectarian or more ecumenical.

So which direction should we go?

Well, one way to make that decision is to think theologically about the kinds of errors related to those decisions. If we become more exclusive we make Type 2--false negative--errors. That is, we begin disfellowshiping other Christians. We've become stingy with God's grace. Conversely, if we become more inclusive we make more Type 1--false positive--errors. We begin extending fellowship to sinners. We've become too profligate with God's grace.

My point here is that I think these patterns of errors can be evaluated on theological grounds. We can ask questions like, should we be a Type 1 error (inclusive) or Type 2 error (exclusive) church? We have some control over this.

Basically, given human fallibility and sin errors are inevitable. But you can pick your errors. So which error does God want us to make?

In my personal opinion, if errors are inevitable I think God wants me to lean toward making Type 1 errors. If errors are inevitable I should err on the side of grace. True, my "liberalism"--my tilt toward Type 1 error--in this regard will cause me to extend grace where, according to conservatives, I shouldn't.

I am not immune to that criticism, in being "liberal" I know I'm risking a higher Type 1 error rate. But I prefer this to the alternative of running a high Type 2 error rate, being judgmental of people among whom God is at work. And I make that choice for a host of theological reasons.

But of course, others might see this all differently. Regardless, I think this analysis highlights an important issue:

You are going to make errors about God's grace. You can't control that. So in light of that fact, what kinds of errors are you going to make?

Because you do have some control over that.