Search Term Friday: Type 1 and Type 2 Errors and Deciding Who Is Going to Hell

I think a lot of people are surprised to find out that I don't teach theology. Nor do I teach any classes in our College of Biblical Studies.

I am an experimental psychologist. Which means that I mainly teach undergraduate and graduate statistics classes. And from time to time on this blog I've used statistics to illustrate various theological issues or ideas.

One such attempt was using what is called Type 1 and Type 2 errors to illustrate how we make decisions about who is or is not going to heaven or hell. Every week search terms about "type 1 and type 2 errors" link people to that playful but serious post:

As I said, a large part of my day job is teaching statistics. Still, I often let theological issues emerge in my statistical lectures. Worlds collide was it were. For example, consider how Type 1 and Type 2 errors can help us think about who is going to heaven or hell.

What are Type 1 and Type 2 errors? When researchers look at trends in data sets they have to make a decision about if the trend they are looking at is real or illusory. By "real" I mean that the trend is due to some underlying causal mechanism. However, trends can emerge in data by mere chance. Think of the constellations in the sky. We see patterns up there--the figures of the Zodiac--but the patterns are the product of random forces.

In short, given our knack for seeing order in randomness researchers need tools to determine if a given trend is real or illusory. In the social sciences this tool is called Null Hypothesis testing. The Null Hypothesis is the assumption that the trend you are seeing, despite appearances, is illusory. It is "due to chance." Generally, an Alternative Hypothesis is pitted against the Null Hypothesis. The Alternative Hypothesis is the assumption that the trend is real, due to some systematic relationship between the two variables.

In Null Hypothesis Testing you assess the viability of the Null (by using probabilities) to see if the Null or the Alternative Hypothesis is the best explanation for what you are seeing. Starting with the Null the researcher faces one of two choices: "Reject" or "Fail to Reject" the Null. If you reject the Null you think the Alternative hypothesis is the best explanation for the data: You think the trend you are seeing is real. If you Fail to Reject the Null you stick with the Null and conclude that the trend you are seeing is illusory, likely due to chance.

Now what is interesting about all this is that the whole process is error-prone. You could reject the Null (think the trend is real) when, in fact, the trend was illusory. Or, you could think the trend is illusory (fail to reject the null) when, in fact, the trend is real.

In short, there are two kinds of mistakes you can make. The first is called a Type 1 error when you reject the Null (consider the trend to be real) when the Null is true (the trend is actually illusory). The second is a Type 2 error when you fail to reject the Null (consider the trend illusory) when the Null is false (the trend is actually real).

Personally, I find this language confusing. An easier way to think of it is to note that Type 1 errors are "false positives": You make a positive claim but are wrong (i.e., you claim the trend is real but it is not). A Type 1 error is like crying wolf. You reject the Null and cry "Eureka!" But you are wrong. You didn't make the discovery you thought you made. Conversely, Type 2 error is a "false negative." Instead of crying wolf a Type 2 error is a Trojan Horse kind of mistake. You didn't raise the alarm, but you should have. You missed the threat (or that trend in the data set).

To illustrate all this I draw the following table on the board a couple of times a year:
Again, that table isn't very easy to understand so I use the following illustration to help my students get the logic of Type 1 and Type 2 errors. Imagine, I say, a guy who has just started dating a girl. He remembers that on their first date she mentioned that her birthday was coming up this week. But the guy can't remember the exact day. It might be today. Or maybe not. Embarrassed to admit to her that he didn't remember he decides to make a guess. He has two choices. When he sees her today he can say "Happy Birthday!" Or he can say nothing, hoping that today isn't her birthday. The reality behind the situation is pretty simple: Either today is her birthday or it isn't.

The four possible outcomes--guesses plotted against reality--are given below:
Saying "Happy Birthday!" when it is not her birthday is like a Type 1 error. It is a false positive: I'm saying it is your birthday when, in fact, it isn't.

Conversely, staying quiet when today is her birthday is like a Type 2 error. It is a false negative: Today is my birthday and you said nothing to me, you missed it.

Now at this point you are probably wondering, what does any of this have to do with theology? Well, the interesting thing in all this is that Type 1 and Type 2 errors are pretty much everywhere. And they often occur when you have to make decisions about people.

Think about hiring practices or college admissions. On the front end you have to make a choice: Will they thrive or fail? After the selection (the hire or admission) the reality unfolds. They either do a good job or they don't. They either graduate or they don't. In short, whenever we make decisions about people we often make Type 1 and Type 2 errors. We hire people who flake out on us. Or admit people who can't make the grade. These are Type 1 errors, "false positives." But we also pass on people who would have made great hires. Or we deny admission to students who would have graduated. These are Type 2 errors, the "false negatives."

And here is where the theological application emerges. Christians often divide the world into two groups, the saints and the sinners. The saved and the lost. The elect and the unregenerate. The church and the world. Think of this as a kind of "admissions decision." Are you "accepted" into the "church" or not?

And these are not trivial considerations. How does the church define where its borders will be? Where does the right hand of fellowship begin and end? How inclusive or exclusive should we be? Who needs to be evangelized?

Who is going to heaven and who is going to hell?

Importantly, Type 1 and Type 2 errors will be involved in this process. For example, there might be people we fellowship who, in the eyes of God, stand under judgment. These would be Type 1 errors. These are the errors conservatives think liberals are making. Liberals are extending the label "Christian" to people who are actually damned. In the eyes of conservatives liberals falsely extend grace to people who stand under God's judgment.

By contrast, liberals condemn conservatives for making Type 2 errors, damning and excluding people God loves.

What makes these disagreements tense is that this side of heaven no one knows who is right or who is wrong. Which is frustrating for all parties.

But I think something more can be said about the matter.

Specifically, although we don't know in a given case if we are making a Type 1 or Type 2 error we can choose the kinds of errors we will generally make. And this raises a very interesting question: Knowing we are going to make errors, what kind of errors should we make?

What should be our theology of error?

Let me explain what this looks like.

As I said above, Christians, while looking at individuals and groups, make judgments regarding the status of other people. Are these other people "Christians" or not? Likely, these judgments are a mix of doctrinal and moral observations. What do these people believe and what do they do with their lives? Plot these judgments on a horizontal axis, from sinners to saints.

While we are making these judgments God is making God's own judgments. Plot God's judgments on a vertical axis, also going from sinners to saints.

With both axes plotted we have something like the following, with the red dots representing people or communities:
A couple of observations about this diagram. What I'm trying to show here is that, while there is some agreement between God and the judging person, there is also some discrepancy between human and divine judgments. That is, when humans see someone as virtuous or wicked God, for the most part, agrees. However, the association isn't perfect. We might see someone a very virtuous while God knows him to be hypocrite. Or, we might label someone a "sinner" where God sees this person as a saint. That is, generally vice is vice and virtue is virtue, in heaven and on earth. But there are locations of disagreement. Our perceptions of morality don't always align with God's perspective.

Now, in light of these judgments Christians have to make a choice, grouping people into the categories of Saved or Lost. In statistical language, the point at which this judgment is made is called the selection ratio:
You can think of the selection ratio as a kind of "cut off" score where a decision is made. Like a minimum SAT or ACT score for a college admission. Scores above the cut off are "selected," admitted to the college. Scores below the selection ratio are denied admission.

But the selection ratio isn't the only thing in play. In heaven God has God's own opinion about who is Lost or Saved. And God's decision is going to define the ultimate outcome. In statistical language God's decision is called the base rate. God's base rate is represented below:
With these ideas in hand we can now see how the church can make Type 1 and Type 2 errors. Specifically, using some sort of selection ratio the church carves up the world into the Saved and the Lost. The former are called "the church" and the latter are objects of judgment and evangelism. But behind all this is God's base rate, God's own judgment about who will be saved and who will not be saved. And given that human and divine opinions are not in 100% agreement (due to our human limitations) we have the possibility for error. That is, we might see someone as Lost who, in the end, God will Save. Or, we can see someone as Saved who, in the the end, God will damn. These errors can be seen when we plot the selection ratio and the base rate together:
These errors are just like the Type 1 and Type 2 errors encountered above:
Okay, we are nearing the theological payoff of these observations. Here's the important point: By adjusting the selection ratio we can affect the kinds of errors we will make. We have two choices here.

One thing we can do is adjust the selection ratio upward, moving it rightward. This is what we call "raising standards." As the selection ratio moves toward the right fewer and fewer people are selected because the standards are going up. Think again of college admissions. If we raise the SAT/ACT scores we move the selection ratio to the right. This reduces our overall admissions as we become more "selective."

The advantages and disadvantages of increasing the selection ratio are obvious. On the downside we are reducing quantity. But on the upside we are improving quality. And by improving quality we are reducing a kind of error. Specifically, we are reducing our Type 1 error rate. We are making fewer "false positives." By increasing quality standards we aren't admitting people who can't make the cut.

However, and this is the key insight, in decreasing our Type 1 errors we have increased our Type 2 errors. By becoming more selective we've reduced our false positives but have increased our false negatives. That is, we are turning away more and more students from the school when many of these students would have been successful if we had admitted them.

This is, I would argue, a model for how conservative churches are handling the judgments of Saved and Lost. Conservative churches have increased the selection ratio, they raise holiness and/or doctrinal standards. And they do this to be "conservative," to reduce false positives. They want to be sure to fellowship only the true Christians. They don't want to waste fellowship on the Lost. This strategy is shown below:
By increasing their standards of fellowship the conservative church has become more selective, judgmental and discriminating.

The trouble, as we see in the figure above, is that when you move the selection ratio rightward you are, indeed, decreasing false positives. However, while you are reducing false positives you are increasing your number of false negative (Type 2) errors. That is, by raising standards you are disfellowshiping people whom God loves. You think a lot of people on earth are going to hell when, in fact, they are going to be with you in heaven. In short, heaven is going to be a lot more crowded than your selection ratio led you to believe.

So what happens when you lower standards, becoming more inclusive as we see in many liberal churches? This is what happens when you move the selection ratio leftward:
As you can see, when you move the selection ratio to the left, fellowshiping more and more people, you systematically reduce your false negative errors. By casting such a broad net of fellowship you "catch" all the Saved. But your inclusiveness also catches a lot of the Lost. That is, in reducing your false negatives you've increased your false positives. Just the opposite situation we saw in the conservative church.

And here--finally!--we can get to the issue regarding our "theology of error." That is, although we can never know when and where we are making errors (i.e., we don't know God's base rate) we can, generally speaking, choose the kinds of errors we will make. Specifically, we can increase or decrease our selection ratio. We can become more exclusive or inclusive. More sectarian or more ecumenical.

So which direction should we go?

Well, one way to make that decision is to think theologically about the kinds of errors related to those decisions. If we become more exclusive we make Type 2--false negative--errors. That is, we begin disfellowshiping other Christians. We've become stingy with God's grace. Conversely, if we become more inclusive we make more Type 1--false positive--errors. We begin extending fellowship to sinners. We've become too profligate with God's grace.

My point here is that I think these patterns of errors can be evaluated on theological grounds. We can ask questions like, should we be a Type 1 error (inclusive) or Type 2 error (exclusive) church? We have some control over this.

Basically, given human fallibility and sin errors are inevitable. But you can pick your errors. So which error does God want us to make?

In my personal opinion, if errors are inevitable I think God wants me to lean toward making Type 1 errors. If errors are inevitable I should err on the side of grace. True, my "liberalism"--my tilt toward Type 1 error--in this regard will cause me to extend grace where, according to conservatives, I shouldn't.

I am not immune to that criticism, in being "liberal" I know I'm risking a higher Type 1 error rate. But I prefer this to the alternative of running a high Type 2 error rate, being judgmental of people among whom God is at work. And I make that choice for a host of theological reasons.

But of course, others might see this all differently. Regardless, I think this analysis highlights an important issue:

You are going to make errors about God's grace. You can't control that. So in light of that fact, what kinds of errors are you going to make?

Because you do have some control over that.

This entry was posted by Richard Beck. Bookmark the permalink.

23 thoughts on “Search Term Friday: Type 1 and Type 2 Errors and Deciding Who Is Going to Hell”

  1. In deciding which error to make, this is when I remember the commandment to not judge. I do not know what God's grace is doing, but I surely should not limit it. And thanks for the stats post. I majored in Theatre and one minor was English lit, but my other minor was Psychology, heavy on statistics -- I needed the balance.

  2. I enjoyed this thoroughly. It helped me to understand from where I am coming from myself. As I grew older, I began to see the Christian's journey as one in which he or she crosses the threshold from an emotional faith, meaning the family theology, into a theology of choice. I just didn't know what it was until now. For years I have at times emotionally missed the Church in which I grew up; yet when I would ask myself, do I accept the conclusions that many in this church would demand of me, my choices and my answers were "No".

    In regard to deciding who goes to hell, I think it interesting how those in conservative churches can go down the list of their theology and without any compunction, accept hell as the ultimate fate of all who disagree; yet, step out of their theological world at will into the world of entertainment, politics, sports, etc, and embrace others involved as if hell did not exist at all. An example would be a conservative Christian sitting in a concert of a favorite entertainer, greatly enjoying the show and wildly applauding the individual as a talented artist. If you were to tap the Christian on the shoulder and ask, "But don't you believe this person is going to hell?" While the answer, if they were in a Bible study, would be "yes", at that particular moment the Christian would take the question as being intrusive, arrogant and rude.

    Like you, I choose to make Type 1 errors. I do this in accepting all as children of God, and that salvation is coming to that awakening, an awakening that is molded and shaped by the gifts and the light we are given. And hell? That is when we fear to keep God and humanity in the same thought.

  3. That's terrific! It gives new meaning to the old Twainian saying about lies, damned lies, and statistics! ;)

    Cf.: "And Jesus spake to the people in in graphs and ratios, and they thought, 'WTF?'"

    Right option, of course, Richard -- with dominical authority, as per the parable of the Tares!

    And then there's our old drunk-on-grace friend, "We're-all-bastards-but-God-loves-us-anyway" Will Campbell:

    "Can we stand the idea that God is kind to the wicked? That they, no less than the oppressed, are slaves of the powers whom God has chosen, with his very blood, to liberate? Do we really dare to accept a liberation with and alongside the wicked? Because God in Christ offers no other liberation."

  4. So, all the conservative Christians I know would argue that a Type 1 Error has far graver repercussions than a Type 2 Error. If you mistake a "saved" person as "unsaved" you will still see them in Heaven - no harm, no foul. But if you accept a sinner as "saved" when they are in fact "unsaved", you are losing an opportunity to confront the sinner and possibly convert him. His damnation is on your hands. A Type 2 Error is less risky and therefore preferable.

    I have my own opinions about this, but I'm curious what you think.

  5. I just had a coffee friend argue that the Bell Curve is pronen by statistics. I disagreed. If I am right, what kind of error is my friend making? Is this even applicable to th discussion?

  6. The way it would have happened in the church I grew up in would have been more like a Type 2 error results in driving someone who is saved out of the church and having them warn others away and a Type 1 error results in driving someone who is unsaved away to the point where they will only ever hear 'have you ever heard of Jesus?' hard-sell witnessing and not anything that actually respects how far they'd gotten. Type 1 causes issues for one person, Type 2 has a domino effect.

    Throw the sheep and goats scripture passage in there, and you can have a situation where all the compassionate people who made the 'mistake' of bending the rules a bit for the good of another get tangled in Type 2 errors and begin warning off anyone else willing to do that from identifying with Christianity. You end up with a bunch of goats who can stay in the church because nothing ever gives them a good reason to not conform to the superficial set of guideline, a bunch of sheep who keep getting driven off because they'll bend rules if it means someone gets fed or has a place in a pew without being the sermon illustration, and a bunch of people who fundamentally can't meet the guidelines no matter how they live or believe -- the way our Sunday School classes were run, the push to show dyslexics the church doors started at age seven or eight and the kids with less than perfect memory around ten or eleven -- run off and told God can never accept them unless they are transformed into someone completely different.

    (Which is how we've ended up with a church culture in America where some very loud fundamentalists have convinced a whole lot of people that no one who meets the Biblical criteria for being a sheep can be a proper Christian and go to heaven. On both sides of the church doors.)

    In my experience, false negatives are the bigger problem. False positives, you've got someone culturally inside the Christian community and open to influence by example if nothing else. False negatives, you've got people driven away from the church unjustly and sometimes pretty well psychologically broken by the process.

  7. From what I have observed and experience over the years, conservatives prefer Type 2 error because it guard's the identity of the "body"; it protects its uniqueness. "Too many questionable ones dilute the strength" is the way they approach people and ideas.

  8. Because we as humans will make errors one way or another, is there a relationship with how one errors concerning their own salvation before judging others? What's amazing about the judgmental types (making Type 2 errors about others despite repeated admonishment to examine ourselves, especially in the NT) - it's as if it has never occurred to them they might be making a Type 1 error about themselves?

  9. Interesting! What are your thoughts on churches that have a very strict definition for who is in communion with "the Church," but avoid setting limits at all on who is ultimately saved - for instance, Orthodox churches? Is there a way to adapt this framework to their soteriological situation?

  10. Love this post! One quibble-- you describe conservatives as not wanting to "waste fellowship" on the lost. But I see it more as a "who can you trust" insider outsider issue. Conservatives want to influence the lost without the danger of being influenced by them.

  11. May I please be allowed, as a scientist, to ask two questions? Are you presenting this post as scientific theory or proof of something (hypotheses, charts, graphs, statistics)? And, whether you are or not, are you speaking as an individual universalist, or on behalf of any church or group? Thank you.

  12. Richard, I think this article is amazing, very computer-nerd-like.

    I’d like to make some observations:

    1) I think I’m with you. If we make type 1 errors, aren't we inviting both the lost and the saved to the church, the place where they may best hear and experience the gospel?

    2) The problem conservatives have with liberals, in my opinion, is that the former think the latter is making it impossible to the sinner become a saint. In the conservative mind, the liberals are the true sinners, closing the doors of heaven. In fact, they think there's no gospel or God in the liberal's churches, they're all lost.

    3) When you say "people whom God loves", my mind goes back to your post about God’s unconditional love. Are there people who God doesn't love? Are people in hell hated by God?

    Sometimes, I wonder if Hell is in fact product of God's love. You know... "God only punishes his children, his beloved ones"

    (Sorry if I wasn't clear, English is my second language)

  13. In the parable of the weeds (Mc 13:24-30) the servants are told not to pull the weeds up, least that they may uproot the wheat with them. I suspect that this makes a good argument to prefer Type 1 errors.

  14. Although as a former-Christian Atheist, I think universalism is the least harmful of Christian reasoning on Heaven and Hell. I think Heaven and Hell thinking is rotten to the core and silly to use any sort of statistical reasoning on. But perhaps using silly reasoning to talk people (conservatives) out of even worse error is forgivable.

    Let’s consider your logic on another issue: “Which religion(s) is true?”

    Imagine your god’s views toward the truth of all religions: Maybe Yahweh-Jesus (your god) feels all religions are true but Christians create instead think only their religion is true — thus creating a too many false negatives. So by your logic, wouldn’t it seem the right bet to say all religions are true so as not to judge more harshly than Yahweh-Jesus?

    Thus Christians should be Pluralists.

    We could do the same on many issues, but in the end you are using your intuitive judgement to decide — not logic. A conservative Christian would say we need to rely on scriptures because our judgements are soiled by sin whereas holy scripture is flawless.

    Although the graphs are fun, I don’t think it will be persuasive to anyone but to those who agree with you. But I hope I am mistaken.

  15. You have to admit though that his extended metaphor/analogy of “Pascal’s Wager” is really quite ingenious. “Decision theory” can be woven into nearly every facet of life. There are also correlations to Daniel Kahneman’s work on “hedonic psychology”. I wonder if God sees us as little red dots?

  16. The god-in-my-head doesn’t see people at all, yet alone as red dots — see my post on “Roadkill Theology”. In fact, that god doesn’t see, feel, mourn or any of that human stuff. But instead, I will confess that I have a Mother-Monkey god feeling in my heart and head (see this post).

    Yes, Cercatore, indeed when I read Richard’s post, I was reminded of Pascal’s Wager. I used to teach statistics and research to graduate students and love this way of thinking of things, but it did seem silly to have a god and a god judgement involved at all. But if I viewed this post as a way to talk to conservative Christians, I could see how this could be useful, albeit really mistaken, just as Pascal’s Wager is highly mistaken though not intuitively obvious. But indeed, when I taught stats, I told students that the problem with stats is that our brains naturally generate statistical crap — in other words — statistics is not intuitively obvious and we must spend much time to train our brains to intuit anywhere close to correct. But the problem is, most people trust their intuitions — this is the biggest obstacle to science.

    And I agree that to understand humans, it is best to develop systems which understand that humans intuitions often lead to non-rational choices whereas prescriptive decision theory was intended to tell people they should. The problem is, value is always part of the picture but after being overwhelmed by the math and tree branching, that human value (like imagining a judging god) can be lost. People often miss where the values got snuck in.

    Even Hedonic Psychology has its short comings.

    Here is a patient I saw two days ago:

    His blood sugar was 290, I told him he needs a work-up for diabetes, but he knew diabetes may need pills or insulin injections. He said, “I ain’t doin’ that. It is a matter of quality of life.” So he had no idea what “quality of life” meant, because he could not imagine his horrible future, but instead (after further questioning), he just thought he died a little earlier with uncontrolled diabetes, not that his last 10-20 years may be hell. Humans are far more dumb than they even imagine — their intuitions are horrible for many things.

  17. So I am free to label this as opinion dressed up as "science"? I would call that pseudoscience, like astrology. Given your day job, seems a little shady to me. Graphs and charts and numbers have no real purpose, function, or meaning in this context, since they do not measure anything. But perhaps that was the point? No wonder there are skeptics out there.

  18. Sorry, here is the roadkill theology link:

    And here is the Mother-Monkey god link:

  19. The main problem I have with this is in my experience many on the far right of the theological spectrum will not admit that there is any uncertainty about who is saved and who is not. For them is is simply a horizontal axis. Their whole worldview is predicated on special divine revelation. So to get them to admit that they might make an error may be a bridge too far. But this does seem to be a useful framework for those not so rooted in that particular ethos.

  20. So if I choose to error on the side of type 1 errors, I extend more grace and assume God is at work. But if I build another statistical chart around over whether or not God is at work, and place "not at work" on the side of sinner and "at work" on the side of saint and again choose to error on type that is a truly peculiar place.

  21. I like this. Humanity corrupted, none free from the corruption, wickedness is our normal state despite our clever attempts to justify ourselves as the righteous over against wicked others (Girard so helpful here). I wonder if Richard's preference for Type 1 errors won't become increasingly understood as basic Christianity. Why? Because our security is in Christ alone, not who is ultimately in or out. This requires discernment of Christ and his followers do this together, certainly. So there is a difference between following him or not! But our discernment, discipline, imagining judgment (future discipline) is all submitted to Christ himself. This entails humility, patience, training as participants in the steadfast love of the Lord that is Jesus himself. If we get straight that we are not given ultimate authority to judge the future (e.g. wheat and tares, Paul's argument in Romans 2 that Christ is the judge even for Gentiles--most of us, or the Lamb is on the throne--divinity submitting to scapegoating inaugurating the end of sacrificial violence), then we can let God handle any Type 2 issues. I wonder if they even exist for God who is not in rivalry, has no equal, only knowing a love poured out toward redemption of all creation.

Leave a Reply