In a recently-published editorial in the New York Times, Tulane Professor of Jewish Studies Ilana Horwitz makes the claim that when it comes to educational attainment, religiosity is beneficial for working-class young men. The claims in the article are clear illustrations of two logical fallacies: faulty generalization and cherry-picking.
Horwitz notes that the dataset for her analysis is the National Study of Youth and Religion (NSYR), a longitudinal dataset that tracked a sample of adolescents from 2003 to 2013. However, the NSYR is problematic. For instance, in Wave 1 of the survey, which was conducted in 2003, just 6% of parents were nonreligious. Nationally representative data from the General Social Survey in 2004 indicates that close to 14% of the adult population was nonreligious at that time. It appears that the NSYR undersampled nonreligious households.
The severe undersampling of nonreligious households in the NSYR dataset is just one of several problems with Horwitz’s conclusion.
Setting aside the potentially very serious problems with representativeness, if we look at the NSYR itself, we find Horwitz making a faulty generalization. The fourth wave of the NSYR includes a measure of educational attainment with the following values: (1) No degrees, (2) High school or GED, (3) Associate or Technical degree, (4) Bachelor’s, and (5) Graduate. While this is technically a nominal variable, it can be treated as a continuous variable to get a sense of whether there are meaningful differences in educational outcomes based on religiosity. In other words, higher values indicate more education, while lower values indicate less.
Here are the means for the various (non)religious groups:
| (Non)Religious Affiliation | Mean Educational Attainment |
|---|---|
| Evangelical Protestant | 2.99 |
| Mainline Protestant | 3.33 |
| Black Evangelical Protestant | 2.73 |
| Black Mainline Protestant | 3.17 |
| Catholic | 3.23 |
| Jewish | 3.90 |
| Mormon/LDS | 2.83 |
| Not religious | 2.89 |
| Other | 3.00 |
| Indeterminate Christian attender | 2.85 |
| Indeterminate attender | 2.56 |
| Indeterminate Christian non-attender | 2.73 |
| Indeterminate non-attender | 2.62 |
Just three religious groups have statistically significantly higher educational attainment than do the nonreligious: Mainline Protestants, Catholics, and Jews. And even those differences are very, very small. Yet, Horwitz suggests that religion is important for educational attainment.
In particular, she notes that it is the community of support that religions provide that helps young men to succeed educationally. To develop such a community of support, one would need to attend religious services regularly. The NSYR contains a question asking participants whether they regularly attend religious services. To get a sense of how important religious attendance is to educational attainment, I used a statistical technique called regression.
My analysis showed that there is a statistically significant relationship with attendance at religious services, but it is completely meaningless.
To explain why it is meaningless, a comparison is in order. Finding that two variables are significantly related is kind of like winning the lottery. Technically, it means that the relationship observed is unlikely to be due just to chance. But let’s continue with the lottery comparison. Just because someone won the lottery doesn’t mean it matters. Winning $2 with a scratch-off ticket is still winning the lottery, but that is a far cry from winning a Powerball jackpot worth hundreds of millions of dollars.
Rather than simply asking whether there is a relationship between religious attendance and educational attainment, we really need to know how big the relationship is. That is where the faulty generalization enters in. Knowing that someone regularly attends religious services accounts for just 0.3% of the variation in educational attainment (out of 100%). To put that into perspective, 6.5% of educational attainment can be accounted for by knowing someone’s political views (left or right) and 3.7% is accounted for by someone’s health using NSYR data.
Horwitz’s claim is like winning $2 with a scratch-off ticket, then demanding that all the major media outlets feature her lottery win. She is insinuating a meaningful relationship based on very weak premises.
The second logical fallacy in Horwitz’s editorial involves cherry-picking data. She argues that working-class men are those most likely to benefit from religiosity. It is possible that such individuals may benefit slightly more from religiosity than, say, highly-educated women, who are substantially less likely to benefit from religious participation, based on the research of Joseph O. Baker and Buster G. Smith.
Even so, what this suggests is that the relationship between religious involvement and educational attainment is not robust enough to make it a compelling generalization to all people. As plenty of research has shown for the last 100 years, elite scientists are substantially less religious than the general population, and while the relationship between educational attainment and religiosity is complex, claiming that religious involvement improves educational outcomes flies in the face of lots of prior research suggesting just the opposite.
Religious involvement’s contribution to educational attainment is so minimal as to be meaningless.
To justify this claim, Horwitz turns to qualitative data, highlighting the situation of “John” (a pseudonym). While qualitative data is important and can be extremely useful in advancing scholarly arguments, it can also be cherry-picked. Cherry-picking involves selecting only the cases that support one’s argument and ignoring the cases that do not.
What is missing from Horwitz’s editorial is an attempt to balance the qualitative data with counterexamples from similar qualitative data (see the work of Phil Zuckerman or Jesse Smith) or compelling quantitative data that shows how important religious involvement is for all people. As noted above, using the same data she used, the actual answer is that religious involvement’s contribution to educational attainment is so minimal as to be meaningless.
It may be the case that some people are more committed to education as a result of their religiosity. But we also know that some people reject education on the basis of their religiosity, like Jehovah’s Witnesses, who have the lowest educational attainment of any (non)religious group in the US. Horwitz has presented a very one-sided argument based on faulty generalizations and cherry-picked data. Based on a fair interpretation of the data, when it comes to education, religiosity doesn’t make much of a difference.
