This article was adapted from the premier book on sex differences in intelligence, Smart and Sexy by Roderick Kaine.
The politically acceptable explanation for gender differences in intelligence studies and tests is that discrimination accounts for all current disparities between men and women in intellectual Fields, starting first and foremost with the standardized tests themselves. The question is: does the data support this? The most fashionable (possibly faddish) explanation for test differences between gender (and race) come from social psychology and is termed ‘stereotype threat’. Stereotype threat is a type of implicit discrimination that is supposed to result from traditional stereotypes based on the gender of test takers. It is supposedly all pervasive throughout society, and constantly present everywhere you are. This obscure and unfalsifiable ether is supposed to depress the scores of females in tests. Society has historically held that women were not as intelligent as men. Stereotype threat proponents argue that knowledge of this history combined with the manner in which questions are designed and phrased leads to lower test scores for women in a sort of self fulfilling prophecy.
It is important to consider who should be affected by stereotype threat and other types of discrimination and what kind of pattern in the distribution of scores should result if it was having a widespread impact. Discrimination of this type is assumed to be universal and omnipresent. It is supposed to be present in both the society at large as well as in the test that all students are taking. If it is universally present then it should have a universal effect. It should affect women equally at all levels of test taking proficiency and should result in a uniform downward shift of the score distribution compared to men. In the graph below, hypothetical male and female test score distributions are super-imposed and the expected influence of stereotype threat is shown. The male distribution does not have as high a peak and extends out farther in either direction to reflect the greater male variability in scores found in virtually all IQ tests. I am not the best artist so you will have to forgive me if it isn’t as pretty as it could be.
Hypothetical male and female IQ distributions before and after stereotype threat
If it does not fit this pattern, explanations for why women at different test taking levels react differently to stereotype threat must be invented. The more disagreements from this trend, the more explanations that must contrived, and the more parsimony is lost (which generally means a theory is weaker). The difference between average IQ at least suggests that this might be happening, but the 3-5 iq points generally reported is a relatively small difference and mainly suggests that whatever universal influences do exist, their importance must be relatively minor. (However, there is reason to suspect that male IQ advantage is severely underestimated)
The discrimination theory is not the only possible explanation for this overall shift. The difference could just as easily, in fact probably much more easily, be explained by biological differences in brain development. Especially the fact that males in general grow to be larger, which translates into larger brain sizes on average, which for reasons that should be obvious correlates with higher IQ.
The discrimination and biological differences above have one thing in common: they are universal and thus are not good explanations for greater male variance which is the root source of most male/female disparity in the highest levels of achievement. A consistent universal factor should have a consistent universal effect for all levels of ability as shown in the figure above, and the only consistent universal difference in mean IQ scores between gender are small. Assuming stereotype threat is real, which is doubtful, it is not impossible that of the small difference that does exist, stereotype threat only makes a small contribution in addition to other factors like biological development. In such a case, the individual contribution of stereotype threat would be vanishingly small and would approximate complete irrelevance.
In the case of gender, stereotype threat is pretty much ruled out for the above reason. However, racial gaps do take a form that would be consistent with the idea of Stereotype threat. However, there are other reasons why it is also doubtful in the case of race. For more exploration on why conclusions drawn from stereotype threat studies are doubtful for methodological reasons, I recommend this paper by law professor Amy Wax: Stereotype Threat: A case of overclaim syndrome? (I have a special love for this paper because insisting on using it in the sex and intelligence wikipedia page years ago is what brought down a flock of feminist harpies who eventually got not only the paper, but also my user account banned from wikipedia.) Seems like Wax really likes sticking her neck out and fighting the good fight.
At best, stereotype threat is something that exists and has only a very small effect and at worst it is an example of publication bias amongst journals where positive results that support politically progressive ideas (like discrimination against women) are overwhelmingly published relative to studies that don’t confirm progressive beliefs or which might positively refute progressive beliefs.
Diederick Stapel was previously a highly regarded and influential Dutch social psychologist who did a lot of work on stereotype threat, among other things, until it came to light that he “routinely falsified data and made up entire experiments.” Another example of his politically biased work was a “scientific” article which sanctimoniously claimed to find that meat eaters were more selfish and less agreeable than vegans. Unfortunately, it is impossible to be surprised by outspoken priggishness from vegans. Thanks to this media attention, Stapel is now the most notorious charlatan in the field of social psychology, which is saying a lot for what appears to be a regularly fraudulent and pseudo-scientific discipline. Social Psychologists as a group do not make the data they collect available for outside review 2/3rds of the time. This stinginess with data is actually against the ethical rules established by Social psychologists themselves and suggests that there are likely many more Stapels out their who simply haven’t been caught. A survey by the Harvard business school found that 70% of social psychologists admitted to cutting corners in reporting data, 30% reporting unexpected findings as if they were expected from the start, and 1% admitted to falsifying data. Another meta-analysis of papers published in high-tier psychology journals found that 50% of papers surveyed contained at least one statistical error and 15% contained an error so severe that the conclusion drawn would have had to have been reversed. Yet another meta-analysis which looked at whether or not positive results from stereotype threat studies could be replicated found that almost half could not, and that a further 25% were confounded by methodological issues. A substantial majority of the findings were unreliable.1,2,3
Bias is rampant in the humanities, but especially in social psychology, both among individual researchers and among the journals publishing papers. Beyond the objective critical evaluation of papers, the field itself is essentially an ideological and political echo-chamber that is considerably more left-wing politically than the general population. 80% of social psychologists identify as liberal, while only 3 out of 1000 identify as conservative. Contrast this with the general population which is 40% conservative and only 20% liberal. Were these sorts of numbers occurring with a protected class, these same people wouldn’t hesitate to use it as incontrovertible proof of discrimination. Considering what is now known about the biological origins of cognition and intelligence, it is generally difficult to take claims of discrimination seriously when groups also display a relatively lower intelligence profile. However, in this case there is no reason to think that conservatives as a group have an intellectual profile below the general population. Social conservatives tend to be a little lower in intelligence relative to liberals, but free-market conservatives (libertarians) tend to be smarter. Being very partisan, either liberal or conservative, tended to be associated with high IQ as well. Increased income levels, which are a proxy for IQ, also moves people right ideologically. In other words, there is nothing that differentials in biologically determined intelligence can do to explain the lack of conservatives, and even moderates, in the humanities.4,5,6,7 Presumably academia wasn’t always so partisan, and thus its current state is a classic case of successful entryism.
In a survey of social psychologists, it was found that conservative responders feared negative consequences from revealing their political affiliation and that they were right to do so as liberal responders expressed willingness to discriminate against conservatives in approving papers, grant proposals, and hiring decisions. The more liberal a social psychologist is or the more consequential the decision would be for the conservative, the more willing liberal social psychologists are to discriminate. That willingness to discriminate against (or for) articles and proposals for ideological reasons has been empirically confirmed in several instances. In one study, reviewers were sent a manuscript which purported to show the mental health of a group of leftist political activists compared to a control group. Reviewers who were sent a version which showed that the activists had better mental health consistently felt that the paper was more publishable and even felt that the statistics were more adequate than reviewers sent a version that showed the activists had lower mental health. In another case, a research proposal which either wanted to study discrimination or reverse-discrimination was sent to 150 review boards (where reverse-discrimination is a euphemism for anti-white european discrimination). The proposal on discrimination was approved twice as often as the proposal on reverse-discrimination. In college admissions, it was found that reviewers would attach greater value to the criteria (grades vs. test scores) which would allow them to pick the candidates with similar partisan politics. Lastly, controlling for research productivity and academic achievement, another study found that conservative researchers were working at lower quality institutions relative to equivalent liberal colleagues than would be expected. The irony that a group which commonly publishes on the asserted negative consequences of discrimination would prove to itself be extraordinarily discriminatory is stunning.8,9,10,11The pattern of ideologically driven academics significantly undermines the ability of an objective outsider to trust the conclusions coming out of certain fields, especially when it is related to such a politically charged subject as gender (and race) differences in test scores. It is quite clear that the overwhelming majority of researchers working on this topic possess a politically desired outcome of these studies. The great potential for this systemic Lysenkoism to motivate the production of inaccurate results which are contrary to reality can’t be overestimated. The objectivity of the field concluding stereotype threat is a real and large effect phenomenon is highly questionable. Calling this cynical skepticism “anti-intellectual,” a common criticism of conservative thinkers, is only so in the sense that these “scientists” have mis-defined the word “intellectual” to describe their political ideology and therefore themselves. Like most things on the right the “anti-science” feeling exposed by some is just a reaction to leftist entryism in academia and the dominance of pseudo-scientific articles surrounding politically partisan topics.
- Can stereotype threat explain the gender gap in mathematics performance and achievement? Stoet, Gijsbert; Geary, David C. Review of General Psychology, Vol 16(1), Mar 2012, 93-102. http://dx.doi.org/10.1037/a0026617
- The (mis)reporting of statistical results in psychology journals. Marjan Bakker and Jelte Wicherts. 2011. Behavior Research methods.
- Psychology rife with inaccurate research findings. Psychology today. 2011
- Jonathon Haidt’s post-partisan psychology page.
- Social scientist sees bias within. New York Times. 2011
- Is there a relationship between political orientation and cognitive ability? A test of three hypotheses in two studies. Markus Kemmelmeier. 2008
- Income and Ideology: How personality traits, cognitive abilities, and education shape political attitudes. Rebecca Morton. Jean-Robet Tyran. Erik Wengstrom.
- Political Diversity in social and personality psychology. Yoel Inbar, Joris Lammers. 2012
- Publish or politic: Referee bias in Manuscript review. Stephen Abramowitz, Beverly Gomes, Christine Abramowitz. 1975
- Human subjects review, personal values, and the regulation of social science research. Ceci, Peters, Plotkin 1985.
- Political Partisan Prejeduce: Selective Distortion and weighting of Evaluative categories in college admission applications. Munro, Lasane, Leary.
14 Replies to “Stereotype threat and pseudo-scientists”
Great post! I love how you not only argue clearly, but include all the references.
> a group which commonly publishes on the asserted negative consequences of discrimination would prove to itself be extraordinarily discriminatory is stunning.
If you spent any time with progressives, this is not surprising at all. For them open-minded, intelligent and ‘agrees with me’ are synonyms. Many meme-complexes (religions) include the superiority of adherents as one of the memes. This is just the version for the progressive religion.
This is a must-read: http://isteve.blogspot.com/2012/10/john-list-on-virtual-nonexistence-of.html
“So we designed the experiment to test that, and we found that we could not even induce stereotype threat. We did everything we could to try to get it. We announced to them, “Women do not perform as well as men on this test and we want you now to put your gender on the top of the test.” And other social scientists would say, that’s crazy — if you do that, you will get stereotype threat every time. But we still didn’t get it.”
So in this interview John List admits the extreme dubiousness of the idea of stereotype threat — and yet if you look at the actual paper describing his experiment, you wouldn’t know it. Evidently he thought it had a better chance of being published if he spun it as a modest refinement of the body of knowledge surrounding the concept rather than as a challenge to it.
Not only is there a massive publication bias in favor of work with left-leaning implications, but many of the academics who do publish such work don’t even believe it themselves.
Update your priors accordingly.
This was a great comment, sorry I didn’t notice it when it came out. Stereotype threat is pretty clearly bogus and even the people who came up with it acknowledge it only accounts for very little, if any, of group differences.
What are your thoughts on this 20-year review?
Empirical support was accrued for mediators such as anxiety, negative thinking, and mind-wandering, which are suggested to co-opt working memory resources under stereotype threat. Other research points to the assertion that stereotype threatened individuals may be motivated to disconfirm negative stereotypes, which can have a paradoxical effect of hampering performance. However, stereotype threat appears to affect diverse social groups in different ways, with no one mediator providing unequivocal empirical support. Underpinned by the multi-threat framework, the discussion postulates that different forms of stereotype threat may be mediated by distinct mechanisms.
Well, it seems to agree with my impression that it is somewhere between a negligible effect and a completely made up finding that only exists because of publishing bias. Clearly this concept is violating the “law” of parsimony if unique and varied conditions are required for many different cases.
Especially when genetic differences much better explain known gaps between groups. I believe there is a quote from the guy who invented this, last name steele, where he clearly states that this effect, if actually real, can not account for the actual observed gaps. Only the very small difference that is sometimes found after already controlling for IQ.
Anyway, thanks for the link.