Quantifying how useful the Myers-Briggs types are for description

The Myers-Briggs personality types is the most popular system of personality types. You can learn the details of the theory in the interactive version I developed.

The developers of the Myers-Briggs Type Indicator and its large number of proponents in online forums claim that Myers-Briggs/Jungian theory ("the cognitive functions") is the true scientific explination of personality. Unfortunately, they are wrong and the theory is not valid. It is not true, it was just made up.

However, while it is not valid as a scientific explination, the 16 types can be used descriptively with at least some validity and this has caused a lot of confusion in debates over if the Myers-Briggs types are good. A common question people ask is "how can you say the MBTI is not legit when I read my result and thought it was accurate???". The answer is that the Myers-Briggs types can be used validly as adjectives to a non-zero degree, even if Myers-Briggs type theory is fake.

This is actually pretty hard to explain, and many people critcizing the MBTI get this wrong. Vox has an article titled "Why the Myers-Briggs test is totally meaningless", while this article has many good arguments against the MBTI I do concede its title is incorrect. The problem with idea of Myers-Briggs types is not that they are 100% meaningless or incoherent. An good analogy I think is to fictional characters. The TV show Sex And The City is a made up story about people who are not real and things that did not happen. However, categorizing people as one of the four characters has some descriptive power that is not literally zero. It would be wrong to say that the idea of being like Carrie Bradshaw is "meaningless". But it would also be wrong to say that the idea of Carrie Bradshaw not being meaningless is evidence that Sex and the City is the true theory of personality.

Some people do get this and have made the argument that while the Myers-Briggs types may not bue "true" they could still be "useful". My favorite blog, Slate Star Codex, has made this exact argument:

And this seems reasonble. Can't we just acknowledged that the theory behind Myers-Briggs is wrong, but at the same time still used the categories it inspired when they seem to capture meaning we want to communicate?

My answer is that, in addition to Myers-Briggs theory not being scientifically true, categorizing people by the 16 types (using them descriptively) is not even very useful. And I have what I think is a very clever analysis to demonstrate this. I will compare the descriptive power of using the Myers-Briggs types versus the descriptive power of using the full set of English adjectives.

A method for quantifying descriptive power

The basic principle of this analysis is an extremely general one: Things that are the same should have the same representation, things that are different should have different representations.

We will look at this in several different ways. The first is with fictional characters. Labeling characters with Myers-Briggs types is one of the major uses of Myers-Briggs types on the internet, and the general consensus of a lot of people is that this is pretty meaningful.

So, the principle impies that two characters described with the same Myers-Briggs type should be rated more similar than two characters described with independant types.

We can get estimates of how similar to characters are seen as just by surveying people about it. The survey looked like this:

I had more than 100 thousand people take this survey and in the end I had 3,112 pair-wise comparisons between two characters where at least 50 different people had rated their similarity. I'm will only use relationships that had at least 50 ratings to minimze measurement error in this analysis.

The table below shows the most extreme comparisons at each end.

So, now that we have community verdicts on how similar characters are we can try to predict them with Myers-Briggs types. The Myers-Briggs types we will take from personality-database.com, a website where people vote on what types fictional characters are. Myers-Briggs types can be exact matches (the same type), or partial matches. We will use the number of letters that are the same as our measure of type similarity, which will then range from 0 to 4. Some examples of this calculation are in the table below.

Now we can graph rated similarity versus similarity in Myers-Briggs types for all the pairs of characters to see if there is a correlation. The graph below shows this (with some spreading of the points post facto for legibility).

And we see that using Myers-Briggs types we can successfully categorize fictional characters by what they seem like to some degree. Characters that have been voted as belong to the same type, were more likely to be rated as being similar than characters that different Myers-Briggs types. The R squared value for the correlation is 0.057 meaning we can predict 5.7% of the variance in rated similarity with Myers-Briggs information.

This value seems pretty low... but intepreting effect sizes is hard. What is the cutoff for when a descriptive model goes from good to bad? I don't know if there is one. However, we can compare between two models and see how they perform relative to each other.

The model we will compare to is just "adjectives in general". In my character personality quiz I model each character as a vector of numbers respresenting their ratings for a collection of different adjectives. I can estimate a similairity score for two characters just by correlating their vectors. And then compare these estimates to the rated similairty values. The graph below shows this:

The R² for predicting similarity using my descriptive model is 0.58, or 58% percent of the variance predicted. My model absolutely blows the Myers-Briggs types out of the water. My model is more than 10x times as good the Myers-Briggs types. And my model is not that advanced or insightful a model of personality. I just selected some English language adjectives from the dictionary. If you choose to describe a character using only a Myers-Briggs type, you are giving up at least 90% of the power to describe characters that you have already just as a speaker of English.

A replication of the analysis with a different dataset

My first analysis pretty much debunks the idea that categorizing fictional characters by Myers-Briggs types is particularly useful. It could be argued that describing characters does not matter and describing real people is a totally different thing. I don't think this is a very good argument as humans seems to use the same mental machinery to evaluate fictional characters as real people and the average Myers-Briggs typology fan seem to find the idea of describing characters at least as compelling as identifing as a type themselves. Typing characters is an extremely popular use of the Myers-Briggs types (Personality-Database.com was the #4,032 most popular website in the world in 2022, while the offical MyersBriggs.org came in at only #118,525). However, I will conceded that describing fictional characters and describing real people can't be the exact same problem, so a replication with real people is in order.

I led with the analysis on fictional characters because the data is very reliable there. The Myers-Briggs types have usually received hundreds of votes on Personality-Databse.com. My character model has been refined so I think its pretty close to the maximum possible performance you could get. To get a dataset as good as the data I have for fictional characters would be very hard. e.g. recruiting pairs of real people and then recruiting 50 people who knew both of them to answer surveys about how similar they are is not feasible.

The best data I have the includes the three neccesary components for two people (Myers-Briggs types, adjectives, and similarity between pair) is from my personality test for couples. In this test, the couples complete an adjective rating survey for both members of the couple. At the end, they answer some demographic questions, which included self-identified Myers-Briggs types and pairwise similarity measured with this question:

I got complete data for 1,752 couples. Just like we did with fictional characters, we can compare predicting pairwise similarity with both Myers-Briggs types and an adjectives based approach to see which one does better. The performance of both approaches is in the table below.

Again, the adjective based approach significantly out performs the Myers-Briggs types. Only 4x times better instead of 10x times better this time, though I think the difference would continue to grow if I had a better data set. I just shoehorned the similarity and Myers-Briggs types question into the Couples Personality Quiz which was not designed for this question.