Are Personality Tests Just Horoscopes for White-Collar Professionals? (Part 1)
We designed a single-blind experiment (with scientific controls) for the MBTI, Enneagram, and DISC Assessment. Is your leadership training and pre-employment test any better than corporate astrology?
Decades ago, one of the authors of this essay ritualistically read the daily horoscope in the newspaper. Being a gullible fool, I didn’t just read for funsies; I actually followed the horoscope advice. Over time, a part of me started believing in astrology.
The punchline to this confession is how it all ended. I only stopped believing in astrology when I discovered I had been reading the wrong horoscope all along1. I accidentally realized my error on a day I read the horoscope, followed its advice, achieved great results, then went back later to save the newspaper clipping as a memento. That’s when I discovered:
I had achieved good outcomes by following advice written for someone else’s astrological sign.
The correct horoscope for my astrological sign provided the opposite advice. If I been a little smarter and followed the advice intended for me, I would’ve achieved crappier outcomes.
That’s when I reached the rock bottom of intelligence. In an ironic misalignment of the stars, I screwed up so badly at the practice of astrology that I successfully falsified (disproved) its legitimacy. The idea of falsifiability isn’t new; the philosopher of science Karl Popper came up with the idea almost a century ago. Whereas Popper arrived at the idea through intellect, I failed my way into it through brain-numbing stupidity.
Fortunately, my intellectual decay halted and reversed course before I could transform into a troglodyte or Scientologist. That’s the silver lining to reaching rock bottom: the only way to go is ↑ up ↑.
Today, I no longer believe in astrology. Instead, I have to contend with something remarkably similar to horoscopes: psychometric tests. Personality tests are so widely used in leadership training regimens and pre-employment screenings that an entire wing of the Institute of Conventional Wisdom is devoted to it.
The Backassward Approach to Falsification
Have you ever noticed that horoscopes and personality tests use a similar method to prognosticate the future?
Horoscopes: Ask for birth month/year → assign a label (Libra, Year of the Dragon, etc.) → dictate instructions/analyses for that label.
Personality tests: Ask introspective questions → assign a label (ENTP, Achiever, etc.) → dictate instructions/analyses for that label.
When I accidentally falsified astrology during my blunder years, I did it by approaching the prognostication from the wrong direction: I followed the instructions/analyses for the (wrong) label while identifying with a different label. It’s like you offered me a birthday cake, left the room, and returned to discover that I had mistakenly eaten someone else’s birthday cake…using the wrong end of my gastrointestinal tract. Puts a new twist on the term “ass-backwards,” doesn’t it?
If I could consume the wrong cake with the wrong end of my GI tract for months without any negative consequences, what does that say about the usefulness of astrology? If it doesn’t matter what horoscope I follow, then astrological predictions are no better than rolling the dice. Or flipping a coin2.
Compare and contrast astrology with systems that are intolerant of mistakes: if a pianist bangs randomly on the keyboard, you hear a cacophony instead of music. If a surgeon uses a random cutting tool on an arbitrary part of the body, the patient will die. If a pilot randomly pushes buttons and pulls levers in the cockpit, he will die along with a flying metal tube full of wailing passengers.
So if astrology is no better at predicting the future than a dart-throwing monkey, why do so many people believe in it? Hell, why did I believe in it?! After years of skeptical inquiry, I have a partial answer:
Horoscopes are vague enough to relate to most readers. Even if a horoscope doesn’t precisely apply to today’s circumstances, most people can recall a relevant experience that kinda-sorta fits.
The prognostication method is designed to focus reader attention on a single horoscope. If presented with a single option that sounds kinda-sorta legit, it only takes a few daily horoscopes to hook gullible fools (like me) into believing astrology has predictive power.
Horoscopes are like the cheap birthday cakes sold at discount grocery stores: made of 100% feel-good sugar and devoid of any nutritional value. Do personality exams used for employment and leadership trainings have any nutritional value?
Let’s Take Psychometric Tests Backassward
We’re going to approach the MBTI, Enneagram test, and DISC assessment from the wrong direction. We’ll show you personality descriptions (i.e. the test results) from all three assessments in a completely random order. Your job is to evaluate which descriptions match your own personality, and which ones sound nothing like you. Identifying with multiple personality descriptions is fine; it doesn’t mean you have dissociative identity disorder, but it might qualify you to run for political office!
To make things interesting, and to give this exercise some scientific validity:
This will be a single-blind experiment. You won’t find out which test is which until next week.
We included two astrological systems as scientific controls. You won’t find out which of the tests were psychometric assessments and which ones were horoscopes (the controls) until next week.
To obfuscate the origin of each test, we will provide 12 descriptions per test. That means the Myers-Briggs test will omit four personality descriptions selected at random, trimming 16 down to 12. The Enneagram of Personality and DISC Assessment will use repeats (also selected at random and paraphrased) to bring the total personality descriptions up to 12. The astrological systems have 12 zodiac signs already, so those will simply list 12 horoscopes in random order.
Your affinity with the astrological signs will serve as a baseline for comparison against the personality tests. If you identify with fewer legitimate personality descriptions than with a random selection of astrological signs, then the psychometric tests might have more predictive power than astrology. If you identify with as many or more personality types than with horoscopes, the personality test is probably equivalent to (or worse than!) corporate astrology.
Ready? Let’s get started.
Psychological Test 1
Psychological Test 2
Psychological Test 3
Psychological Test 4
Psychological Test 5
Come Back Next Week for the Results!
The gold standard in clinical research is the randomized control trial. Our single-blind and controlled experiment is nowhere close to reaching that standard, but it’s reasonably high-effort for something you’d find on a random internet blog. What’s more, falsification (proving what is not) needn’t be as robust as confirmation (proving what is). When you see a single black swan, you refute a million observations that all swans are white.
Likewise, the purveyors of personality tests can provide a million testimonials and research articles to sell their products. When we reveal the origins of each mystery test next week, you’ll find out if this single experiment was enough to refute a mountain of endorsements.
In the meantime, we recommend you take a free version of each psychometric test so you’ll know what personality types you are during next week’s big reveal. The Open-Source Psychometrics Project lacks the polish of a professional testing service, but their paywall-free personality tests get the job done:
Open Extended Jungian Type Scales (open-source equivalent of the Myers-Briggs Type Indicator)
See you next Friday!
Back in those days, newspapers were printed on actual paper. Blocks of newsprint would change size every day; they look nothing like the orderly webpages today containing a grid of headlines and “Read More »” links to the actual news articles. My old newspaper was formatted ambiguously, so it wasn’t always clear which astrological sign (Gemini, Sagittarius, Scorpio, etc.) was associated with a particular horoscope. Some days I correctly read the horoscope for my astrological sign, and other days I inadvertently read the horoscope for another (wrong) sign.
Not the proudest time of my life.
Caveat: this applies to astrology but not to low-probability, high-impact events. You can engage in Russian Roulette-like activities (e.g. junk bond trading, BASE jumping, lion taming, autoerotic asphyxiation) for years without any negative consequences…but when those consequences arrive, you can end up in the Career Swamp or snuff out your own life. We covered this at length in Risk and Uncertainty in Leadership: 2-Dimensional Thinking and its sequel, 3-Dimensional Thinking.
This is an interesting topic for me and I'm looking forward to part 2. Having worked in the corporate world I've taken all of the above..., some I agree with, some I flat out don't like.
I first became a manager in 1995. While the company I worked at didn't provide any formal management training I had studied leadership and management on my own. I thought I knew what I was doing. My management style was based on the golden rule. I managed people how I would like to be managed. Seemed like a great idea.
At the time I took on the management role I was taking a creative writing course for fun. Early on the company that hosted the course asked students to take a personality survey to help the company fine tune their material and approach to the students. I answered 72 questions and sent in my survey.
After a month in my new role I received my personality survey report. The survey used the Kiersey Temperament Sorter and the sixteen MTBI personality types. I read the three page report and, man, the report nailed me. It described how I thought, how I liked to work, my approach to doing things. I was stunned how 72 questions could so superbly describe me. Then I got to the last page where I learned that my personality type represented 3-5% of the general population.
I was running a department of 20 people. I had been managing them how I would like to be managed. My specific personality type represents 3-5% of the population. There might be one other person in the department of twenty who had the same personality type as me. Yet, I was managing the group like it was a whole bunch of me's. I knew I had to change fast.
Luckily, I came across the Platinum Rule: treat others as they would like to be treated. Versus using one management style for everyone I adapted my approach to each member of our department. My job got a lot harder, but I was a better manager for it.