Personality Tests > Even Popular Tests are Controversial

One of the most popular personality tests in the world is the Myers-Briggs Type Indicator (MBTI), a psychological-assessment system based on the work of psychologist Carl Jung. Two and a half million Americans a year take the Myers-Briggs. Eighty-nine companies out of the US Fortune 100 make use of it, for recruitment and selection or to help employees understand themselves or their co-workers.

The MBTI asks the candidate to answer a series of ‘forced-choice’ questions, where one choice identifies you as belonging to one of four paired traits. The basic test takes twenty minutes, and at the end you are presented with a precise, multi-dimensional summary of your personality. The MBTI test classifies people into types based on 4 bi-polar dimensions;

Extraversion-Introversion (E-I)
Distinguishes a preference for focusing attention on, and drawing energy from, the outer world of people and things versus the inner world of ideas and impressions.

Sensing-INtuition (S-N)
Distinguishes a preference for gathering data directly through the senses as facts, details, and precedents (Sensing) versus indirectly as relationships, patterns, and possibilities (INtuition).

Thinking-Feeling (T-F)
Distinguishes a preference for deciding via objective, impersonal logic (Thinking) versus subjective, person-centered values (Feeling).

Judging-Perceiving (J-P)  
Distinguishes an outward preference for having things planned and organized (Judging) versus a flexible style based more on staying open to options than deciding (Perceiving).

The sixteen personality types resulting from the cross-products of these four dimensions are illustrated below.

As you can see, there are 16 distinct personality types, so someone may be classed as ESFP or INTJ, or some other combination. This is obviously a different way of looking at personality from the big 5 personality trait theory of Costa & McCrae.

Psychologists judge the worth of any personality test by two basic criteria: validity and reliability. Validity indicates that a test measures what it says it measures and reliability indicates that a test delivers consistent results.

Validity of MBTI
The validity of a test estimates how well the test measures what it purports to measure. There are two types of validity that should be considered:

1. Construct validity - does the MBTI relate to other scales measuring similar concepts?
2. Criterion-related validity - does the MBTI predict specific outcomes related to interpersonal relations or job performance? 

The National Academy of Sciences committee reviewed data from over 20 MBTI research studies and concluded that only the Intraversion-Extroversion scale has adequate construct validity. That is high correlations with comparable scales of other tests and low correlations with tests designed to assess different concepts. In contrast, the S-N and T-F scales show relatively weak validity. No mention was made in this review about the J-P scale.

Overall, the review committee concluded that the MBTI has not demonstrated adequate validity although its popularity and use has been steadily increasing. The National Academy of Sciences review committee concluded that: ‘at this time, there is not sufficient, well-designed research to justify the use of the MBTI in career counseling programs’, the very thing that it is most often used for.

Reliability of MBTI
Reliability is the degree of consistency with which a test measures what it is said to measure. Test length greatly affects reliability with longer tests tending to be more reliable. Reliability can be measured using reliability coefficients, and for short personality tests these should be in the range 0.70 to 0.80. The MBTI reports reliability coefficients for its four scales on general population samples in the ranges from 0.61 to 0.87.

The practical effect of this is that even though the MBTI claims to reveal a subjects’ inborn, unchanging personality type, as many as 75% of test takers are assigned a different type when they take the Myers-Briggs a second time.

Academic psychologists and commercial test providers have a tendency to put a different ‘spin’ on how valid and reliable these personality questionnaires are, with the test providers unsurprisingly ‘talking up’ both validity and reliability.

The following quotes are from David M. Boje, Ph.D., Professor of Management in the Management Department, CBAE at New Mexico State University (NMSU).

“…do not treat the archetype scores of M-B as anything more than Astrology”

“The test is not valid or legal to use for personnel assignments, hiring, or promotion.  It does not have predictive validity for such uses. It is a useful guide, and no more. Problem is, people go to a workshop, get excited and treat M-B as a secret window into the mind of their co-workers.”

Robert Spillane, Professor of Management at the Graduate School of Management at Macquarie University argues that research shows that efforts to predict performance from personality and motivation tests have been consistently and spectacularly unsuccessful.

"[Tests] trivialize human behavior by assuming that (fake) attitudes predict performance. Not only is this incorrect but testers offer no explanations for behavior beyond the circular proposition that behavior is caused by traits which are inferred from behavior,".

"The technical deficiencies of most personality tests have been known for many years. Yet they are conveniently ignored by those with vested interests in their continued use,"

You can easily find hundreds of quotes like these, in which noted and published psychologists call into question the use of personality tests. However, judging from the number of and increase in personality tests used, they are likely to be part of the selection process for the foreseeable future. As someone taking one of these tests you just have to hope that the HR professionals who have selected the test have realistic expectations of validity and reliability and have been trained to interpret the results properly.