Dissecting the DISC

What test instrument can quickly assess a candidate’s personality preferences; is cheap; available to almost everyone; marketed by dozens of vendors under a variety of names; and, is recommended “unreliable and untrustworthy” by most testing professionals? Yes, there are others, but I was referring to the DISC.

(Although this article focused on the DISC, you could apply its comments to any test, assessment, interview, exercise, role-play, and so forth used to separate qualified from unqualified applicants.)

Some Background

DISC development began in the early 1900s when the Army asked psychologist William Marston to investigate why different soldiers who received the same training behaved differently. He published a report about 10 years later entitled “Emotions of Normal People.”

As far as we know, Marston’s objective was to describe “mental energy”… not assess and classify people for a job. Shortly afterward, another psychologist used Marston’s theory to develop a pencil-and-paper test. It asked people to choose between pairs of adjectives (i.e., which adjective is most like you and which adjective is least like you); then it added items together and reported scores for dominance, extraversion, need for security, and need for structure. (Remember that Marston was NOT trying to hire the most qualified people for a job — just explain normal behavior.)

So far, so good.

After that, the DISC grew in popularity and became another chapter in misused pseudoscience. Vendors and users alike tested everything making wild predictions about future performance. I have not verified it, but there were even some rumors about separate DISC profiles developed for inanimate objects, barnyard livestock, and certain vegetables. What happened? In short, as one great philosopher said, “When the only tool you know how to use is hammer, every problem looks like a nail.” That is, to the uninformed DISC user, every hiring problem looked like a DISC profile.

Test Construction

There are four things that define a good hiring test (i.e., one with scores you can use and trust):

  1. It is well-constructed (e.g., all individual items supposed to measure expressiveness, actually measure expressiveness … and not dominance, security, or structure). This is termed inter-item reliability.
  2. It delivers the same results over time (e.g., for the same person, November 2008 scores are the same as November 2007). This is termed test-retest reliability.
  3. Scores predict performance (e.g., all high performers have similar profiles and all low performers have similar profiles). This is termed criterion validity.
  4. It incorporates a theory of job performance (e.g., you are trying to predict job performance, right?). This is termed knowing your butt from your elbow.

Just common sense, right?

Overall, the idea of checkingoff descriptive adjectives seems like a good way to demonstrate preferences, but this design has some serious limitations. Consider, for example, the following items taken from one version of the DISC: Fussy, Self-reliant, Persistent, Optimistic, God-fearing, Devout, and Moderate. Can you clearly identify to which of the four factors they relate? Neither can I. Fuzzy items lead to fuzzy results.

In another case, a DISC vendor proudly published on the Web a study it claimed proved the VALIDITY of its DISC (i.e., if you recall, validity proves test scores actually correlate with job performance). Unfortunately, the vendor must have been disoriented because he only reported inter-item reliability — not validity — leaving the reader wondering if the vendor either successfully completed a class in psychometrics or read the Standards for Educational and Psychological Testing (both are requirements for test professionals).

Article Continues Below

Personality Scores and Job Performance

Are scores on personality tests highly predictive of performance? There is a long history of research showing: maybe. That is, if the employee has all the other performance elements necessary to do the job, and the personality test is job-related, personality can make a difference. Self-descriptive test scores represent how the applicant wants to present themselves — it may not be reality. So, even if an applicant tells an interviewer he/she is organized, it’s no guarantee he or she will be good with details. It is just bad science to claim the DISC (or any other personality test) will accurately predict managerial performance, capability for organization, character, or personal responsibility.

What about the four DISC factors? Remember that Marston never intended the DISC be used for employee selection? Decades of personality research shows only factors consistently relate to job performance: being extraverted, not being neurotic, and being conscientious. Skills-based research, on the other hand, shows that intelligence is the greatest predictor of ability (e.g., people who are job-smart outperform those who are not). I don’t know about you, but it seems the DISC falls short on both. But, if you have been using the DISC as a hiring tool for some time, and rigorously monitor your results, you probably already know that.

What about the most-least scoring system? Well, that leaves something to be desired too. You see, if there are eight expressive adjectives and I check-off four … but you check-off a different set of four expressive adjectives … have we taken the same test?

The same goes for comparing expressive adjectives with ones for dominance, security, and structure. Ipsative scoring (as it is termed) tells us a great deal about individual preferences, but it is generally discouraged for hiring by test professionals.

So What Good Is the DISC?

If you need some help understanding broad differences between people, and you are willing to take a test never intended for serious business use, then the DISC is a fun tool. If you understand that when you describe yourself as primarily dominant (or any other factor), your scores will probably show you are dominant (amazing!); or, when you want to facilitate a quickie workshop in communication styles, the DISC is for you.

Always remember, the vast majority of personality tests were designed to evaluate differences between people — not evaluate people for jobs.

Topics

6 Comments on “Dissecting the DISC

  1. Wendell, once again a solid look at squashy practices. There are two factors at play here: Publisher/vendor claims/assertions (or lack thereof) and uneducated or undereducated users.

    In my role as a Staffing Expert Panelist for SHRM, I went around from booth to booth at a SHRM and the Staffing Management Conference asking assessment/testing exhibitors a few basic questions about their assessments:
    How do you help me comply with the Uniform Guideline on Employee Selection Procedures?
    Can you show me technical report summaries on various validation analyses where the assessment has been used?
    What descriptive details do you have on the criterion?
    Can you tell me about the various norm groups you have in your data base?

    In most cases, there were no answers, poor answers, or in one case a wild answer: “Well we did not use inmates like one our competitors!.” There were also some exceptionally well prepared people, but few and far between. As in any practice area, variation exists, so: Buyer Beware.

    My conclusion was this: Assessment suppliers (publishers or resellers) have low expectations of their customers to ask well grounded questions. As such, the suppliers staff their conference booths with individuals ill-equipped to address best practices and sound psychometric principles.

    At one of my recent presentations on objective candidate evaluation methods I asked two questions along these lines:
    Are you using some form of tests or assessments?
    Have you taken a college level course on tests and measures?

    The majority of hands went up for the former and a mere smattering of hands remained up for the later.

    Any recruiter or staffing professional who is using tests and assessment should give serious consideration to completing one or more college level courses covering tests, testing methods, assessment practices, etc. Explore your local or on-line university for one offering a degree in Industrial-Organizational Psychology. This is the professional body of knowledge regarding the measurement of people in the world of work.

    An educated consumer is the best customer.

    Joseph P. Murphy
    Shaker Consulting Group
    Developers of Virtual Job Tryout®

  2. This may be too basic for the people interested in testing, but I was not sure what DISC stood for so I looked it up on Wikipedia.

    Dominance – relating to control, power and assertiveness

    Influence – relating to social situations and communication

    Steadiness – relating to patience, persistence, and thoughtfulness

    Conscientiousness – relating to structure and organization

  3. Nice work Wendell. I couldnt agree more with what you are saying. I can’t overstress the need for due dilligence and understanding of the basics foundations of good hiring (understand what outcomes are of value, use proven and accurate measures of those outcomes, and evaluate the effectiveness of the measures within your hiring process). I also want to stress that the ever popular Myers Briggs is a DISC type tool. See the article I wrote a few years back for my opinion on that one

    https://staging.ere.net/2005/04/21/alphabet-soup-a-cautionary-note-about-using-myers-briggs/

    The bottom line- Be careful out there! there is a lot of second rate junk out there

  4. I have been asked by a reader to comment on the AVA (Activity Vector Analysis)…an instrument similar to the DISC in design.

    Adjective check lists in general are all self-reports…You have to assume the subject has great self-insight and is being absolutely truthful. Self-reports are hard to verify.

    Also, does it make sense that checking off a few word pairs can generate huge reports about public, private and projected persona? Maybe we should just forget the whole psychologist licensing thingy and give everyone a checklist to complete?

    Check lists are repetitive (the technical term is “ipsative”). That is they compare this word with that word. As a result, final scores may look the same, but be based on entirely different responses…It is like looking at different tests with the same score.

    Personality, in general, does not mean someone has skills to back it up. About 40 years ago, psychologists blew-off personality tests as parlor games because there was conflicting data proving they predicted performance. Since then we have learned that personality is only one part of the performance puzzle.

    Personality CAN make a dfference between two equally-skilled people…but taken alone, personality only contributes a small part. Intelligence, interpersonal skills and organizational skills are much more important.

    I used the DISC once to compare performance among stockbrokers…guess what? Successful stockbrokers were pushy risk takers. (I didn’t need a test to tell me that…I trained stockbrokers for several years –if you were not a pushy risk taker, you failed!)

    But my subject group was small, all had completed four intensive months of training, completed several difficult tests, and survived the first critical year of selling…You tell me: did this mean the DISC scores predicted success or did they just confirm those who were successful?

    Unfortunately, the DISC had absolutely no correlation with relationship salespeople.

    Finally, as Charles and Joseph said….ask the hard questions…If the vendor cannot answer them, confine the test to the workshop. Always remember: You cannot teach a pig to sing. For one thing, pigs can’t sing. For another, it ticks-off the pig.

  5. I cannot find any validation reports for DiSC which include criterion validity. Is anyone aware of any such research? The validation report released by DiSC, as mentioned above, does not contain criterion validity studies and has no indicators of item correlation with job performance. At a loss for conclusions on what current research says about DiSC from a criterion validity stand point.

Leave a Comment

Your email address will not be published. Required fields are marked *