Promises, Promises: How to Identify a Bad Hiring Test (Part II of II)

eeocIn Part One I explained why a test user has to be exceptionally careful about trusting a vendor’s claim their test is suitable for hiring. Without due diligence on the part of the test user, junk tests lead to hiring too many wrong people and turning away too many right ones, amounting to an estimated 20% and 50% of yearly payroll. In this section, I’ll continue explaining how to identify good hiring tests.

Personality Versus Skill

One would think a score on a personality test would predict a skill (i.e., do high analytical-trait scores actually mean high analytical skills?). Unfortunately for people who rely on personality traits to make hiring decisions, no.

There are several theories about personality. Some people would argue that personal traits are organic brain structures; others argue they represent a description of how we want people to perceive us; and, still others argue they define how we perceive ourselves. I tend to believe people are genetically predetermined to think and act in certain ways which they continuously modify depending on life experiences. No matter what your theory, a personality score is no more or no less than a self-reported opinion that has extremely low correlations with specific job-skills.

Birds of a Feather

One thing self-reported tests do well is identify people who tend to think and behave alike (remember we are not talking about being job-skilled). If you hire people who score high in bleeding-heart tree-hugging, you will probably have a workforce that is so tree-focused it cannot be productive. Likewise, if you only hire people who score high on dominance, you will probably get cold-hearted autocrats who share a survival-of-the-fittest mentality. Hiring people based on personality scores can lead to a workforce that suffers from “group think.” Groups that think and act alike are often unable or unwilling to consider all sides of an issue. The inaccuracy of interviews produces wide performance variances. Adding more-accurate tests without knowing the consequences will narrow your variance between employees, but not in ways you might expect.

Legality, EEOC, and Discrimination

When a test vendor claims his or her test has been approved by the EEOC, you can be sure they do not know what they are talking about. The only time the EEOC “approves” a test is when a user gets audited; and, after thorough investigation, proved they followed Uniform Guidelines and the Standards for testing. The EEOC is not a certifying test body. Vendors who claim their test is EEOC-compliant should be required by law to wear big floppy shoes, rubber noses, and grease paint when they make sales calls.

When a test vendor claims his or her test has no adverse impact, you can be sure the test is probably not doing its job. Let’s use an example to explain how this works. Assume the National Forest Service has discovered people with big feet are 173% more efficient stamping-out forest fires than people with small feet. Incensed at this insult to their heritage, powerful lobbies of small-footed Munchkins converge on Washington claiming the Forest Service treats Munchkins unfairly. They demand legal protection.

Fearful Congressmen, anxious to get the stature-challenged lobbyists off the lawn and back home, pass a law declaring Munchkins a protected group. Is the Forest Service now required to hire all Munchkins regardless of their shoe size? No; they just have to show that big feet are a job requirement and business necessity for firefighters. Big-footed Munchkins will always make the cut, but at the Munchkin group level, there will always be disproportionately fewer Munchkins than other groups. Does the Forest Service have adverse impact? Yes. Are they justified? Yes. Are they guilty of violating any laws? No.

Now suppose a test vendor approaches the Forest Service. They claim their test is a great tool for selecting firefighters because it does not discriminate against Munchkins. Remembering the Forest Service has already shown big feet are 173% more effective at stomping out fires, do you think the test would actually identify the most physically qualified firefighters?

Biology Trumps Logic

A long time ago people lived in small bands. When someone met a stranger it was vital to quickly assess whether they were friend or foe. As you might imagine, people with poor judgment did not live long enough to pass on their clueless-gene. This genetic legacy lives on today every time a recruiter or hiring manager has an exceptionally strong drive to “get to know” a candidate. You often read articles describe it only takes 30 seconds to form a long-lasting opinion or hear managers claim they “know ’em when they see ’em.” These are deep-seated biological drives doing their survival job.

Unfortunately, getting to know someone is not the same as learning whether he or she is job-skilled. Just look at the employees who are hired. Don’t you wonder why only 20% of the salespeople generally produce 80% of the sales? Why good producers almost always make bad managers? Why top employees out-produce bottom ones by at least two-to-one? Or, why people are so adamant about using any other hiring tool other than face-to-face interviews? It’s biology.

Article Continues Below


What are some of the symptoms of a bad hiring system? A shortage of promotable candidates; excessively long training; attempting to fix broken employees by sending them to workshops; not knowing the details of the Guidelines for selection or the Standards for testing; an 80/20 workforce; failing to recognize interviews are weak hiring tools; neglecting biological drivers, or, not having the patience to work through about six candidates to find one who is qualified.

Legal Indemnification

So what if a vendor offers you indemnification from lawsuits? Big deal. Lawsuits are often the least of our worries. Only a handful of hiring challenges ever get to court. Most of them are against big companies and most are settled. Hiring the wrong people and rejecting the right ones is much more expensive than lawyers. Organizations need to worry more about job performance. The cost estimates of poor performance ranges from 10% to 50% of payroll every year.


Here is the problem and the solution as succinctly as I can describe it. You have to be your own watchdog. There are literally hundreds of test vendors promising to solve your hiring problems. You can often screen out the blatantly unqualified ones by giving the vendor a call and asking, “Is your test specifically designed to predict future job performance; and, can you please send me documentation describing the development process and any validation studies you have conducted?”

HR and corporate attorneys, above all, must know the basics of the Guidelines and the Standards. In my experience, corporate attorneys know a great deal about what happens after the fact, but they cannot be expected to know how to use the Guidelines and Standards to minimize the potential for litigation. Organizations that follow the Guidelines and Standards maximize the potential for mounting an effective defense, minimize the potential for an EEOC challenge, and save the organization 20% to 50% of base payroll every year. That’s the highest ROI of any imaginable organizational program. The Guidelines don’t require you to follow them, but they are best practice.

Any system, device, interview, source, or process intended to separate qualified candidates from unqualified ones is a test. You don’t need to worry about lawsuits as much as you do productivity, turnover, and training expense. Be patient. It takes about six candidates before you find one who is qualified. An effective hiring system does not discriminate against skilled people. Skilled people come in every color, ethnicity, gender, age, and level of physical ability.

If you are unfortunate enough to be challenged, your management will probably demand to know you why you did not follow best practices. They might have resisted every time HR insisted on following the Guidelines and Standards, but don’t expect management to forgive you when the EEOC determines your organization did not follow them.


12 Comments on “Promises, Promises: How to Identify a Bad Hiring Test (Part II of II)

  1. Wendell,

    This may be the best 2-part series I have seen on testing.

    Before any test can be valid, I would hope that the basic job definitions and success criteria had been identified and measured.

    Yet, in helping organizations improve their hiring processes, the issues I confront include murky job definitions, vague or way too specific competency requirements, and little to no analysis or data on who has been successful and why.

  2. Wendell,

    I agree with you wholeheartedly. Most of these tests do more harm than good and your ability to provide differentiation to the facts is very beneficial to all Human Capital professionals – especially those who approach their jobs from the standpoint of empowerment and cost.

    Looking Good!

  3. I laughed out loud while reading the point you made about biology getting in the way – some how we’ve passed on far too many “clueless genes” lately…

    Likeability is such a big factor in the new role based work culture that Kevin Wheeler discussed in his piece today – the more collaborative our work environment becomes, the more likeability will become a factor. New assessment tools may be needed to get a sense of organization/presentation, mental horsepower, growth potential and likeability, so that the, “I can feel it my gut” type of assessment is lessened.

    Aren’t these the keys of a successful collaborative “role based” worker?

    As always – thought provoking ideas – thanks!

  4. It’s curious and rather ironic to me that (it’s alleged) *the need for being able to get along with others in the workplace has increased while the need for physically being around others has (and will continue to) decrease.

    Can anyone come up with a clear list of informational/mamagerial tasks that can only/best be done in immediate physical proximity? (Obviously, things like lanscaping, hair styling, construction, repair, etc. would require a physical proximity.)



    *Unless they’re an arrogant, pushy, boss-type. These folks never seem to have to “learn how to play well with others” unless it brings tthe threat of a lawsuit. -kh

  5. Likeability is always a factor, but let me put it this way: Who do you want in your company…a likeable incompetent employee, a likeable middle-of-the-road or a likeable highly skilled employee?

  6. Another point is how to avoid having “likeable” mean “just like me”…. I contracted at a place where diversity meant: “We hire all kinds of mainly white upper-middle class people, just like us!”

    Happy Weekend,


  7. Totally agree. We have a responsibility to create and use things that work and are supported by evidence. Using shoddy tests is not only legally risky and likely to be unproductive, it’s sheer laziness. Good testing is not hard, but it does require some thought and planning.

    A lot of this comes back to taking the entire assessment process seriously. It’s easy to throw stones at imperfect processes, but do too much of this and you’re likely to have a boulder dropped on your head.

  8. As usual, I agree in principle with nearly everything you wrote. Where I disagree, it’s most likely in the way you wrote it (or I read it) and not what you wrote.

    For instance: You wrote, “I tend to believe people are genetically predetermined to think and act in certain ways which they continuously modify depending on life experiences. No matter what your theory, a personality score is no more or no less than a self-reported opinion that has extremely low correlations with specific job-skills.”

    I agree people are genetically wired to think and act in certain ways. You don’t think that affects job performance? An individual who is easily distracted, is short of patience, lacks curiousty, and relies on his/her gut for decisions – you don’t think that affects performance? I believe it predicts how an individual will handle decision making and problem solving if left to their devices AND if they do not deliberately learn skills and are motivated to change their ways. The problem I see isn’t the testing but how people apply it. You’re right it doesn’t predict job skills, but it certainly predicts how they might approach the job if they had the skills.

    Second, “Hiring people based on personality scores can lead to a workforce that suffers from “group think.” Again, the problem isn’t the results of an assessment but how the manager applies them. If they hire everyone with a similar thought process and values system, then yes…they will get group think. But that’s the mistake of the hiring manager not the test.

    And like you’ve written before, the tests are only part of the whole picture.


  9. Of course, everything resides in the brain…KSA’s personality…everything…For your first point, I can tell you from personal experience and my own peer-reviewed studies that high ratings on intellectance are almost always independent of being intelligent. In short, an actual skill level and a self-reported skill level are usually two different things…That’s why personality test scores have such low correlation with performance.

    For your second point, I once knew an HR group that would only pass-along amiable employees for interview. In the course of a few years, everyone was so sensitive they could not get anything done…HR problem? Manager problem? Who knows? My point is selecting everyone with the same personality preferences will problably lead to group think.

  10. Curiosity killed the cat – Satisfaction brought her back.

    The ultimate view is one that reflects propensity to give and take responsibility via choice. As one’s presence or level of truth, with a will of compassion leads to understanding clears a venue to play in an OD model of pure reason. What views we share reflect our formed paradigms from Presence (dynamic), Solids (Static logic’s)which may only serve to eradicate contradiction and emerge 10 more… down to levels of truth called Attention – a basic of self absorbed flows that literally, this paradigm is stuck sucking. Paradigms are forming reality based on investment from stretching, sticking or shrinking the ego’s life energy (dynamis)- the truth does change the facts – not the other way around – unless of course one likes playing victim to genetics or the like.

    At the static truth, one never truly changes or alters yet the viewer’s adopted belief viewed from the ego’s paradigm either shrinks or stretches others views for “group think” sticky drama survival or the instinctual drift of addictions to being right. Since the world of facts is motion – which is constantly persisting from altercation, paradigms change as work is done in the value stream of Manpower (invisible) which this energia animates the Material, Machinery and money stream. Paradigms are emotional investments that often become stuck or in our own way of our ever-changing choice dynamics – and often forced – not from compassion to understand a human being but from being human. Not being able to let go of logic to the point of making Human capital systems create real scarcity reality for others to suck money for self. These will do as they like, they never change and their fear movie is almost unreeled – but of course this is the choice they like. Integrity and transparency with compassion to understand will trump those who enjoy dimming the light dynamis with solid secrets for security and power. Them critters are amusingly always right.

    The new generation’s paradigm is made malleable from video games. They are attract to presence paradigms with knowledge and attention management systems. Clearly, the funnest way to make money is staffing with a games theory paradigm system that reflects each player’s choices to ebb and flow responsibility above their ego with a host dynamic mission statement that harmonically aligns them to uni-form their manpower to an interdependence mission bigger than their sticky, shrinking or stretched out egos.

    Don’t be scared of your light – In kind,


  11. Hi Ken,

    I’m sorry, but I lack the intelligence to understand the slightest bit of what you just said. Could you restate it in simple English?



  12. Hello Keith,

    Ahhh, English hard to do – but the universal language of math – much easier.

    Your not less intelligent – I was being ignorant and pedantic – that’s me just mirroring/showing us how easy it is to err to be(ing)human.
    A human error root cause system that fool-proofs manpower flows exists. It reflects the user’s and the company’s ever changing levels of consciousness and culture. It has an informatics mirror effect because we are too close to ourselves to view our flow thus differentiate flows from culture or conditions. If Wendell likes the existing IP for compassion (company passion) purposes – to do less harm by not typecasting potential – he will decide to explore the informatics empirically. I have already proven time and again with it that a person’s perception, intention and subsequent work condition expresses their manpower contribution. My newer software version will reflect manpower in dissonance/consonance ratios to forecast staffing consequences and manpower/culture harmonies. These ratios surface and un-baffle group think tendencies inherent in all – its a survival ego “thang” that creates traffic and noise, masking our manpower flow. The new software un-stymies the flow and is easier to run because it shows the direct mathematical relation that manpower = money (in real time) that is all for now – I’m doing it again.

    Godspeed Explorers,


Leave a Comment

Your email address will not be published. Required fields are marked *