The Einstein-Clavin Effect

Some readers might recall Cliff Clavin, a character on the old “Cheers” television program. Cliff was a postal worker with a massive blind spot. In spite of all evidence to the contrary, Cliff thought he was smart. Unfortunately, Cliff’s IQ was only slightly higher than first-class mail. Albert Einstein, on the other hand, was a theoretical physicist who expanded our understanding of the physical universe. He was so smart that researchers kept his brain in a jar for study (after he died, that is). Now, let’s suppose both Cliff and Al decided to apply for a job. Let’s further suppose they both took a test that asked questions about their intelligence, problem-solving ability, school subjects, success attitudes, sales ability, customer service, and management style. Will their test scores accurately predict ability? Hopefully, you said, “No way! A management test, personality test, sales test, or any other kind of self-reported test, generally predicts success only if someone is too dull to fake good. That is, we could probably trust a low score, but we would have to be very cautious of high ones.” Excellent response! We should also not be surprised to learn that controlled research studies confirm the fact that people who “fake good” on self-reported tests can outscore folks who give honest responses. Burn that into memory: People who “fake good” on self-reported tests can outscore folks who give honest responses. This is the problem with many tests marketed for hiring. At first blush, they may seem like the answer to all your prayers, but experience shows they give a false sense of security. Al’s high score in “problem solving” for example, might be the same as Cliff’s, with one “small” difference: Al has an abundance of mental horsepower that Cliff lacks. Validity? Validity means someone conducted a formal study that showed test scores predicted performance for their job. The same validity study cannot be assumed to work for your job. Validity is local ó local to the organization and local to the job. Local. The only time a test user should trust someone else’s validity data is when he or she knows (really, really knows) that both jobs are virtually the same. But since everyone insists that his or her company is different, using external validity studies become problematic, yes? Well, let’s just make validity even more complicated. Validity scores are often assumed to fall along a straight line: a score of 10 equals 10% performance, 50 equals 50% performance, 100 equals 100% performance, and so forth. That’s what traditional statistics evaluate: straight-line, normally distributed relationships. The trouble with relationships, however, is that they are generally not linear. A 20% difference in test scores seldom translates into a 20% difference in job performance. Test scores and performance ratings are often error-filled, and test scores can be too low, just right, or too high. For example:

  • Performance is seldom linear. Unless we have something to count (e.g., units per hour, dollars per month, and so forth), the most we can say about performance is 1) people are at the top of their game, 2) they are doing okay, or 3) they are fish bait. In spite of the fact that HR asks us to rate employees from 1 to 10, most folks cannot accurately describe ten one-point differences between Billy Bob and Sally Mae. Nor can they put overall values on performance when, for example, Billy is better at closing but Sally is better at customer service.
  • Test scores are not like thermometers. While people can often sense a few degrees of temperature rise or fall, they cannot reliably identify a few points of performance difference. Like Billy and Sallie in our last example, there are simply too many factors to consider and too many things that interfere.
  • Speaking of interference, there is no such thing as a “perfect” test. Test scores tend to float up and down. I heard of one applicant who was given the same test by several organizations (the Wonderlic, a highly popular test of mental alertness). She started out average on her first trial, but after she took the test a few more times, she became a genius. Recall this story the next time a vendor brags about his or her widespread test popularity (nothing comes without a cost in this business).
  • High or low? Some managers tend to hire the best and brightest, put them into jobs that are dull and predictable, and act amazed that employees either turn over faster or demand fast-track promotions. For example, I once worked for a self-proclaimed world leader in testing whose consultants consistently designed assessment systems that hired the best and brightest for green-field startups. Guess what happened one year after the plants were up and running? Does the phrase “all chiefs and no Indians” mean anything? One size only fits all when you wear body paint ó and not everybody looks pretty in paint (compare Demi Moore with Michael Moore, for example).

Sorry about that last mental image. It was cruel, but a few weeks of therapy should help. Putting Your Gut First? Psychologists tend to be a pretty liberal bunch. While I was in grad school, many of my classmates argued for the “job equality” of men and women. Nothing wrong there. So I tried a little experiment in cognitive psychology to see if inner feelings matched public words. I divided the class into four groups and gave each group private instructions: Group 1 was to brainstorm a list of desirable business and management adjectives; Group 2 was asked to brainstorm a list of undesirable business and management adjectives; Group 3 was asked to brainstorm a list of “male” adjectives; and Group 4 was asked to brainstorm a list of “female” adjectives. When everyone was done, I asked each group to report. Guess what? Male adjectives matched the desirable business and management list, while female adjectives matched the undesirable business and management list. Their inner feelings “short circuited” their public statements! This exercise demonstrated how internal stereotypes can unconsciously affect external decisions (even among folks who argued they knew better). The same error-prone stereotyping applies to models such as social styles, MBTI, DISC profiles, sales styles, and leadership styles ó fun, but often impractical, unrealistic, and downright pejorative to qualified applicants. Take a Flyer on Poor Test Data? A few articles ago, some readers mounted a micro-attack against the use of empirical data to make hiring decisions. The argument was, “Tests cannot tell us everything about a candidate. Sometimes you have to ‘go with your gut’ and ‘take a flyer’!” (I think that means to take a chance). Okay. Of all the hiring opinions I have heard, that is certainly one of them. We can argue all day from the sidelines, but are most line managers willing to take a chance on an untried candidate? I floated the idea of “taking a flyer” with a few of them. Their reaction was not positive. In fact, the managers I spoke to were downright hostile that anyone would even think of asking them to either interview an unqualified applicant or hire someone who could not demonstrate skills before starting a job. Hmmm, I wonder why? Conclusion Testing is like quicksand. It looks harmless and easy, but it is very deep and has the potential to swallow users without a trace.

Article Continues Below
  • Always ask the test vendor to demonstrate his or her test “works” for your job and your application.
  • Never accept a test vendor’s word that a test has been “validated” unless you have evidence that test scores predict performance in your jobs.
  • Know how to set cut-off scores: too high, just right and too low.
  • Understand if high, medium and low scores can be trusted.

And finally, always be certain your tests can separate the Einsteins from the Clavins.

Topics

4 Comments on “The Einstein-Clavin Effect

  1. Nice review of testing concepts.

    But lets not mischaracterize ‘taking a flyer’ as it was discussed here on ERE-

    We are talking about selecting between otherwise QUALIFIED candidates, in situations where capacity to do the job is not the major question, but deciding who might be a higher performer IS the question.

    It would be silly to advocate hiring unqualified people- if that were the case, why interview at all? Just hire whomever shows up !

    Regarding Einstein- there are reasonable people who now believe that his first wife (Mileva) may have made more than a small contribution to his ?miracle year of 1905?. Not only did she do almost all of the actual writing of the work, she was one of Europe?s few female physicists and had fairly extensive exposure to other physicist?s work on similar topics.

    And the funny thing is, after he ditched her to marry his cousin, he never did anything extraordinary again ? in fact, his later work was notable for it?s contrast to his past brilliance. It drives Einstein worshippers nuts, but it does seem plausible considering sex roles in that time and place, and his later resentment and truly nasty treatment of her. Not that anyone but those two could ever know, but it?s an unusual set of facts.

    Maybe employment testing should be done overnight, so employers might get ?two for one? deals a la our recent past President and his wife 😉

  2. An interesting theory, Martin, but not one that stands up to basic scrutiny.

    If Einstein’s wife had done the work, surely contemporaries of the time would notice that he was unable to adequately defend his theories and postulates.

    The theory can only be advanced now because those people who could speak directly to it are no longer around. They would know if Mileva? was the brilliant one and Einstein the plagiarist.

    A better story would be Mileva was the emotional support or the production support that allowed Einstein to work in a theoretical mindset while she took care of the mundane. Married couples often function as one organism, splitting up tasks. When Einstein left her, he literally left half his brain behind.

    Which, when applied to tests for competency, make them even more difficult. We aren’t allowed to probe into the personal life of candidates, but the support they receive at home, and the pressure they feel to achieve, is often more important than their capacity to do the job.

    Which is why every sales manager I’ve ever had has told me to get married, buy a home, and have a couple of kids. Now that I’m engaged and about to buy a home, I never worked so hard.

  3. There is no doubt that Cliff Clavin is THE real true genius compared to Big Al. The following dialogue proves the point.

    Survival of the fittest
    [Cliff Clavin talking to Norm in Cheers]:

    ‘Well ya see, Norm, it’s like this… A herd of buffalo can only move as fast as the slowest buffalo. And when the herd is hunted, it is the slowest and weakest ones at the back that are killed first. This natural selection is good for the herd as a whole, because the general speed and health of the whole group keeps improving by the regular killing of the weakest members.’

    ‘In much the same way, the human brain can only operate as fast as the slowest brain cells. Excessive intake of alcohol, as we know, kills brain cells. But naturally, it attacks the slowest and weakest brain cells first. In this way, regular consumption of beer eliminates the weaker brain cells, making the brain a faster and more efficient machine. That’s why you always feel smarter after a few beers.’
    Hard to argue with logic like that.

  4. Hi James-

    It’s actually a fascinating point that you raise; how important is the milieu of your candidates / employees in the performance equation?

    Its used to be an axiom that behind every good man?(we can modify for the enlightened era to say that behind every good person??.)

    I know from experience that spousal attitudes can have a huge impact on retention and performance. Maybe there are some tests to predict how they will respond to a given employment environment?

    And think of the impact on employment branding if the target audience is thought of to include spouses and family members- or even friends (for the new generations who make their own-quasi families)!

    I don?t think the theory that Einstein plagiarized anything holds water; only that the work would not have happened as it did without his wife in the picture. Regarding defense of the work after the fact; Its human nature that once something is proven and explained, it appears to be blazingly obvious; try NOT to see some flaw in a paintjob or some image in a brainteaser once you know its there- quite impossible?

Leave a Comment

Your email address will not be published. Required fields are marked *