Knowing is Good, but Showing is Better

Consider the following disparate facts: In 2004, approximately 58% of traffic accidents were due to improper driving (source: InfoPlease.com); in August, approximately 36% of employees turned over (source: www.nobscot.com).

You are probably asking yourself, “What do car accidents and employee turnover have in common?” The answer is, “More than you think!”

I always like to be up-front. If you believe your job consists of finding candidates and letting someone else screen or train them, save your time. Read something else. If you seriously care about hiring and promoting fully qualified managers, salespeople, and professionals, read on.

Suppose some very important government officials woke up this morning with a conscience (yeah, I know it’s a stretch, but work with me). These enlightened civil servants unanimously decided to put you in charge of reducing traffic accidents. They gave you the power to renew current licensed drivers as well as issue new ones. They said you could use any criteria you like. Where do you start?

First, you might want to investigate the job. Naturally, you would talk to drivers with good safety records and experienced driving instructors. After many interviews, you would learn the following competencies were critical:

  • Eyesight
  • Physical coordination and reaction time
  • Basic operator knowledge
  • Knowledge of driving laws
  • Analysis and judgment

Knowing vs. Showing

Knowing that impairment of any factor could lead to driving accidents, you decide to administer a broad range of tests:

  • Personal opinion. Each candidate must be recommended by another driver. This would screen out the blatantly unqualified (assuming you could trust the reference).
  • The personal interview. Each candidate would be interviewed about driving skills; basic operator knowledge; driving laws; and past and/or future driving decisions. This would screen-out people who did not know the right answers, but you would have no proof other than their word.
  • Basic skill tests. Each candidate would be tested for eyesight; coordination; physical reaction time; operator knowledge; and driving laws. This would be considerably more valid and reliable. Candidates who, under controlled conditions, demonstrated physical skills and knew driving laws would be much safer than ones who didn’t.
  • Simulations. Each candidate would “drive” a realistic simulator that presented them with a wide variety of road, weather, and traffic conditions. Again, you could be reasonably sure candidates who successfully completed the simulation would be safer drivers.
  • Psychological evaluations. Each candidate would be evaluated for his or her potential for road rage; use of drugs that would impair judgment; physical health; emotional stability; and so forth. Again, you could be reasonably sure candidates who successfully completed the evaluations would be safer than those who did not.

Now, what do you think would happen to traffic deaths and accidents if only the people who passed this battery were permitted to drive? Right! Millions of dollars and tens of thousands of lives would be saved annually.

Bring it on Home

But what might critics claim? It’s too much work. Too many candidates fail. No one cares about the poor candidates. We should give everyone a chance. Some drivers can be “coached” or trained.

Yes, those are possibilities, but what do you think would happen to traffic accidents as we loosened-up on the testing criteria? For example, what would happen if we were to license everyone and let them “sort it out” on the road? Nah! That’s silly. Let’s drop everything except the interview. Yeah. That should be OK.

Let’s apply this argument to organizations, and more specifically, employment and promotion decisions. Just as driver competency correlates with traffic accidents, employee competency correlates with turnover and productivity. Whether it is a management or professional position, getting the right person into the right job requires multi-trait, multi-method testing.

This method refers to a few simple concepts: carefully determine job requirements; use several testing methods; and measure critical competencies more than once.

Article Continues Below

You don’t need a rocket scientist to argue each time something is left out of that equation, employee quality (e.g., productivity and turnover) would suffer.

What Kind of Multi-Trait, Multi-Method Test?

Everything that separates one candidate from another is a test. This includes references, interviews, photographs, application forms, resumes, sources, written questions and answers, and so forth. Put another way, unless you hire everyone who applies, you are using some kind of test.

Now that we all agree every recruiter and hiring manager is in the testing business, the decision becomes much simpler: how predictive is the test he or she is using?

Predictive accuracy is not complicated. All we want to know is whether the test measures something critically important to the job, whether scores are accurate, and whether the test is stable over time.

Consider the personal opinion “test” in our driving example. Do personal opinions measure something critically important to the job? Are they accurate and consistent? Probably not – unless the candidate behaves like the lead character in a Steven King novel.

Personal opinions are filled with subjectivity. Some subjectivity is good; but unless the job is very simple, personal opinions are weak predictors. Why? It’s a people flaw. Humans tend to think their personal opinions are good indicators of job competencies. Although we all like to think we are excellent judges of character, research and rational analysis show otherwise.

What about the people who are brimming with confidence about their interview skills? Can they be wrong? As amazing as it seems, sometimes better-crafted interviews lead to better predictions, but interviews are still based on “knowing” not “showing.” However, people who have never taken a college course in measurement are often as open to new ideas as a witchdoctor is to germ-theory!

Knowing and Showing

The more a test measures knowing instead of showing, decision-quality drops. It’s a fact. It’s called “test error.” Every test, whether it’s references, interviews, photographs, application forms, resumes, sources, or written questions and answers, contains some amount of error. Error can never be completely eliminated, but it can be reduced by following a few professional steps:

  1. Interview job-content experts such as trainers and job holders to clearly understand the key competencies required for the job (i.e., skip the managers).
  2. Measure every key competency using tests that are job-related and validated (i.e., make sure scores predict job performance).
  3. Use more than one test type (i.e., cross-check).
  4. Clearly understand that “showing” (what you see) is better than “knowing” (what you hear).

Topics

6 Comments on “Knowing is Good, but Showing is Better

  1. It often irks me that the philosophic nature of recruiting is reduced to bean counting.
    Now, recruiters while not necessarily practitioners of the maieutic; are, however, philosophers in our own rude fashion.

    We make broad inquiry of human nature in the agora of employment.

    One of the techniques of philosophical inquiry is to test analogies.

    What would happen if we used Dr. William’s assessment methods for drivers in fact?
    1) Fewer drivers would pass the assessments
    2) There would be fewer drivers-therefore fewer accidents (Eureka!! It works!)
    3) Further, only bad drivers would be eliminated, further weighting the reduction in accidents. (If this were quantifiable)

    In truth, what would happen is many drivers would be off the road, many which earn a living at driving–there?d be fewer trucks, fewer shipments, and fewer people making a living. The economy would collapse?many competent drivers would, through error, be unjustly deprived of their right to drive—but we’d all be safer.

    But would we really be safer?

    No, for the simple reason that in order for the scenario to play out as predicted–the assessments would have to be accurate and reliable. Anyone not in the business of selling or promoting assessment tools knows that notion is nonsense.

    Who would be the selector??Que custode?

    Let us now extend this analogy (or, the actual steps as described in the article) to corporate hiring.

    By further increasing selection criteria we get fewer candidates, if any —– but, they are all perfect, or, within a few degrees.
    Because perfection is an ‘objective’ standard as determined by structured assessment. The candidates tend to be similar in education, personality, and outlook. This will result in stifled innovation; group think, collective incompetence and, ultimately, the death of the company that adopts it. Perhaps not this quarter but sooner than you would think. There is example in the business section every day for those who choose to read it with understanding.

    Bill Wager
    Hunter Green
    40 Exchange Place
    NY, NY 10005
    212-742-0990
    billwager@hunter-green.com

  2. Based upon a Survey by Progressive — Allstate also had one that agreed —

    The survey found that 52 percent of reported crashes occurred five miles or less from home and a whopping 77 percent occurred 15 miles or less from home.

    This, of course, should be intuitive since most people do the great majority of their driving within a close proximity of their homes, but many drivers seem more concerned about safety issues when embarking on a long, cross-country trip than when heading to the grocery store.

    Hmm, I always thought that my excellent Driving Record and the fact that I keep getting it in the mail would be enough to support the fact that I am a good driver. Wow, dang.. how silly of me.

  3. ‘Psychological evaluations. Each candidate would be evaluated for his or her potential for road rage; use of drugs that would impair judgment; physical health; emotional stability; and so forth. Again, you could be reasonably sure candidates who successfully completed the evaluations would be safer than those who did not.’

    Now this is an interesting thought. Who’da thunk of including such a radical idea? Evaluate the potential for rage and drug use? Brilliant! Physical health and emotional stability? You’re skatin’ out on the legal thin ice on these last two, Dr. Williams, but I couldn’t agree more!

  4. Sorry.. I did not realize that recruiting was a philosophy. I thought it was a job where people found and accurately evaluated candidates for a specific job.

    Assessments come in many forms: driving tests, aircraft simulators, tryouts, training programs, probationary periods, INTERVIEWS, resumes, and so forth. Anything that separates people into a ‘qualified’ and ‘unqualified’ group is an assessment. And, the closer the assessment is to the job, the more accurate it tends to be. There is a long and well-established history of specific tools that most accurately predict job performance –and personal subjectivity is not among them.

    Using better and more accurate assessment tools produces clones? Well, you got me there. Accurate tests and assessments consistently identify the most-qualified people. I don’t think many executives who would have a problem with that; but, I do have trouble understanding how a company staffed with fully competent people would be in danger of failure.

    So, which part of a good assessment process do you disagree with:

    >Hiring the most fully-qualified applicants?
    >Rejecting the fewest unqualified applicants?
    >Using hiring tools that are race, age and gender-blind?
    >Using hiring and placement processes endorsed by the EEOC?
    >Realizing the favorite tools of many recruiters and hiring managers are weak predictors of job success?

  5. Dr Williams:

    Your response forms a tautology by simply returning to your prime argument, That is: that your methodology is what produces all these wonderful results. The desirability of the results is not in question; the methodology is.

  6. If you believe our jobs consists of about hiring and promoting fully qualified managers, salespeople, and professionals on. Think again, it?s about money.
    At the end of the day we are talking about money. Work and money; hiring and testing, it?s all about money. We split hairs to take money out of the equation but it always comes back to money. Everyone (and pardon the pun) is on the dime or wants to be on the dime. Will pay a dime or not pay a ruddy dime.

    I think we can agree that:
    1. Candidates, their aptitudes and the work they produce, interact in complex patterns and are influenced by task and situation variables.
    2. Highly structured environments tend to be more successful with employees of lower ability; conversely, low structure environments may result in better job success for high ability candidates.
    3. Anxious or conforming employees tend to do better in highly structured environments; non-anxious or independent employees tend to prefer low structure.

    Therefore: highly structured work, in highly structured environments can get by with lower ability, anxious or conforming employees who need highly structured tools to aid with selecting and promoting them. This will bring them peace of mind but costs them in time and money. Anxious and conforming managers seem to sleep better when the fires of fear of making a decision and the anger of making the wrong one will be dowsed by throwing money at it.
    The more organic, unique or creative work-product typically has known more success in low structured environments because of level independence needed. Typically this culture is comfortable about making decisions and is less-anxious about the reward. Their proclivity with decision making tends to make them more accepting that they may make a bad one from time to time but more often, a bad hire will be influenced by changing task and situation variables and not their decision.
    In the end it comes down to the cost of doing business within the values of the company?s culture.

Leave a Comment

Your email address will not be published. Required fields are marked *