Whether or Not You Realize it, You’re Using Assessments

Assessment! What a concept! Imagine a world where job applicants are screened for their job skills?before being hired! Wow!

Assessment = Judgment = Test = Interview = Application = Resume

It’s so simple, it’s complicated.

Folks, anyone who screens resumes or applications, conducts interviews, or reviews past work history to predict future job performance is already using assessments. Ads, postings, websites, and referrals may bring applicants to your doorstep, but assessments separate employees from wannabes.

So wake up and smell the coffee. Assessments are not weird, foreign, or unusual. They are used every time someone places an ad in a trade newspaper, posts on a job board, conducts an interview, uses a “smart” Internet application, “sells a pencil,” or gives a test.

The sooner we realize our profession is up to its armpits in the assessment swamp, the sooner it can change our image from a “learn-as-you-earn” job to one where highly skilled professionals command respect. Can anyone escape the assessment swamp? Sure: advertise everywhere and hire anyone who applies.

Sourcing as Assessments

When we post ads to a specific newspaper or website, or ask for personal referrals, we are using a source-related assessment. In other words, we expect these sources to minimize unqualified job seekers.

For example, posting on an Internet board excludes people who are not computer-savvy enough to navigate the Internet and post to the site. Posting in the WSJ excludes people who are not business-savvy. Posting to a site catering to a specific ethnic group excludes everyone who is not a member of the group.

You get the idea. Sourcing choices are a subtle form of assessment because they act as not-so-subtle pre-qualification screens.

Interviews as Assessments

A recruiter or hiring manager might think, “Hmmm. Interviews are ways to get to know someone. If I get to know you, you should be able to do the job.” Well, if our objective was to know someone at a surface level, then interviews would be appropriate.

However, anyone with a few years of career experience knows that interview skills and job skills are two entirely different things. As we all know, even the most likeable people can turn into incompetent employees.

The problems with using interviews as assessments are well-known: the interviewer often has an unclear idea of the skills required; question techniques might be leading or unclear; the applicant may not be able to recall a good response; applicants fib; personal appearance may affect the decision; and so forth.

In a desperate attempt to add credibility to the interview, people advocate wrong-headed questions like the “gotcha!” (e.g., when candidates are invited to say something negative about themselves); the “one-question-wonder” (e.g., when interviewers have a secret internal job-standard against which everyone is measured); or the “pseudo-shrink” (e.g., where the interviewer thinks he or she is a practicing psychologist and asks deep questions like, “If you were a tree, what color would you smell like?”).

Yes, interviews are assessments. They have questions, answer guides, and pass/fail standards. Once you screen out bottom-feeders, they tend to be about the same as chance at predicting performance.

Automated Assessments

Automated assessments usually include some form of computer interface that presents applicants with a situation/question/simulation/application blank, asks for a response, and scores the answer. The assessment could be located in a kiosk at the front of a store, on a computer in an office, or even presented through the Internet. Its purpose is the same as the interview: screen-out unqualified applicants.

Does it work? Maybe. It is probably better to think about how well it works. Garden-variety interviews have almost no predictive power. Structured interviews are better because they are more highly focused. Tests and simulations are among the best predictors because they are exceptionally hard to fake.

But that’s not all you need to know about assessments. You need to master the dreaded validity beast.

Article Continues Below

Before I started this profession, I was certain that motivation test scores were closely associated with intelligence. That is, I was convinced that high/low scores on motivation tests were valid (there’s that word again) predictors of intelligence. But after a few studies, I saw that 99% of the time, motivation scores turned out to be wrong. I should have known better.

Do you really know whether your test/assessment/interview predicts job performance? I don’t mean war stories; I mean solid investigations to determine whether the applicant is qualified or unqualified.

What can you do to ensure that test scores are valid? First, understand that “performance” can mean a lot of things.

What Is Performance?

Some people (professional and otherwise) get confused about performance. Is performance the number entered on performance appraisals (which everyone lies about anyway)? Is it the score at the end of a project when everyone gathers to take credit for success or assign blame for failure?

Or is it something else?

It helps to think of job performance as a fruit salad: little bits of pears, apples, peaches, pineapple, and grapes residing in a bowl. Each fruit represents a different set of skills.

Pears equal cognitive ability or the employee’s ability to analyze situations and make good decisions; apples represent the ability to learn and apply new information; peaches symbolize planning and implementation skills; and so forth. Each piece of fruit represents a job-related skill that, when used properly, contributes to overall job performance.

Most folks tend to put individual skills into a blender, then press the annihilate button. Within seconds, individual skills disappear into a performance puree. In the process, they lost the ability to examine individual pieces and have no idea whether the pears were exceptional or the peaches unsatisfactory.

A valid test requires comparing apples to apples: that is, on-the-job learning to learning-ability tests; on-the-job persuasiveness to persuasiveness simulations; on-the-job planning to planning tests; and so forth. Comparing fruit puree to 35 dimensions of personality, an industry job “norm,” or a specific test may produce data that looks and sounds credible, but it can produce nonsense numbers.

Performance is not a single dimension. It is the score at the end of the game, the baby at the end of the pregnancy, or the black eye at the end of the long argument.

Someone once argued that his assessment system was so simple, it did not require a Ph.D. to use. I suppose this person thinks this is a good thing. Hopefully, after reading this article, you will realize that “simplicity” and “highly effective” don’t play well together.

You may not need a Ph.D. to be a competent assessment professional, but if you expect your assessment/test/application form/website/interview to accurately predict job “performance,” you better think like one.

Topics

5 Comments on “Whether or Not You Realize it, You’re Using Assessments

  1. This is one of the key points I make every time I train supervisors on hiring practices.

    Not only is anything you do to narrow down your pool considered a test for validity purposes, it’s considered a test LEGALLY (e.g., per the Uniform Guidelines, per Connecticut v. Teal).

    In addition to matching your test with your measure of performance, the other big MUST DO is matching the test to the type of job. For example, if you need someone with good writing skills, why on earth are you just using an interview??

  2. It would be nice to see some metrics on which methods are used, and how are they weighted, and comparitive results, particularly when methods are changed, and how they pan out, as in ‘How long did the person stay?’

    Or a step further, if it could be measured (on employee evaluations or yearly reviews) how well they performed, giving extra credit for longevity perhaps.

    Anyway, to hear the ‘Psycho-babble’ of ‘If you were a tree, what color would you smell like’ was certainly worth it for now.

  3. There are hundreds of examples showing turnover can be reduced and performance increased if the hiring organization is willing to do the following:

    1) learn explicitly what competencies are required for the job and define the business necessity for them
    2) communicate clearly what the job will be like
    3) use only tests, interview questions, application methods, simulations, and so forth, that are job-related and are proven to predict job performance
    4) keep all sources open to ensure qualified people of every class and group have an opportunity to apply
    5) use the same rigorous criteria to select managers
    6) don’t allow conflicting organizational practices to frustrate employees from doing their jobs
    7) provide essential training to supplement skills when necessary

    Now…compare this with what usually happens:

    1) recruiters work from a job description prepared by the HR department and edited by a manager who often does not do the job…
    they generally ignore the real job experts (e.g., people who actually do the job)
    2) people sugar-coat the job so it will sound attractive to the candidate and hide negative information
    3) reject best-practices because they are too ‘hard’ or time consuming, and rely instead on unsubstantiated personal feelings and opinions formed during short conversations.
    4) selectively choose sources that minimize diversity
    5) promote managers based on their performance as individual contributors and, once promoted, protect even the most dysfunctional
    6) hire on one set of criteria, train on another, manage on a third, reward on a fourth, and allow organizational blocks to frustrate employees
    7) try to train the untrainable or change core human behavior

    Ready for the challenge?

  4. To the Good Doctor – I tried, believe me, to find anything that I disagreed with – I couldn’t. Wondering if your support merics (Part 3?) would be friendlier towards good interviewing w/o the many flaws you point out?

    I, too, am amazed how few companies really put any thought into this process.
    Jon

  5. Sorry…Although they are the most widely used assessment tools, interviews are better at ‘screening-out’ than accurately predicting job qualifications

    Think of interviews as a test…they have something to measure, questions to ask, and answers to score.

    Something to measure: the more data the better the interview coverage (both depth and breadth)…data can come from job descriptions (so-so), managers (so-so) or thorough job analyses (most complete)

    Questions: the better the question technique the more accurate the answer…techniques can be easy to fake (open-ended and closed-ended) or difficult (behavioral or situational example)

    Answers: the more related to the job, the more accurate the answer…answers range from ‘nobody didn’t say nothing wrong’ (blecch!), sounds-like something I would expect (so-so), relates to something similar on the job (better), or sounds like exactly like what the job would requre (complete job-specific behavioral example).

    Still, until we can find a way to read minds, interview effectiveness will be suject to lying and personal interpretation.

    Why aren’t good interviews followed? In my experience:
    1) recruiters have little or no incentive to screen-out more candidates
    2) it’s hard
    3) someone else lives with the consequences of a bad hire
    4) managers are resistent to play

Leave a Comment

Your email address will not be published. Required fields are marked *