How to Leave the Interviewing Stone Age

Once upon a time, there were no human resources departments. Applicants were interviewed by managers and hired or fired on the job. For most employees, work was often simple and labor intensive. Not much changed as the need for workers grew, except management created a new department to process paperwork and administer benefits. As you can imagine, new employee skills were only tested on the job. Eventually, the “paperwork and benefits” department was assigned the tasks of placing help wanted ads and pre-screening applicants.

For the most part, applicants were still interviewed by managers and hired or fired on the spot. For most employees, work was still often simple and labor intensive. As you can imagine, new employee skills were only tested on the job. Throughout this time, interviewers’ primary objective was to screen out blatantly unqualified candidates (i.e., people they either disliked or who drooled on the paperwork) and forward them to the hiring managers. Without any special training or education, their interview questions sounded something like this: “Tell me about yourself. Why do you want this job? Do you have any relatives who work here?”

As you can imagine, new employee skills were only tested on the job. Time went by, and interviewers became more confident, often to the point of believing they were trained psychologists. The personnel department even creatively renamed itself “human resources.” Questions changed slightly and became something like this: “What color do you prefer? What is your greatest strength? If you could be an animal, which would it be, and why?” As you can imagine, new employee skills were only tested on the job. Nothing much changed except interviewers sounded sillier, and applicants read advice on how to fake well and get the job. But would anyone be surprised to learn that research shows that interviews are most predictive of future job performance only when they meet three criteria:

  1. The interviewer works from a competency-based document that outlines the skills necessary for job success or failure. This is not a job description and it is not a job evaluation band. It is a list of measurable competencies based primarily on interviews with successful job holders.
  2. Article Continues Below
  3. Interviewers have learned to phrase questions in such a way that answers are difficult to fake and examples are job-related. They have learned that past job behavior – not necessarily job performance – is a very good predictor of future performance. (Just like speed, strength, and reflexes are good predictors of winning at tennis.)
  4. Finally, each interview question must have a scoring guide consisting of desirable and undesirable answers. An interview is not a conversation to get to know someone. It is a verbal test. It has something to measure (required job skills), something to ask (structured questions), and a standardized answer key (right and wrong answers).

Structured interviews are usually called “behavioral” because they attempt to discover the specific behaviors associated with job performance. The assumptions, as mentioned before, are 1) if learning difficult information is an important competency for the future job; and 2) if the applicant says he or she learned in the last job; and 3) if the applicant can demonstrate that learning was successful; then 4) the interviewer can assume the applicant would probably be successful in the new position. What you should remember:

  • Interviews are tests and subject to all the conditions of a good test.
  • Job descriptions and job evaluations seldom provide enough information on which to base an interview.
  • Interviewing is not a learn-as-you-earn activity.
  • If skills are not accurately evaluated pre-hire, then the job will evaluate them post-hire.
  • Few people have the skills necessary to do a competent interview.

If you forget the above, remember that poor interviewing leads to increased turnover, lower individual employee performance, and higher training expenses.

Topics

12 Comments on “How to Leave the Interviewing Stone Age

  1. Wendell – I guess you haven’t read Schmidt and Hunter’s ‘The Validity and Utility of Selection Methods in Personnel Psychology
    Practical and Theoretical Implications of 85 Years of Research Findings.’ The author’s clearly and explicitly state that past performance is a the best predictor of future performance.

    Lou Adler

  2. Dr. Williams,

    I was extremely excited to read your article this afternoon as I agree with you 100%. I’ve had first hand experience in seeing an entire operation turn around simply by using behavioral based interviewing techniques.

    I worked for one of the largest shipping organizations in the World. Our department was charged with hiring PT operation staff.

    When I came into department the turnover was between 60%-50%. There were major issues such as low morale and poor attendance. There were also massive issues with meeting production time and ensuring timely delivery of packages.

    By utilizing behavioral based interviews, aligning ourselves with the right sources to attract candidates and integrating a retention strategy, within months the turnover dropped significantly. We were able to find high performing people by assessing their past behavior. Our department also worked closely with the operators to monitor employee morale.

    Our turnover decreased from 50% in 2000 to 24.4% in 2001. By 2003 the turn over was down to 13%. The impact of aligning the workforce with the right people transformed the entire operation. Our Division went from being one of the worst in District to being one of the best.

    It is amazing to see happy people producing and operations functioning smoothly. True HR professionals understand the need to support the business units by providing the best service.

    Successful work environments are a product of hiring qualified or the right people, good training and development programs, implementing a retention strategy, providing good perks and motivating and inspiring people to work. Thank you so much for the wonderful article.

  3. LOL…That was a GOOD one Lou! I really enjoyed it!

    The analysis you referred to was a meta-analysis. In lay terms, it was an ‘average’ of common selection methods (and represented a seminal piece of research).

    If you read the validity table in that report, you will see the ‘average’ correlation of a stuructured interview is .51. A ‘structured’ interview is defined as one backed by a formal job analysis, a standard set of questions based on that job analysis, multiple trained interviewers, data integration, and a standardized scoring guide.

    Sorry, the garden variety interview is still a poor predictor.

  4. You’re right..I don’t understand your question.

    Exactly which part of this article do you consider ‘off’? The necessity for job analysis? The need for structured interviews? The need for a scored interview guide? These are all well-researched elements of structured behavioral interviewing. Just read anything by Campion (one of the most widely recognized BEI academic experts)

    As to past performance being the best predictor of future performance…well maybe. Actually, mental alertness tests (MAT) tend to have the highest correlation with job performance. For example, the best behavioral interview generally correlates about .50 with performance. This means only about 25% (.50 X .50) of the variance in performance ratings are explained

    A good MAT generally has about .80 correlation…or about 64% of the variance in performance ratings is explained. Better than a 200% improvement.

    Past behavior is a good predictor of future behavior but is not the best, by any means…Think about it…

    1) Does past behavior always align with future requirements? Seldom
    2) Do applicants attempt to put themselves in the best light? Generally
    3) Do interviewers usually have individual bias? Usually
    4) Do applicants tend to hide weaknesses? Probably
    5) Do interviewers tend to ‘screen-out’ using questions of evaluate competencies?
    6) How many interview questions are at the core-competency level? Few

    All these questions undermine the predictive validity of the interview.

    In general, it is always dangerous to place full confidence in one or two research reports…Otherwise, professors would never require I/O graduate students to read thousands of research reports.

  5. Excellent read! An insightful, yet ‘Duh’ perspective on the modern interview practice. However,in applying these ‘modern candidate evaluations’ I don’t find us applying these ‘tactics’ to screen candidates in, but rather out. We’ve made our own profession even more challenging; which to me is great, as we do end up selecting that top 20 percentile candidate vs. finding out (on the job) someone has not met our team objectives.

    I do have a series of questions related to this subject: What is the deal with the term ‘company culture’, and the idea of fitting in to it? How does this term accurately get defined for a particular firm and why more often than not do HR teams (who more than likely have not been around long enough to hire EVERYONE) seem to think they know this so called culture better than a hiring Manager?

    I understand the premise, but todays ‘company culture’ usually means thinking or working the way ‘we do’ – What’s happened to retaining a qualified candidate who offers a fresh perspective even though he/she may not be an ideal fit for the position they’ve interviewed for? What’s happened to hiring based on a belief, not just a need?

    Thanks again Dr. Williams. You’re always on point!

  6. To put the issue of how to select successful candidates into perspective, let?s look at some facts:

    * Voluntary and involuntary turnover in the US workforce (151 million or so) is currently more than 30% annually. (http://www.bls.gov/news.release/jolts.t03.htm)

    * The cost of a bad hire averages 2.5 times annual salary, according to a study from Right Management Consultants (http://www.corporate-ir.net/ireye/ir_site.zhtml?ticker=rht&script=410&layout=0&item_id=849080).

    * According to a study from Leadership IQ, 46 % of all new hires fail within 18 months (quit, are fired, or written up for poor performance or behavior). Poor selection is the primary cause. (http://www.leadershipiq.com/news_whynewshiresfail.html)

    Clearly we face a major crisis in selecting quality employees, and we need to solve it ? not conduct polite (or not-so-polite) academic debates! That means using all the tools available, rather than looking for the one ?magic bullet? that we?ll never find ? and then integrating those tools to get a complete, accurate picture.

    Every tool has its own strengths and weaknesses.

    1. Resumes? According to a recent article in Time magazine (http://www.time.com/time/archive/preview/0,10987,1186550,00.html), 14% of resumes contain false education information, and 43% were found to contain ?significant inaccuracies.?

    2. Behavioral interviewing? An excellent technique ? but many college career centers now have effective courses on how to beat/foil the best behavioral interview questions. There are at least five times as many websites on how to beat behavioral interviews as there are about how to conduct them! To be effective, interviewers need to be extremely well-trained and highly skilled, and to practice the technique constantly. Most will never meet those criteria.

    3. What about psychological profiles? These can be useful, but in one study 46% of test-takers admitted to cheating in order to ?get the job.? There are now at least sixbooks in print giving step-by-step instructions on ?How to Master Psychometric Tests? or ?Ace the Corporate Personality Test!? ? complete with sample tests ? and they work!

    4. Background screening is extremely good at catching convicted felons, active drug users, people claiming false degrees, etc. However, it doesn?t work so well for finding people whose bad performance or unacceptable behavior is not a matter of public record ? i.e., the overwhelming majority of bad hires.

    5. Reference checks used to be the gold standard. If people who worked closely with the candidate in the past could tell you, candidly, specifically, and in detail, about the candidate?s performance, on-the-job behavior, and competency levels ? you would be well on the way to having the ?complete picture? you need to make consistently effective selection decisions.

    The bad news is that corporate legal departments have scared most HR departments into providing only ?name, rank, and serial number? on former employees, due to largely ungrounded fears of ?defamation? claims.

    The good news is:

    * 40 state legislatures have passed laws protecting companies who provide reference information when the reference was given in good faith.

    * Online reference assessments, which are done confidentially (and usually with a liability release from the candidate), have proven effective at getting high-quality information from former supervisors, co-workers, and customers about candidates? past performance, behavior, and competency levels.

    * Internal and external recruiters can be trained effectively to get high quality information from references. This provides a high ROI for key hires.

    * SHRM has put its reputation and resources firmly behind the ?reference reform? movement ? both legislatively and professionally ? to help ensure that in the future we will all be able to access more of the information we need to make good hires and avoid hiring mistakes.

    In summary, we need to know about past performance ? and past behavior ? and psychological makeup ? and more in order to make the best decisions. It?s a question of using all the most cost-effective, reliable, and appropriate tools we can find, while knowing exactly what each one can do and can?t do ? and then knowing how to integrate all that information to see the complete picture of that eager candidate on the other side of the desk.

  7. Wendell – COL! You missed the point! And didn’t answer the question. Behavior is a sub-set of performance – not the other way around.

    I’m in 100% agreement with you that most interviewers are bad. We only disagree on how to make them better. Behavioral interviews don’t work unless they’re tied to comparable performance. Past performance can’t be faked if the interviewer digs deep (in a structured manner). On the other hand behaviors that don’t tie directly to real job needs can be faked. For example a highly motivated sales person with a track of success farming a mature market, might be a total failure if he’s required to build a new territory hunting for prospects. Unless you relate the behavior to actual job needs you induce potential errors into the selection process. That’s why there are few success stories with your approach to behavioral interviewing.

    However, looking at what behaviors were required to achieve comparable performance is a valid predictor of success – more than 80% predictive validity when you add a skills test and in-depth performance-based references.

    Regardless you made some good points – even if some were a bit off. Remember there’s more than one way to skin a cat, and seeking alternatives approaches to accomplish the same task is a worthy behavior to possess.

    Good Luck

    Lou

  8. Greg,

    The reason why it is important to become intimately familiar with the company culture is that even a strong candidate technically and skill-wise can be a train wreck if his/her personality does not match the personalities that they will be working with. The Manager and HR likely do not have the skills to adequately assess company culture because this falls within the realm of psychological group dynamics. The Manager could likely be blind to his/her affect on group dynamics and HR likely has an idealistic view of what they want the company culture to represent itself. Recruiters cannot get a good feel for the company culture by asking ‘Tell me, what would you say is the culture in the work group that the incumbant will go into?’ A good recruiter, when recruiting for a Manager or above position should consult more than one person about the culture. Best practices should include consulting the company’s organizational development manager or cultural officer if they have one. This will help you avoid presenting ‘great’ candidates that for apparently no reason do not get selected because they do not fit in to the work group dynamic.

  9. Boy!! Who would have known?

    …Campion and company has made BEI user unfriendly?

    …Academics (and, presumably educated people)don’t understand the practical world of hiring?

    Sorry…In case you might have not read the articles, those silly academics have already investigated the ‘far-simpler’ technique you described earlier. It could be better.

    Furthermore, when an entire community is trashed to justify an uninformed personal opinion, a reasonable person would conclude we have just left the land of intellectual debate.

    By the way…if anyone wants to read more about the best way to do a BEI (i.e., the most accurate way), just send me an email at rww@scientificselection.com and I will send you a copy of Mike Campion’s paper.

    Judge for yourself.

  10. Wendall – I suspect you’d agree that if interviewers did the following they’d get a 90% (or better) correlation with OTJ performance (81% variance explained)
    1. Job analysis
    2. Structured interview (performance or behavioral)
    3. MAT
    4. Skills testing
    5. Detailed performance-based reference checking
    6. Honesty test
    7. Determination of cultural fit (why not compare the culture and environment of major past accomplishments to the accomplishments and culture/environment of current job?)
    8. EQ (team skills) evaluation of some type comparing team results in past to that required in the current job.
    9. Put emotional biases in the parking lot

    Our only point of disagreement is that there are ways to do this far more simply than you, or Campion, suggest, and obtain 90% user adoption. (natural BEI user adoption rate is less than 20%.)

    Regardless, I agree that few interviewers or corporations have a process in place to do this. And even those that use BEI don’t do it too well. The reason is that Campion and company have made it user and candidate unfriendly. This is called sub-optimization. Something academics either don’t understand or overlook in the practical world of hiring.

  11. I don’t anyone would disagree with your assertion, Lou. The problem lies in accurately assessing past performance in interviews. Interviews are fraught with rater bias, inconsistencies, and subjectivity. As researchers and consultants, the best we can do is to add as much structure as possible to this process in hopes it is followed.

  12. Nicole – actually we have no problem training managers to measure past performance using just two basic questions. Here’s what one in-house trainer had to say after she gave her first class in Performance-based Hiring to a group of hiring managers. (FYI – Jason is our Director of Training and led 10 of their trainers through our train-the-trainer program. We received this email yesterday – July 20.)

    ‘Jason, we just had our first class at ABC. Everything went Great. The managers who attended are really excited about this new way of interviewing. We got a lot of feed-back, so I?m very pleased.

    We are so excited about these classes and I know this will help us do a better job in hiring ?Top Employees?

    Thanks so much for your help and for being there for us.’

    Pat

    PS – this is a company that just dumped 300 pre-paid BEI workbooks because their managers refused to attend the class. And those hiring managers that did attend the class found BEI contrived and unworkable.

    Lou

Leave a Comment

Your email address will not be published. Required fields are marked *