Employment Tests Are Anything but Irrelevant

Last week, I found myself wearing down several hours sitting in an airport by catching the latest and greatest in the HR/employment sphere through LinkedIn, when I stumbled onto an article by Dr. Charles Handler titled “Employment Tests Are Becoming Irrelevant for Predicting Job Success.”  I was intrigued. After all, I am a consultant for a company in which a core area of our business is from said employment tests. Especially with that title, shock value achieved.

The article brings to light a number of interesting ideas about big data via social media and how it stands to influence the way we look at pre-employment. However new and edgy gathering such data via social media may be, it isn’t without its flaws.  Furthermore, if someone has to stand up for employment tests, I begrudgingly accept.

From the first excerpt, “The Impact of Publicly Available ‘Free-range’ Data,”

People born in the past decade or so, along with all persons to come, will begin accumulating a personal digital fingerprint that will be associated with them from cradle to the grave … We are even starting to see research that suggests we can gauge an individual’s job success from social media data such as one’s Facebook usage.

To test a hypothesis, I logged onto my own Facebook and started with the news feed summary. Within the first minute, I counted 12 females, nine males, three mentions of children, five mentions of weddings, announcements of seven birthdays (complete with age), and two mentions of religion.

Hypothesis confirmed: Facebook is a Title VII nightmare. Even for the most well-intentioned onlookers, things cannot be unseen. If Facebook vetting is a part of your process, sooner or later a plaintiff lawyer is going to come knocking with a discrimination suit. The ability to defend that (insert protected class) didn’t influence your decision is compromised once the individual’s Facebook record is shown to the jury.

We know from employment tests that “The Big Five,” referenced in the above study predicts performance. Until we find a search engine that filters out all of these criteria (any developers out there?), why not measure those personality traits the best way we know how, objectively, and free of sensitive information. Your HR and legal departments will sleep better at night. Reading the conclusion of the study: Dr Kluemper appears to agree.

“We’re not advocating employers use this technique,’ Kluemper said about the Facebook ratings.”

In his second heading, “The Rise of Structured, Sanctioned, Verifiable, Shareable Personal Data,” the article states that they:

They were able to show that a variety of rationally hypothesized factors (i.e., # of positions held) impacted key variables such as turnover.

Gauging “turnover likelihood” based on number of jobs on a resume is something that people have been doing with resumes for the past 40 years. We just have switched mediums. While this is interesting, what I find most helpful is whether Person A has a personality that predisposes him/her to turnover because s/he is lazy and unmotivated. The great news is that employment tests can do that. In fact, decreased turnover is the most frequently observed positive outcome of implementing these tests.

Article Continues Below

They also found evidence of less-rational relationships such as the fact that those who gave more recommendations had higher job success levels than those who received them.

Interesting — definitely — but not shocking. Tying recommendations to job success — everyone has a few friends who didn’t quite “make it.” We still love these friends all the same, but I don’t imagine they have a lot of people asking for recommendations on LinkedIn. Contrarily, higher performers at higher levels within organizations probably receive more requests for recommendations. It’s lonely at the top. I don’t envy the number of requests those top-achievers probably get.

I’m not all-knowing. I won’t even claim to be half-knowing. Let’s say big data revolutionizes how we look at our job seekers. What do our job seekers do? They adapt. I ran an Internet search of “Make an amazing resume,” to get 68 million results. Let’s use the example in the article about Klout:

Currently, a great example is the Klout score, which can be used to help evaluate someone’s level of engagement in social media and is often cited as a key piece of data for those being evaluated for marketing and advertising jobs.

Step 1: Make a profile on every social media venue available. Step 2: Connect to everyone you “kind of” know on these networks. If there was ever a time to add your dental hygienist, high school principal, that one bar down the street, (and yes) even your exes … today is the day! Social media big data could likely lose its value quickly.  I’m reminded of my friends who are professional recruiters and what happens to a resume when they “tweak” them. Social media is very much a “resume for the new millennium,” with the same candidate-to-candidate inconsistency.

Contrast this with assessments, which are getting increasingly more advanced in how they measure these pre-employment predictors of performance. You want someone with attention to detail? Assessments have real-time simulations that actually test it. Organizational skills? Give them a simulated e-mail account that updates real time and has 100 backlogged in the inbox. Assessments have been around for many years and they continue to become more engaging, accurate, and consistent. Furthermore, many people far smarter than I are improving these assessments in ways so advanced that that will keep assessments around for a long time. Item response theory analyses, ideal-point assessment scoring, and more realistic job simulations/in-baskets are just the beginning.

What’s the take home here? Assessments are not only still relevant, they are becoming even more valuable. In fact here are three concrete areas where assessments continue to grow:

  • Manufacturing in the U.S. is back with a vengeance on the heels of increased shipping costs and stabilized domestic demand, and supported with very strong government initiatives. Manufacturing isn’t making the same mistake it made last time. Other countries have cheaper labor costs, so these organizations are going to compensate by paying to bring in elite, highly vetted candidates who often take two or three assessments as part of the hiring process.
  • Healthcare is going lean.  Healthcare reform is forcing individual healthcare providers to consolidate into systems. In addition to being providers of healthcare and offering remedies to patients, physicians are now also looked to as employees who are responsible for maintaining a company reputation. Healthcare professionals with high emotional intelligence have been shown to have higher patient satisfaction scores (Weng et al. 2011)  This trait is also nearly impossible to train. Hello employment assessment.
  • Retailers are losing revenue from theft at an alarming rate. Although these numbers (30+ billion dollars) are best-case estimates, integrity/ethical reasoning assessments are of the quickest growing type of assessments out there. Sending more inventory where it is supposed to go equals more dollars in the pockets of all of the employees, managers, and shareholders.

Don’t bail out on pre-employment assessments so fast. Assessments still have our back … at least for a few more years.

Adam Hilliard is an I/O psychologist and consultant for the Pittsburgh-based Select International. He specializes in designing highly defensible selection systems for high-volume hiring initiatives and startups in legally tenuous environments, maintaining a client base of 50+ organizations in the manufacturing, healthcare, retail, and call center service industries. He is very active with the Society for Industrial and Organizational Psychology, with involvement in a number of standing committees, and regularly presents at its annual conference.


8 Comments on “Employment Tests Are Anything but Irrelevant

  1. Adam, exceptional reply to Charles. Thank you.
    Fact is, very few people actually have sound academic foundation of knowledge on the design, validation, and implementation of pre-employment assessment. Ocular analysis is more common that validation analysis. – “It looks good!”

    Charles states getting companies to provide data for criterion analysis is difficult – actual performance data. However, he asserts randomly available data with no direct tie to job demands may be more valuable or useful. Interesting.

    Pre-employment assessment is a form of measurement rigor and discipline for a business process called staffing. From our experience, those organizations that invest in understanding their own data achieve a high return on investment.

    It is pretty big leap of faith and effort to go in search of meaning in the wild west of internet data.

    I think some good words of advice when it comes down to where the meaningful candidate evaluation data resides – “know thyself.”

    Build a good measurement system for your jobs on purpose. It is an act of faith. Scouring the web for a valuable nugget may be more like discovering fools gold.


  2. Adam

    Excellent article that shows my shock value title has done its job!

    I am absolutely in total agreement with you about the value of tests. I promise. However, I can say for sure that there are new ways of ascertaining suitability for jobs on the horizon. Personally I feel that the jury is still out on the value of these things. I am never going to advocate for dustbowl empiricism. But I do feel there is some room in the hiring funnel for AI that helps “direct traffic” and route the most qualified folks to the openings that make the most sense for them.

    My point in the article is that technology is driving change and that some of this change is going to look pretty darn crazy. AND many folks could care less about I/Os and all that we do and are going to go right for the latest fad available. I wanted to be sure to create a warning about this.

    To me the most positive thing about what I see happening now is the impact of all the data we are collecting on the criterion side. I feel we are about to have a chance to learn more about how our tests are working based on the love companies are showing for their data.

    So, lots to look fwd to and thanks for sticking up for testing!

  3. Thank you both for your comments Joseph and Charles.

    I certainly see how your comments overlap. Big data has made us more aware of the value of collecting data, but while the “love” is growing, I am very leery of the significant uptick of it coming from social media, given what seems to be an increasingly risky legal employment sphere coupled with what this may open up orgs to in terms of audits. This medium carries such readily available protected class information. We as a field have done a great job creating tests that minimize Adverse Impact – I only hope that big data doesn’t start providing us with data that correlates with these subgroups, just because it can.

    Could you imagine a world where “liking” an employer’s job opening facebook page could be classified as a recruitment step by the EEOC/OFCCP, and then becomes auditable for Adverse Impact? That’ll make it very difficult to minimize exposure, and with how the OFCCP checks for evidence of AI…Good luck Coca-Cola (56 million likes).

    The jury is definitely out as to whether this stands to be a “fad” or not, but I’m anxious to see more empirical information on this. It’ll be interesting to see how this unfolds in the coming 2-3 years.

    Thanks for your comments!

  4. I fully agree that we are headed for a major legal event related to the use of these types of data for employment purposes. In fact one of my main reasons for writing about these things now is to begin to help us all think about the complete ramifications. I think it may come in another year or two and it is going to spell bad news for some of the firms that are using wide open data to predict for employment purposes.

  5. Charles I love how you stirred the pot with a controversial headline as usual. Nice work. And Adam to Charles point, good for you to refute the ideas presented. Its only this kind of constructive dialog that might bring some people to their senses on this stuff. Big Data was all the rage at a recent conference I attended – I kept thinking the whole time – wow have you guys checked this against the law? There was even one person in the room who said they don’t interview candidates that are not living within a 15 mile radius of the office because they turnover after they experience the commute. Really? That’s like the banks red lining areas on the map where they don’t lend money. Big data sounds like big brother to me and Ive yet to see anyone make it useful to the Candidate Experience. Good posts guys.

  6. Adam, you wrote:

    “We know from employment tests that “The Big Five,” referenced in the above study predicts performance.”

    (See Wikipedia for Five Factor Model (FFM) of personality traits: http://bit.ly/16JzQHx

    “What everybody knows is frequently wrong.” – Peter Drucker

    I submit that most of those saying (believing or knowing) that personality assessments predict job performance are publishers of personality assessments, or, for whatever reason, those who have chosen not to delve into the particulars of predictive validity.

    Peter Cappelli (Professor of Management, Wharton School) (http://bit.ly/16JHstB)on “Assessing Personality” wrote: “For practitioners, however, the conclusions from these editors are remarkably useful and unambiguous: Don’t rely on personality to assess employees.” (Full Cappelli article text here: http://bit.ly/17YIyRY)

    Referenced ‘editors’ paper: “Reconsidering The Use of Personality Tests in Personnel Selection Contexts” full text here: (http://bit.ly/H2mKT).

    Of the Big Five, only “Conscientiousness” measures can appreciably improve predictive validity, when used in conjunction with measures of general mental ability, integrity, work samples and structured interviews, all of which make greater contributions to predictive validity for job performance, according to the seminal meta-analysis by Schmidt and Hunter. http://bit.ly/QK64aW

    That said, pre-employment assessments, other than the ipsative (self-appraisal) personality type, deliver exceptional value through predictive validity for job performance and job learning. Normative assessments of traits and abilities can compare individuals to chosen reference populations – e.g. the general working population or top performers in a particular job or job family. Moreover, improvements in selection process (predictive validity) drop right to the bottom line and recur, annually, throughout the hired employee lifecycle to deliver huge returns on assessment usage.

    Assessments will not become irrelevant, any time soon. We will, however, find ways to (i) raise predictive validity to ever higher levels (ii) upgrade the quality of jobs through job analysis/design/matching and (iii) improve the candidates’ experiences and abilities to present themselves, through validated assessment.

  7. @ Adam: Thank you. I suspect (no proff) that these alternative methods mentioned willl turn up largely the same people that other methods have already turned up elsewhere.

    @ Adam, Dr: Handler: An anecdotal case-
    In all the years I have been recruiting for over 200 different types of position, high, low, and inbetween for companies from tiny startups to Fortune 500, I can’t recall a single instance of my clients incorporating a formal employment assessment test as part of the hiring process. Do you have any figures n what percentage of people nationwide are hired using an AT?

    @ John H: The point is “not to discriminate” but rather “to discriminate in a way you can’t be prosecuted for,” like startups and age or EOCs and veterans.

    @ Richard: Excellent points…



Leave a Comment

Your email address will not be published. Required fields are marked *