Employment Tests: Are They Biased?

Employment tests may not be perfect, but one new study suggests they’re no more racist than we humans. I wonder what Charles, Wendell, and ERE members who are interested in assessments have to say. Leave a comment here if you have thoughts.

Article Continues Below


5 Comments on “Employment Tests: Are They Biased?

  1. Great article. This is the first time I have seen any work that has broadened the scope of race bias on tests beyond the actual content of the test. In my mind this is excellent research that does a good job of calling to attention the fact that employment decisions go beyond the use of any one test and that a hire is a result of a process with many judgments to be made. In terms of test bias, we as I/O Psychologists have been working hard over the years to find ways to mitigate the known differences between races. While test bias is definitely an important issue, I dont at all feel it is something that should end up precluding the use of tests. A few thoughts about this. First of all, I believe that the more structure that is imparted into decisions made during the hiring process (i.e., structured interviews, use of tests that have been proven job relevant,etc) the less opportunity there is for bias which is essentially rejecting or selecting people based on non-job related factors. Of course using a very biased test within this structure is not going to help things, but there are many known ways to minimize this possibility. Secondly, I believe that fair hiring is a cultural issue within a company and something that can be facilitated via awareness, training, and good process. Finally, I believe that we as test and assessment creators are still working on ways to eliminate bias. Simulations for instance, are a way to continue to eliminate bias issues because they ask people to actually perform key aspects of the job in question. Also, the more data we collect on the relation between tests, performance, and bias, the more we will begin to develop ways to build better tests. This has already been happening.

    I look forward to the comments of others on this interesting topic and applaud Dr. Scarborough on his program of research.

  2. I agree with Charles that it’s always nice to see studies that recognize the multiple hurdles inherent in all selection processes.

    This issue really gets to the age-old debate about score differences versus differential prediction. Minorities may score differently–on average–but does the test over- or under-predict their actual job performance? Most studies I’m aware of say no.

    And the question is not are “tests” more or less biased than interviews, because interviews ARE tests. So really the question should be which types of tests show more group differences, and we already know the answer to that: cognitive ability tests tend to show the greatest differences. Personality tests and interviews, less so.

    But even that isn’t really the issue. Unless we want to get into a debate about affirmative action, the real question is do the various selection mechanisms WORK–do they predict job performance? Shoddily constructed or applied screening mechanisms result not only in bad prediction but treat applicants unfairly by subjecting them to poor exams.

    BTW, the study actually came out a year ago and is best linked to here: http://www.mitpressjournals.org/toc/qjec/123/1

  3. Bias is often a function of any evaluation process. The act of favoring, by its nature, places more value on some variable. Thus, candidate evaluation methods based upon one test or assessment are more likely to allow the effects bias to creep in. The challenge is to create an evaluation process that captures a wide range of variables to value.

    The meta-analysis of Schmidt and Hunter leave us with one strong guiding principle: Human behavior is complex and to predict it, use two or more evaluation methods.

    Best-in-class candidate evaluations use a multi-method approach such as work-sample based simulations. These types of systems combine as many as six or seven evaluation methods into one seamless experience. The complex nature of the candidate data set from this approach to evaluation allows for a scoring model that all but eliminates bias and adverse impact.

    In-house validation analysis is the only real test of bias and adverse impact. Validation is the process of calibrating a candidate evaluation method to the performance requirements for a specific job, in a specific company. Bias and adverse impact are company specific. Only company specific calibration can document the fairness of an evaluation.

    The percentage of companies using assessments and testing who have actually completed an in-house validation is small. This suggests that many assessment users do not have data to discern if their candidate evaluation is enhancing or minimizing bias. In an EEO claim, the investigators will seek the documentation which speaks to the job relevance and fairness of the evaluation.

    So to reduce bias, seek to develop a candidate evaluation process which integrates data from multiple sources. Conduct your own validation analysis. Document your fairness.

    Joseph P. Murphy
    Shaker Consulting Group
    Developers of the Virtual Job Tryout®

  4. Joe doesn’t the Schmidt and Hunter meta-analysis also find that cognitive ability is the most highly correlated predictor of job performance?

    Todd, the sentence “one new study suggests they’re no more racist than we humans” may be unfortunately constructed- implying that they are quite racist indeed.

    Identity politics are a very tricky issue. I saw Ward Connerly speak at CPAC last weekend on CNN, and in his way, he makes some compelling arguments. On the other hand, we know discrimination is happening every hour of every day…

    Anyway my own theory of assessment (at smaller scales) is team based, which adds an entirely new level of complexity to the question; how racist are your current employees and how will a given individual respond to (and impact) that environment?

    What about situations that are out and out racist but are also equal in terms of rewards and opportunities ?

    Situations that are race-neutral structurally but where people of certain race(s) come up short on rewards and opportunities?

    Makes one’s head spin…

  5. You’re right Martin that it could be unfortunately construed. I didn’t intend to say we’re all a bunch of racists.

Leave a Comment

Your email address will not be published. Required fields are marked *