Would the EEOC Like the Words You Use ?

DeadspinJust ahead of May’s NFL draft, the irreverent sports site Deadspin created a stir when it posted a tool to analyze the words used in scouting reports to describe black and white college football players.

You simply plug in your word picks to see how often they are used for players falling into those two racial groups. NBC Sports’ Craig Calcaterra found, among other things, that the word “gifted” appears .58 times per 10,000 words for black players and not at all for white. Plugging in “worker,” Calcaterra found it applied almost twice as often to white players as to black. And “underachiever”? The Deadspin tool shows it’s used more often to describe black players.

Now, what if instead of NFL scouting reports, this kind of analysis was used on employee performance reviews or candidate interview notes?

The EEOC, which initially was excited about discussing the possibilities, eventually thought better of commenting on something that has yet to occur. But private labor attorneys, though cautious, thought it almost inevitable that some plaintiff’s attorney will eventually use this kind of linguistic analysis to attempt to show discrimination.

Jay Holland
Jay Holland

“I think this could be very interesting,” says attorney Jay Holland, who heads the labor and employment practice at Joseph Greenwald & Laake. “I can foresee instances in which this kind of analysis could have application (in EEOC-type cases).

“Especially,” Holland says, “as recordkeeping and litigation are more and more being done electronically.”

So far, nothing quite like this has yet been used in employment discrimination cases. The reason, explains attorney Todd Wulffson, is that it would take some expensive groundwork for a plaintiff to convince a court that the analysis is statistically valid.

“There’s absolutely no doubt that someone is going to be thinking along those lines,” says Wulffson. For now, though, there’s “not a body of work that shows it’s viable.”

Unless there was some reason to believe a smoking gun would be found, Holland thought a Deadspin analysis would be, “A very expensive and time consuming process … for an uncertain benefit.”

But HR professionals shouldn’t take any comfort in that. Wulffson says, “The hypothesis (of using a linguistic analysis) is absolutely valid. It’s going to happen eventually.”

Dr. Robert Leonard
Dr. Robert Leonard

In fact, in some limited instances, language analysis has already made its way into discrimination claims. “I’ve had cases exactly like that,” says Dr. Robert Leonard, professor of Linguistics, director of the Graduate Program in Forensic Linguistics, and director of the Institute for Forensic Linguistics, Threat Assessment, and Strategic Analysis at Hofstra University.

“You start with the data. You ponder it and cogitate,” he says. “You compare the words and language side-by-side.” It’s rare to find overtly discriminatory words being used, which is why a forensic linguist has to study dozens or hundreds of reports to see patterns or trends.

After “you get down in the trenches and get some idea of the data” you do a sample analysis. And, adds Leonard, “you look at the context. Context is everything.”

Where Deadspin used the online Voyant tool, Leonard said forensic linguists have similar other programs. A Boolean search could also work.

With the world becoming more digitized everyday, these kinds of analyses “are becoming more common and,” Leonard says. But, in the end, “You always have a human” interpret the meaning.

Wulffson, a partner with the California labor and employment firm Carothers DiSante & Freudenberger, and the former general counsel and human resources SVP of a 12,000-employee company, says he, like most employment lawyers, has seen significant language differences in performance reviews.

Article Continues Below

“I’ve never seen in a race case the types of things (that showed up in the Deadspin NFL scouting reports), but I have seen them in a sex discrimination case.”

A man will be “forceful”; a woman “pushy,” he notes. There are “many” cases involving those kinds of differences, he adds.

Todd Wulffson
Todd Wulffson

Interview notes aren’t as likely to produce much simply because there are fewer of them.

“I don’t know any company that allows those interview notes to survive,” says Wulffson. Attorneys counsel their employer clients to destroy those notes to avoid just such a situation.

When those notes existed only on paper, a shredder was the end of the road. Now, with electronic tools recording both the interview and allowing team interviewers to offer comments and make notes, deleting the records is not so easy. Cloud storage and backup copies can mean incriminating notes exist long after you thought them gone.

Still, it’s the performance reviews that pose the most issues.

Holland says he has reviewed evaluations and other employer records where men are described as having strong leadership skills where the same traits in the female employees are described as “pushy and aggressive.”

“These records are usually kept for long periods or forever, so there tend to be a lot of them,” he notes.

In a large company with potentially thousands of employees who might join a class action alleging discrimination, a linguistic analysis of performance reviews would have plenty of reports and reviews available, and the potential award could make it financially feasible.

“Absolutely,” observes Holland, “I could see that happening right now.”

John Zappe is the editor of TLNT.com and a contributing editor of ERE.net. John was a newspaper reporter and editor until his geek gene lead him to launch his first website in 1994. He developed and managed online newspaper employment sites and sold advertising services to recruiters and employers. Before joining ERE Media in 2006, John was a senior consultant and analyst with Advanced Interactive Media and previously was Vice President of Digital Media for the Los Angeles Newspaper Group.

Besides writing for ERE, John consults with staffing firms and employment agencies, providing content and managing their social media programs. He also works with organizations and businesses to assist with audience development and marketing. In his spare time  he can be found hiking in the California mountains or competing in canine agility and obedience competitions.

You can contact him here.


2 Comments on “Would the EEOC Like the Words You Use ?

  1. Thank you, John, for a very interesting article. I’ve been involved in several compliance oriented audits, both EEOC and OFCCP driven, and there is already a high level of technical sophistication in how these can be conducted. For instance, the OFCCP does a lot of statistical analysis of metadata to identify underrepresented employees in particular job categories and levels based on racial and/or gender qualifiers (in theory other qualifiers, like age, could be looked at as well). They are very good at this. Before now, I’ve never thought about the sort of linguistic analysis that you describe in your article but my gut reaction is that it is entirely feasible. And I’m not sure about it being focused just on performance reviews, although I agree that is a likely starting point. Any and all digital communication could be analyzed (and probably will be at some point given that most digital communication will exist somewhere forever). I’ve had some very interesting discussions recently about the inevitable adoption of ephemeral communication tools in the corporate world, at least for some sorts of communication (think Snapchat for business) and reading this makes me even more sure that will happen. Will the sort of linguistic analysis you predict be a useful tool to uncover and discourage prejudicial thinking? Or a club for frivolous lawsuits and overzealous regulators? Probably both.

    Doug Friedman
    LinkedIn Profile

  2. Great article! Just FYI there is a ton of published research
    in academia showing these sorts of effects (and again, especially gender/sex bias). Really interesting to think what this might mean going forward.

    One thing you didn’t touch on (and this could be a big deal) are the references and letters of recommendation that determine initial hiring

    The legal liability isn’t usually on the company in this case (except for adverse impact, if it’s a big enough company)…this type of
    data might actually show recruiters and hiring managers what sorts of
    descriptions or buzzwords to ignore if they want to remain unbiased and make the best decisions. Could be another tool in the tool-belt.

Leave a Comment

Your email address will not be published. Required fields are marked *