Using Bio-Data for Selection

Some of you might have heard or read about Google and its bio-data applicant screening process. As cited in a recent New York Times article, its basic approach is supposed to be simple:

  • Survey current employees on a variety of characteristics and traits, including teamwork, biographical information, past experiences, and accomplishments.
  • Statistically determine which of these many traits differentiates employee performance.
  • Develop an online survey to gather intensive bio-data questionnaire.
  • Score applicant responses based on the number of performance indicators each candidate possesses.

All in all, this is supposed to be a more “scientific” approach to hiring. Don’t rush for the bio-data survey solution yet. At the end of this article, you can decide for yourself whether this sounds like a good process (and be thankful you were not the one who convinced management bio-data was a good idea).


Biographical data or “bio-data” surveys are well-researched in the literature. They work on the same principle as behavioral interviewing: what was done in the past predicts what will be done in the future. The big differences between bio-data questionnaires and behavioral interviews is that a good behavioral interview is backed by a thorough job analysis, interviewers can ask follow-up and clarification questions, and multiple interviewers coordinate the information.

A bio-data form depends entirely on the people who created the items, the scoring algorithm, trained analysts who look for trends, and the specific position. More about these later.

Both bio-data questionnaires and behavioral interviews are self-reported information subject to applicant creativity and being in touch with reality. In general, they both have about the same degree of predictive accuracy. Let’s peel back the bio-data onion.

Our pet whimpers. Dr. Dolittle is on vacation so we don’t know when it started, where it hurts, or whether there are other symptoms. We just know Fluffy is in pain.

Low employee performance is similar. We can evaluate employee satisfaction, voluntary turnover, training success, or terminations. But these are all end results. They don’t actually tell us the root-cause of the performance problem.

Traditionally, performance problems can be traced back to bad management (i.e., incompetent managers, conflicting goals), unpleasant working conditions (i.e., wages, benefits, environment, and insufficient resources), lack of training, and/or poor job skills. There are many reasons why people under-perform.

Finding out the root of the problem is the most important part of developing a pre-hire test. If we don’t know the root cause of low performance, or the root cause of high performance, any hiring solution will be half-baked because it won’t address the issue.

What’s Performance?

How do we measure employees’ performance? Is it measured by judging whether a company’s growing fast? Is it measured by whether a company is profitable? There are too many environmental factors for us to assume that employees are the only growth factor.

How about performance appraisals? We all know performance appraisals suffer from the “no one here is perfect,” “everyone here is perfect,” or “forced-rank” syndrome. In addition, performance appraisals tend to be part fact and part management opinion. Basically, we can never really know what performance appraisals measure. Two managers may rate the same employee differently.

Let’s say we have objective productivity data available such as units per hour, mistakes per 1,000, cold calls per month, cross-selling revenue, or customer service surveys. These are better indicators of performance because they are less likely to be affected by other factors, but we still have to account for things that might influence them.

Did the machinery malfunction or was it newly renovated? Were mistakes suddenly calculated on a different basis? Was the territory newly acquired or was there a company promotional campaign?

You get the picture. Accurately defining performance and controlling for outside factors is absolutely critical. Otherwise, you run the risk of measuring garbage.

What Can Current Employees Tell Us?

Assuming that performance data and root-cause data are under control, let’s look at current employees.

Current employees are a great deal alike. That is, they are all “good enough” to stay hired. The differences between high- and low-performing current employees (assuming we are exceptionally clear on the definition of performance) are generally very small. So small, in fact, that performance differences might be due to pure chance (now, wouldn’t that mess up the recipe for success?). Applicants, on the other hand, are very different.

In addition to the applicant-employee difference, not all jobs have the same skill requirements. Does it come as any surprise that singing in the glee club may have nothing to do with administrative skills? I know sales managers who only hire salespeople if they played athletics in high school (the poor-man’s bio-data test). Fifty percent consistently fail within the first year. That’s no better than chance. Did the athletics bio-data question work? You do the math.

Pick up any good book on bio-data and you’ll see that trustworthy bio-data scores are exquisitely sensitive to positions. In other words, salespeople, first-line managers, and administrative support all might have completely different bio-data profiles associated with job performance (there’s that p-word, again).

Article Continues Below

High performers are usually specialized beasts who do not conform to any norm. They are usually so good that they operate on automatic pilot; or they cut corners to achieve their goal. I recall a marketing manager who stole his prior employer’s product secrets and used them to reduce development time. There’s a good high-performance role model? Right?

You may think that you should figure out what your corporate culture is, and then examine whether applicants fit that. But companies are not static. They start as small enterprises founded by highly motivated entrepreneurial folks who dine on the vending machine goodies, shave in the bathrooms, and sleep on cots.

After a while, the free-wheeling entrepreneurial environment changes into a bureaucracy, then it changes again, and so forth. Anyone who recalls the rise in dot-com businesses, or remembers how big business fares when leadership changes, knows that today’s culture-fit may not last.

I once worked for a company that hired smart, highly motivated people for plant start-ups. Two years later, the plant management complained they had “all leaders and no followers!” Be careful what you measure. You just might get it.

Statistical Sense and Nonsense

Statistics are dumb?but useful. They can tell if two numbers are correlated; but they cannot tell if one number causes the other.

This is really important if you want to develop a test that predicts job performance.

I can statistically show that blue eyes and blond hair are correlated, but we all know that blue eyes do not cause blond hair. Jan Lethen, a statistics professor at Texas A&M, cites more correlations as an example of statistical nonsense: shark attacks are correlated with ice cream sales; skirt lengths with stock prices; and cavities with vocabulary size.

When a broad sample of items are given to a broad sample of people and statistically analyzed, some correlations are inevitable. But if the items do not cause the behavior, they are bogus. They end up screening out qualified people and screening in unqualified ones.

Other problems include sample sizes. Statistics represent general trends between two groups, each of which must have the same bell-curve. Bell-curves need about 25 people at the minimum. They really work when the numbers get closer to 250.

Remember our employee-applicant difference discussion? The employee-skills bell curve would be shaped more like a finger. An applicant bell-curve would be shaped more like a ripe pimple. Comparing data with different bell-curve shapes can add major error to the numbers.

When Does Bio-Data Work?

Bio-data questionnaires provide the best results when the following criteria are met: a tight-knit group of similar jobs; a tight-knit definition of job performance; a skilled analyst interviews multiple people looking for causal bio-data items; bio-data items are administered to a large number of current employees and analyzed for performance differentiation; the test is given to a large number of applicants who are hired regardless of their scores; and after a period of adjustment, bio-data scores and job performance are statistically compared.

So, I ask. Just how predictive does the Google approach sound to you?

By the way, I don’t want to publish wrong-headed information. And I know how reporters distort facts to make a good story. So I welcome anyone from Google to post (or Todd Carlisle to address when he speaks at ERE’s San Diego conference in April) to clarify or address how they worked through these scientific questions.


37 Comments on “Using Bio-Data for Selection

  1. The article aptly lists all the factors that complicate making stellar hires. Even if the sun, the moon, and the stars merge and all is perfect for even a short time, things change. This is why some companies enter into analysis paralysis when it comes to hiring. Sometimes you just gotta go with the roster of candidates who is ready, willing, and able within a certain window of time and SHOW them what you want them to accomplish and do your best to give them the tools to do it. None of us realize our greatest accomplishments within a vacuum of complete comfort and optimal conditions. In fact, some of the world’s greatest and most lasting accomplishments were achieved in a setting of huge adversity and horrible conditions. Sometimes, you just have to do it.

  2. The search for the bio-data ‘right stuff’ can lead to a belief that the silver bullet assessment might exist. Assessment Alchemy has yet to produce the silver needed for that bullet.

    People are just too complex to describe well with a singular form of objective evaluation. Performance is also too complex to afford generalization across all jobs, as Wendell notes.

    When a piece of research on any evaluation measure includes multiple positions, across multiple functions, the result falls prey to a statistical outcome known as central clustering. The data converges in the middle, thus obscuring the unique qualities of differentiation. In essence, you measure vanilla. And for those unwilling to do the work required to measure (or create) double fudge- marshmallow-pistachio, vanilla may be OK.

    At the end of the day, measuring vanilla may be the most descriptive information obtained from a candidate, even if it has nothing to do with job performance.

    What makes one candidate fit into the corporate culture may be her or his vanilla factor: how he or she is similar to others. However, research would indicate that what predicts job performance are the traits that make one less like the corporate vanilla, and more like the flavor of the unique job.

    Measuring the ‘right stuff’ for predicting job success typically requires a multi-method approach to objective candidate evaluation, each measure revealing the appropriate ingredient. You don’t detect cinnamon with a vanilla meter. In short, make sure you measure what you need for the recipe.

  3. If we keep knocking down tried-and-true assessment methods at the current rate here at ERE (first interviews, now biodata), pretty soon we’re gonna be left with phrenology (judging people by the bumps in their scalp).

    I agree with Dr. Williams that biodata can be challenging to develop. I also agree with Joe Murphy that good assessment should use multiple measures. But there’s a lot of research out there showing that biodata can be a valuable addition to a hiring process. Let’s not discount it as a method quite yet.

    Advantages of biodata:

    1 – They’re easy to use once developed. You can administer them via computer or (if you’re comfortable with the risks), over the web. With much higher validity levels than training & experience questionnaires.

    2 – Low adverse impact.

    3 – They’ve been shown to work over many years of research. I’m sure the smart people at Google know this.

    4 – What gets included in a biodata measure has ALREADY been shown to predict performance. This is in stark contrast to most other forms of assessment where we HOPE the measure will predict performance but know only afterward if our hunch was correct.

    5 – Unlike interviews, biodata measures have a finite number of responses. Yes, it’s self-report, but I’m guessing there are far fewer ‘expert biodata answerers’ than there are interviewing pros.

    6 – Also unlike interviews, you’re not placing your faith in hiring supervisors that they know what they’re doing when they assess candidates.

    Yes, it’s probably one of the most challenging forms of assessment to create. That’s why there are firms that develop biodata measures as pretty much their whole gig. But you wouldn’t develop your own intelligence test, would you?

  4. Dr. Wendell –

    Anyone leaked what Google determined, or what works, or what correlates? Or is this Part 2?

  5. It is not often I agree with Dr. Wendell – but this article was on the nose.

    There are so many factors to play into any employment environment which can allow an individual to be successful – knowledge, aptitude, ability, environment, peers, supervisors, training and tools provided to make the job easier

    People can be the major wiz at their job in a specific type of structure business environment, but fail terribly if they are placed in business which unfortunately doesn’t match with his personality will create discord and problems later

    Yet, depending on personality testing can also prove to be foolhardy; if the company doesn’t have the tools, or the ability to use the skills of the individual they hired, especially if the applicants were accustomed to more advanced or ‘revolutionary’ tools; then boredom and frustration can also set in, if the company is unable to adapt to those modern needs. This of course can ultimately damage productivity.

    Sometime I wonder also do these tests consider the differences of gender and environment? What about the difference of ethnicity, age?

    Let’s take women for example, women have personality traits that differ from men. What makes them successful in Management or Sales will be quite a contrast from Men.. So do these tests take that into consideration? My success in Sales is based upon my compassion, warmth, friendliness… My husband is more straight to the point, let’s close the deal.. thanks..
    Not to mention Hormones and mood swings, or situations going on in a persons life at that time. How I tested 15+ year ago when I was going through a divorce, will be entirely different today! My tendencies have the possibility of changing day to day.. based upon my emotions as has my ability to ?conform? to American Society.

    Putting people in neat little categories ? whether it be based upon personality, their hobbies, personal life, whatever assuming that for example all rottweilers are killers, all basketball players are tall, and an education is the only way to prove if a person is successful in a career.

    It comes down to environment, and even that is not a predicator, one only needs to look at twins to prove that factor. Scientific Research done on Twins where they researched twins raised separately, by their biological parents, as well as adopted parents

    One example that sticks out with me ? father was an alcoholic, one twin ?became? an alcoholic, the other was not. They asked each twin why they chose the route they did, their response ? they were taught by their father. (note they were able to determine that addiction was also related to genetics from a biological parent, more so than the adopted parent in a cross research project, and there can be a greater potential for high risk environments and genetics)
    An interesting article regarding intelligent testing and the Research in Sweden ? bit outdated but very interesting read

  6. I spent three hours with Joe talking assessment on the way to the Unconference, and one nugget was that scale is vital. Wendell talked about sample size, which hints at the notion (mocked by Sumser) that at small scale, assessment can describe but not predict, while at large scale, assessment (correctly done) is well able to predict.

    Also, Wendell stated that performance differences between those ‘able to stay hired’ are small, but I think Joe would have something to say about that….

    I think this conversation must always reference scale- Starbuck’s seeking 6,000 people for the same job is one thing, but trying Bio-data in your 20 person staffing outfit just means you will be missing good talent.

  7. Another great article, WW. In my independent consulting work as well as my partnership with Charles Handler and Rocket-Hire, I never recommend biodata, though it’s supposedly as good or better than personality, etc.

    The option of biodata presents a compromise to most selection and assessment practitioners. Sure, there is a substantial record of empirical/criterion validity that points to high correlations and low adverse impact. Problem is, that’s all that it has going for it. So after nearly 15 years in the selection space, here’s why I never recommend a biodata tool to clients:

    -I hate to admit it, but I think it is simply just rude to ask applicants about things that a) aren’t related to the duties of the job; and b) are dated (over 5 years old) pieces of information.

    -Biodata inventories are perceived by applicants as not job related and often silly (especially when the question is about school, hobbies, and the like), and this can lead to perceptions of unfairness. They can’t possibly add to your employer brand either (if such a thing actually exists outside of marketing circles.) In fact, John Hausknecht’s review of applicant reactions to hiring tools ranked biodata really low, but not as low as graphology!

    -The line between a thorough qualifications screen and a biodata instrument is fuzzy when the biodata tool asks things like years as a supervisor, experience cold calling, etc. I favor the thoughtful qualifications screen, which can summarize information that often has to be annoyingly culled from resumes and vitae.

    -The line between personality and biodata instruments is sometimes fuzzy too. I’ve seen both biodata and personality tools administered in the same battery and they’re often quite similar to one another. And both often have zero validity.

    -I have seen the ‘classic’ biodata stigma with my own eyes: Positive statistics in one setting often don’t carry over to the next application. I have tons of disappointing, non-significant results for biodata sitting on my laptop next to positive results for more overtly job-related tools.

    -The consulting firms I worked for never forced me to sell it!

    Other selection tools can replace what biodata gives you and be perceived as more fair and legally defensible. Using behavioral interviews, job knowledge assessments, and simulation-based systems will prove both more applicant-friendly and valid than an inventory that asks about high school experiences, hobbies, and whatnot.

    Overall, it just doesn’t seem worth it to me and there is no good reason to favor a biodata instrument over other options.

  8. Hello Bryan,

    There are too many people cooking-up ‘tests’ they think will work. They seem to either ignore (or not be aware) of professional test guidelines.
    There is a reson for these guidelines…they ensure a test really WORKS.

    I strongly believe in Bio-data…but only when it’s done right. I wrote this article to illustrate (or warn-away) people from a similar wrong-headed approach.

  9. The core issue is that Google realized their current process was not as effective or as scalable as was needed. They took action to address both needs.

    Google set out to add an object method of evaluation, contracted with repsected professionals and conducted research, implemented a process improvement that added data and new information to the hiring decision.

    Each of these steps is laudable.

    Now Google will have data to analyze, learn from and use to make adjustments and refinements in their process. Very few orgsanization have data to evaluate. The survey I conducted with SHRM in 2004 indicated only 15% companies have candidate evaluation information in a data base. No data = No Analysis, No objective changes.

    Goolge also has implemented the first round of change management required to move from ego to evidence in hiring decisions.

  10. Would someone please explain how using irrelevant job criteria to hire the wrong people and reject the right ones is ‘laudible’?

  11. There is no perfect method for hiring the right people. Things will always change, either with the organization or with the people who work there.

    The best we can do is keep whatever selection method we use as job-related as humanly possible. Give candidates a chance to demonstrate their skills in some real-life, controlled exercises, and train our hiring managers to be better at our processes.

    Careful selection of the interview/hiring team is also critical. Everyone’s goal should be to employe people who can produce desired results without ‘leaving bodies in their wake’. Only people who have demonstrated their ability to make good hiring choices, or train employees effectively, should be part of the interview/hiring team.

    After that, it’s a matter of continual follow-up and adjusting for change. If we’re looking for a silver bullet, we will be constantly disappointed.

  12. There is no perfect method for hiring the right people. Things will always change, either with the organization or with the people who work there.

    The best we can do is keep whatever selection method we use as job-related as humanly possible. Give candidates a chance to demonstrate their skills in some real-life, controlled exercises, and train our hiring managers to be better at our processes.

    Careful selection of the interview/hiring team is also critical. Everyone’s goal should be to employe people who can produce desired results without ‘leaving bodies in their wake’. Only people who have demonstrated their ability to make good hiring choices, or train employees effectively, should be part of the interview/hiring team.

    After that, it’s a matter of continual follow-up and adjusting for change. If we’re looking for a silver bullet, we will be constantly disappointed.

  13. Before getting into the measurement of bodily responses in pre-employment screening, I have a question on the relative performance differential between ?top? and ?low? performers; because, candidly, now I?m really confused about this performance issue relative to the relationship between top and low performers production capability. Wendell Williams say ?The differences between high- and low-performing current employees ?are generally very small.? Whereas, John Sullivan?s asserts, in Rethinking Strategic HR that ?Top performers can produce 12 times more than the average worker.

    My question(s)?which is it?and what test(s) will evaluate/screen for that?

    As for the notion of creating professional test guidelines to ?ensure a test really WORKS?, consider three tests that measure bodily responses such as the Polygraphy (lie detector), Penile Plethysmography (a test that measures changes in the circumference of the penis), and Vaginal photoplethysmography (a technique for assessing female sexual arousal). There?s plenty of opinion on reliability issues over these ?tests??see limited analysis at the end of this post.

    So, with the recent press about child predators in the workplace, including female teachers; and now a former head of Virginia’s ACLU, just arrested for possession of child pornography [who also worked with children], it won?t be long before someone suggests the use of these ?tools? in pre-employment screening. As absurd as this might sound, there will be those [in the extremes] who will be intrigued by the idea. But before some fool does?they should consider reliability factors?There are better ways?caught once?lockem up and throw a way the key [deterance].

    After nearly a century of testing ‘The inherent ambiguity of the physiological measures used in the polygraph suggest that further investments in improving polygraph technique and interpretation will bring only modest improvements in accuracy.’
    Reference: Electronic Privacy Information Center

    The validity depends on who you ask. There is too much measurement variability regarding the relationship between cognitive arousal and physiological manifestation, yet it?s used extensively in both the mental health and legal systems.

    Reference: Behavioral-Forensic-Evidence: Can We Identify the Sexual Predator.
    Reference: The Penile Plethysmograph in False Allegation Cases


    While this test/devise was first developed to measure blood flow the psychophysiological assessment of female sexual function has a relatively short history in sexology.
    Reference: The Psychophysiological Assessment of Female Sexual Function

    Allow me a final comment on assessment, in a more serous vein. Being a ?headhunter? with nearly twenty years in the business I have an obvious personal bias on at least one best practice in selecting ?top? performers. To mitigate the bias, let me share with you the recent comment [one of many similar comments from over the years] from a long-time client?[paraphrased]? Brandon, it amazes me how you get into the ?emotional bank account? of organizations and their people to help identify top people and to promote one opportunity over another?.

    Empirically [in my assessment], I don?t think there?s a test built [validated or not] that can stand up to the screening rigor of a seasoned third-party recruiter.

    But what do I know?

  14. How does Bio Data Assessments hold up with the EEOC or OFCCP – When one looks at what a bai
    The ‘basic qualifications’ which an applicant must possess means qualifications criteria which the contractor established in advance. In addition, the qualifications must be:

    Noncomparative features of a job seeker (e.g. three years’ experience in a particular position, rather than a comparative requirements such as being one of the top five among the candidates in years of experience);
    Objective (e.g., a Bachelor’s degree in accounting, but not a technical degree from a good school); and
    Relevant to performance of the particular position.

    So how does a proving that Shelly loved her Dog when she was a kid show Valid evidence that it is an objective qualification?
    Is there a Hedonic cost to myself or others if I don’t like green? Eat Tomatoes? Can Sau the alphabet backwards?
    Okay, I realize that is oversimplifying the tests.. and generally they are more specific to Job Relevent information like type of schooling, job experience.. but isn’t hobbies crossing the line?
    Isn’t that delving into the area of personal? Like the fact that I prefer to hang out with the Gay Ski team? Or that I am on the Black Chess Club?
    Sure asking my degree is great, depending of course if it an Industry Standard, and Requirement — but isn’t asking where I went to school subjective? Who is to define whether my A at State College does not make me as qualified for the job as my A average at Harvard?

    These tests may have some strong possibilities of creating possible adverse impact – rather than reducing as some have proclaimed to do.. especially if one considers for example that Asians will generally score higher in Math and Science, and other races may score lower than Whites in regards to verbal ability and comprehension – of course some care can be taken to reduce some of these problems..

  15. Hi: Just to clarify, the following statement has no impact on the use of biodata assessments whatsoever.

    ‘How does Bio Data Assessments hold up with the EEOC or OFCCP – When one looks at what a bai
    The ‘basic qualifications’ which an applicant must possess means qualifications criteria which the contractor established in advance. In addition, the qualifications must be:

    Noncomparative features of a job seeker (e.g. three years’ experience in a particular position, rather than a comparative requirements such as being one of the top five among the candidates in years of experience);
    Objective (e.g., a Bachelor’s degree in accounting, but not a technical degree from a good school); and
    Relevant to performance of the particular position.’

    In reality the above OFCCP definition is merely used to determine when an employer should be requesting applicant flow data in conjunction with the hiring process. The definition does not imply that applicants may only be reviewed on the basis of such qualifications. As an aside, the OFCCP has indicated that when a pre-employment test (e.g., biodata) is administered, the employer is required to seek applicant flow information in conjunction therewith.

    I trust this clarification is helpful.

  16. Wendell,

    First the reference to, Dr. J?s 1:12 stats are in the italics at top of p.5 of Ch. 1, in Rethinking Strategic HR.

    Now, I don’t know where his numbers come from; just as I don’t know where your numbers come from. Neither of you referenced the source of the relevant research data. It could be that I simply may have misunderstood the context of one or both of your assertions. While I doubt that?s the case, I simply thought the two of you might want to compare notes and offer context for ERE readers, to clear up the obvious [or apparent] chasm between your respective positions on this issue of individual performance metrics. The consequences [to an enterprise] of using either of these assertions in talent acquisition issues suggests a difference with a distinction?possibly survival.

    As to the final point, I referenced three bodily response tests, offering another perspective on profiling. Experts make contradictory claims to their relative reliability. Yet POLYGRAPHS for example, according to expert testimony before congress, indicate this test has a high incidence of false positives, and psychopaths as well as others have demonstrated the ability to beat the test. The device?s results are not admissible in courts, as results don?t stand up to the light of day. Yet the NSA, FBI, and local law enforcement agencies continue to rely on such instruments to one degree or another in the execution of their work [combined with interrogation lies that have induced innocent people to confess to guilt]. The results of PENILE PLETHYSMOGRAPH [commonly used in the judicial system] are affected by a host of factors [age, alertness, drug interactions, health considerations, etc.] that can easily skew results.

    Certainly psychological devices like MMPI, pre-employment profiles, and bio-data analysis may well offer insights into the general attributes and characteristics of an individual relative to their fit in an occupation, or propensity to do a good job. But they offer little more than one data point. Moreover, a ?good? score doesn?t insure great, let a lone good job performance?does it?

    Let me pose a hypothetical. Two candidates both from respected competitors [one the source company taped by a recruiter], similar job experience, educational credentials, and other things being equivalent. Candidate A is considered an average performer but knocks [the] profile out of the park; whereas candidate B considered an industry super star by his peers consistently demonstrating cutting edge performance, yet offers a profile that is marginally acceptable [for what ever reason(s)].

    Candidate A, a good fit, was recently downsized, walks in the door on his own and applies for a job at the same time candidate B [who has absolutely no interest in making a move] is pursued by a seasoned third party recruiter?who is able to demonstrate [not readily apparent] characteristics and benefits associated with a move to the client company that this potential key player hadn?t considered in terms of her/his personal interests?to fill this role that?s critical to the futures success of the recruiters client and its position [maybe even survival] in its market.

    I don?t claim to be ?psychic nor the most adept person in the entire universe? [probably not in the world, USA, the County of San Diego, or even Oceanside]; but, in my personal experience, such a scenario isn?t limited to the hypothetical.

    I don?t mean to imply that the hiring process is that simplistic or static, or that profiles don?t have their place. Not to be presumptuous, but your performance indicators would suggest Candidate A is just as good a fit as Candidate B [save the five, or six, or maybe even a seven figure fee]; whereas, Sullivan?s metrics would indicate Candidate B would be not only the better choice, and a superior choice?but the essential choice [regardless of any fee].

    Given these facts, which candidate should be hired?

    What am I missing?

    Inquiring minds want to know !

    Finally, maybe referencing to third party recruiters was little more than shameless self-promotion. Whether it was or wasn?t, allow me to make clear that my intent in referring to headhunters wasn?t to suggest their omnipotence, or even superiority to any other mortal. Nor was it in any way a slight to those hard working, bright people who work in other important roles in managing human capital, including profilers. It was simply a reference to a competent third party recruiters rather well established reputation in (1) identifying [passive] top performers, and (2) ability to move them on to greener pastures. Frankly, I don?t know of a test or profile that can reach out and do that task.

    For more support to this notion of a competent third party recruiters capabilities; consider Finlay and Coverdill multi-year study culminating in ?Headhunters, Matchmaking in the Labor Market? where these two Professors of Sociology, describes ?third party recruiters? as complex, high-level front-line service workers who occupy a dual role in an unusual sales process; and characterizes ?headhunters? as the ?visible hands of the labor market? that have a ?significant impact? on conducting business, managing relationships and in making decisions that are ?extraordinarily consequential? to ?economic and organizational sociology?.

  17. A few points of clarification:

    I don’t know where Dr.J. references his 12:1 productivity estimates, but the research shows 2:1 is common with simple jobs and ‘higher’ among professionals.

    My point is that test differences between employees are generally smaller than test differences between applicants. It’s called ‘restriction of range’.

    A properly done bio-data instrument is similar to a properly done behavioral interview…job-relevant questions that accurately indicate future job performance.

    Final point: any person who says they are a better judge of future performance than scores from a validated test, interview, simulation, (or whatever) is either an accomplished psychic or the most adept person in the entire universe.

  18. Thanks Dr. Wendell,
    I spoke to the Ofccp yesterday, this was one of the topics that I asked about, amongst a few others.

    They mentioned that the Test must be verified and that they have professionals on staff to make sure that the test does not include data that could create potential for adverse impact

    The only thing that makes me scratch my head and this is my personal question, not theirs, is how does an employer reasonably prove that a person loving their dog is an acceptable and effective criteria to determine if the person is qualifed to be able to do the job?

    Isn’t this hard to prove.. so we know that Jenny is kind and caring because she loved her dog, from the time she was 3- but, maybe Jenny didn’t even take care of the Dog, and mom did, and Jenny was more interested in the Horse or Cat.. It is easy to be able to ‘lie’ in these tests, or to assume the truth, is it not?

  19. David Arnold said it best…Valid bio-data is based on job-relevant information…not cats and dogs

  20. Ah, thanks Dr. Arnold, your comments were very much on the mark, but we also know that there are companies who will institute tests w/o respect to validity to their jobs.. there are number of companies who are presenting themselves as proficient testers and will subject employers with tests that may not be focused on the employment criteria..

    So, the Ofccp does have on Staff professionals who can ascertain if the data is consistent, to the job, even if the Employer feels it is.. or has demonstrated that their ‘expert’ really was an expert.

    Re the Dogs comment, hey, it really isn’t my idea…. really, there are some Bio data tests that actually do have the question – did you have a pet, and did you like your pet.. and how did you treat your pet.. Sure you may have seen one or two w/this data on it.. But, with that point aside, I did say that my example was taking it to the ridiculous..

  21. Hi:

    I’m a bit confused by the statement that: ‘They [the OFCCP] mentioned that the Test must be verified and that they have professionals on staff to make sure that the test does not include data that could create potential for adverse impact.’

    In reality, the OFCCP is responsible for ensuring that employers who do business with the federal government comply with non-discrimination laws and regulations. And there is no law or regulation, which mandates that tests or any other hiring tool be ‘verified’. The role of the OFCCP is as follows–if the OFCCP ascertains that an employer’s test or other hiring tool (e.g., criminal background check, educational requirement, experience requirement, credit report) has an adverse impact on the basis of protected subgroup status (e.g., race, gender), it is incumbent upon the employer to show that the hiring tool is job-related and consistent with business necessity (valid) in order to continue using it. The agency employs relevant professionals (e.g., I/O psychologists) to help evaluate these issues.

    Additionally, with respect to the statement that ‘it is easy to be able to lie in these tests’, it is important to note that lying or providing socially desirable answers to inquiries is not a problem isolated to self-report assessments. Interviews, application blanks, etc. are just as susceptible to these test-taking approaches that can impact validity. Bottom line, well developed and valid assessments, interviews and other hiring tools are still extremely useful for evaluating job applicants even though a limited number of applicants may be able to perform beyond their actual capabilities due to certain self-presentation strategies.

    I trust these thoughts are helpful.

  22. Brandon, excellent points…My take on your post can be summed up with a much used and very old phrase (never mind cats and dogs, how about a horse!):

    ‘you can lead a horse to water, but you can’t make him drink’.

    The horse may be the best candidate based on Bio-Data, but how do you determine the motivation and interest level of the candidate? If they don’t want to go there, you are ‘stuck’ with candidate A. Another phrase much used, very basic, but very true:

    ‘Ready, willing, and able’

    All three are key to a successful hire.

  23. Dr. Wendell,
    Based upon an article written by the NY Times, it appears that Google does indeed ask about pets in their test
    Quoted ‘And some fell into no traditional category in the human resources world:
    What magazines do you subscribe to? What pets do you have?’ They also ask what Mailing lists do you subscribe to?
    Those do concern me, they don’t ask what professional mailing lists, but what mailing lists.. isn’t that not getting a bit personal?

    The reason given ‘?We wanted to cast a very wide net,? Mr. Bock said. ?It is not unusual to walk the halls here and bump into dogs. Maybe people who own dogs have some personality trait that is useful.?’

    It was a 300 question survey that was given only to employees who had been there less than 5 mths.. (based upon the article) – interesting

    The test was created by Mathematicians? Granted that is good for statistical analysis, but I guess this goes back to my previous comment about the ofccp – and why they would want to have their own professionals on staff — (Google is Not a government contractor)

    Well, maybe this is a bit better than having to have a 3.79 GPA.. or is it?

  24. While this is a bit lengthy, you might find it both entertaining and that it has some value in how we think about talent…enjoy.

    The following is a profile I crafted for, and about, someone I know intimately. One week ago tomorrow [2-24-07] this person and I attended a program [casting call] for a new PBS show ?Everyday Edison?s?, the second in a series of programs seeking participants with a compelling personal background and innovative product ideas/inventions. The first season is set to air in a few weeks.

    This person finished only the eighth-grade. He?s an engaging, bright, imaginative, and in many respects, talented fifty-something innovator/inventor, and song writer/vocalist. He?s been on his own since he was fifteen. Along life?s path, until in his mid-twenties he?s worked in a variety of jobs in the field of masonry, as well as, in other areas of construction trades. In his mid 20?s he was electrocuted by 7200 volts of electricity.

    Unfortunately, he?s fallen prey to circumstances throughout his life; circumstances that, by in large, have been outside of his ability to control, adjust to, or recover from. He?s survived the Machiavellian coil of a rural public education system, electrocution, and public sector services agencies. He lives on an annual income of less than $7,000.00 [roughly one-half the federal income poverty-level].

    Despite all of this, he?s overcome the challenges of Traumatic Brain Injury (TBI)?relearning facial expression, voice tone control and a host of other issues. Through it all he?s maintained a cordial relationship with his former wife and her husband. He?s done an outstanding job in guiding his two sons into productive pursuits, where his eldest son is working in construction/home improvement, and the youngest boy is in his second year of college, working part-time, and pursuing an education in video production where he?s already acquired experience producing training videos for a local fire services agency.

    Due to his disabilities including short-term memory issues, persistent headaches, and other physical health challenges that sap his stamina, he?s essentially unemployable. It?s a virtual certainty that any battery of employment profiles would rule him out for any job [ when he needs a nap…he needs a nap ]. There?s no pushing his pace in getting out the door [something that can take a couple of hours]; and getting down to business, let alone moving at the speed of business isn?t on his radar screen. Yet he?s got a capacity for high-creativity, with the productive capacity of a savant; but, with virtually no resources or opportunities to improve his lot in life. That is, no resources but one?his inventions.

    This innovator/inventor has long wanted to find a partner to produce, manufacture, and distribute many a product innovation; that are, from my point of view, all good to great ideas. What he?d want in return is a fair royalty. Unfortunately, having been ripped off in the past for an idea that made it into the market as a TV product, he?s been reluctant to expose this or other inventions without a patent. Having no money to get one, his inventions have been little more than a latent dream.

    With that context, let me share what happened at the PBS screening. After flying in from the Midwest he arrived in San Diego to lost luggage, and a missed train connection. Two days later we started out from Oceanside, CA at 5am Saturday, arriving at San Diego State University by 6. We were third in line, a line that eventually swelled to about a thousand?not good odds for anyone let alone someone with such significant challenges.

    Although most of the paper work had been filled out in advance, by the time the balance of his paper work was complete we?d sunk in line to numbers 225 & 226 [yes I presented an idea myself]. The rules allowed for presenting two inventions. He presented two inventions, while I presented one of my own; and we did another of his inventions jointly [my second one].

    To make a along story short, while neither of my ideas made it past a four panel selection team of patent attorneys and engineers, both of his made it through the first round. These judges loved the scale model he made and were especially captivated by the computer generated 3-D rendering that he personally created for the first invention he presented. It also didn?t hurt that he?d recently, somehow convinced a patent attorney to donate what had to amount to $40,000 to $60,000 in pro-bono patent application work. Has anyone heard of a transactional attorney doing pro-bono?not too likely.

    Having made it though the first round, the second round was conducted in a video/sound stage at San Diego?s KPBS studios. Here the first invention survived the second panel of seasoned patent attorney?s and engineers engaged in bring new products to market. His was one of roughly five percent of the thousand plus ideas that will now be given a thorough review for final consideration?a 120 day stand-still.

    What are the odds?

    Along with me, please wish him luck.

    The salient point of all this is that, in any other environment such a person would have been perfunctorily eliminated by his resume [should he have such a thing], a profile [test] of any kind, or even first impression?in the unlikely event he?d ever make it into an interview or selection process.

    I suspect there are a fair number of people like this in the world; people who, for what ever reason are rotting on society?s human talent vine. That?s sad, especially when someone like this, a person with true creative genius for seeing what most others will never see (regardless of the education or training one might have), is ?dismissed without a thought.

    Just think of the potential competitive advantage [ creating the next great product hit(s) ] that such a person could bring to a manufacture or product developer if left to wander the floors of the factory, and the halls of product marketing with no other mission than to offer even a fraction of the great innovative ideas that flood such an individual head every single day of their lives.

    It?s situations like this where I wonder about not only the value of profiling/testing, but the limiting bias within other selection processes as well. Call me cynical, but this kind of talent?in the off-event that any educated professional, manager, or executive might have the foresight to seek out? will never be found through any test or traditional selection process.

    I have no doubt that my little vignette could be sliced and diced as to it?s relevance to the typical talent selection situation. Nevertheless, I thought the exercise, besides being entertaining, it might prove a catalyst for thinking about our concepts, not only about talent selection; but about what constitutes talent itself.

  25. My reference to the 2:1 ratio came from a book by Adrian Furnham…who cites many, many peer-reviewed studies. As an Industrial Psychologist who measures and identifies skilled people for a living, I also reference dozens of examples from many of the Fortune 500. This information is not secret. It can be found in any well-researched book on I/O Psychology.

    Why is there no productivity standard? Few jobs are alike, bosses change, and so forth…Suffice it to say simple manufacturing or processing jobs are easier to quantify…professional, knowledge worker and managerial jobs are not. On the other hand, people with the job-skills and job-motivation tend to outproduce those who do not. But that’s just common sense, again.

    As to the three ‘tests’ you cited: the MMPI was not designed to be a pre-hire predictive skills test; people who ‘fail’ the polygraph are either able (or unable)to control nervous-system responses to emotional stimuli. Penile devices? You seem to know much more about them than I do …so I cannot comment.

    If you are unclear about how to identify the most-skilled person in a job, then I suggest you read the 1978 Uniform Guidelines. It is the ‘Best-Practices’ of recruiting and employee placement. Most recruiting questions are discussed there…including the ‘skills for the future’ question you raised.

    I raise a toast to ANY recruiter who follows the ’78 Guidelines…the rest need to go back to school to learn about tests and measurements (yes, the Guidelnes defines recruiter interviews as ‘tests’). Recruiters can run, but they cannot hide.

  26. Dr. Wendell
    Re ‘suggest you read the 1978 Uniform Guidelines. It is the ‘Best-Practices’ of recruiting and employee placement. Most recruiting questions are discussed there…including the ‘skills for the future’ question you raised.’

    *******EXCELLENT RESPONSE!!!!!!!!!!*********
    Probably the Best Advice ever given to date on this forum

    Guidelines can be found –


  27. Dear Brandon,
    Thanks for an incredible story… we all occasionally need to be reminded that, ‘There but for the grace of God, go I.’ At the same time, I have that awful feeling of ‘What am I supposed to do with this knowledge?’

    Looking up from our desks we see the news chronicle the excesses, the heartlessness and the greed we’ve almost come to accept as routine. And, anyone mad as hell and not going to take it anymore is – unemployable. We are on a runaway train, and we know the bridge is out somewhere up ahead.
    Best regards

  28. Dr. Williams (forgive me for the Dr.Wendell error) or Dr. Arnold..

    The last three posts brought up interesting comments.. and maybe you may be able to help us understand Reasonable Accomodation and Testing

    Okay, I understand that reasonable accomodation must be made in reference to test taking..but does the tests actually take into consideration ability to do the job with reasonable accomodations.

    I for example have ADHD.. true blue diagnosis. So there is some VERY limited protection for me under the ADA – now when it comes to tests, I blank.. can’t do it, no matter how much stuff I know.. but, if asked questions, in a non Test environment, will ACE! No problem.. This has been proven medically..

    Also, more time is needed for me with regards to completing certain projects, whilst other projects I can whiz through with not even a second thought..

    Get me in an office w/limited distractions, no problem can work well alone, but with distractions, and well..

    Companies have found ADHD people to have been some of the most creative and diligent workers, when accomodated.. work that they are excellent at is given to them, and the other is shared amongst other workers..

    Do tests keep this in mind, as we see the numbers in Adults with ADD/HD grow immensely as Scientists are Finally admitting that one does not grow out of the ‘disability’ (said loosely) by the time one is 14?

    Please Advise

  29. Bravo, Brandon! Employing individuals with disabilities has presented employers with challenges on numerous levels – but the biggest and most challenging is getting employers to see beyond the immediate scope of ‘they won’t be able to do that job.’ There are scores of individuals who are available and WANT to work – they don’t want SSDI, they don’t want handouts. They want an opportunity to earn a decent living.

    I think there’s an innate fear of having to handle someone differently than the rest of the employees, will I lose customers because this individual has a condition that maybe my customers will lose patience with, etc. If given adequate consideration, many employers can find ways to utilize the skills of an individual with a disability that they may never have considered in the past.

    Some folks could ace a test, but once an employer sees the extent of the disability, they may lose interest in the candidate. Maybe the candidate cannot work in a high-volume paperwork environment – but maybe if the paper is decreased and computer programs are utilized, you’ve got yourself a great candidate. Or maybe, they have a learning disability that affects their test taking ability – but they are the best customer service representative you could ever hire.

    Tests are but one component of the selection process, and I’m confident that most employers are relying on more than that – but it never hurts to remind everyone that there are some golden opportunities out there for those that can think of work in a different way.

    How about hiring a blind person? They can operate computers using special software to read the text – it’s not as expensive of an accommodation as most would think. How about hiring a person who is deaf? They have TTY phones so that they can communicate with callers. How about hiring someone who is paraplegic, or someone who has brain trauma injury, or one of any other number of disabilities? There are ways to employ individuals with disabilities, but first you have to open your mind to thinking outside the box in structuring work.

  30. Thanks David,
    for taking the time out to answer the questions.. But, I am still confused. Isn’t the company mandated to make reasonable accomodation (depending of course on the number of employees or causes undue hardship)

    Thus is the test also going to consider some disabilities more than others. You mentioned Physical disabilities, but what about other disabilities.. disabilities that for example will not prevent an individual from doing the job, doing it well, but only with reasonable accomodation.

    Let’s take Googles test for example, let’s say that though I do like Dogs, but due to being bitten, I get anxious around them, but I am able to do the work well.. Isn’t then a reasonable accomodation to allow me to be able to have the position, but keep me away from the offices that have pets? or will dis-allowing pets in the office be consindered unreasonable and an Undue Hardship to Google?.. (can think of many people who don’t like being around pets)

    There was an interesting case regarding testing and accomodation — echostar vs eeoc/Alton – EchoStar failed to accommodate Mr. Alton in the application process; EchoStar failed to accommodate Mr. Alton in the job by never trying to install adaptative software;
    EchoStar denied Mr. Alton an employment opportunity because of his disability or because of the need to provide him an accommodation; and
    EchoStar violated a section of the ADA when it failed to use a proper testing device to determine an applicant?s skills. — Not installing 35 dollar equipment related to an 8 Million Settlement..

    I guess, the questions I have are – if after a person takes a test, can they be given an opportunity to explain that w/ reasonable accomodation that answer should be ??? Is there an exception to the Ruling w/ADA in regards to testing?

    Should disabilities be presented before or after initial interviews?

  31. Hi:

    The ADA requires reasonable accommodation for the following reasons: 1. To ensure equal opportunity in the hiring process; 2. To enable qualified individuals with a disabilty to perform essential functions of the job; and
    3. To enable an employee with a disability to enjoy equal benefits and prvileges of employment.

    With respect to the hiring process, reasonable accommodation is an important issue with respect to tests, interviews, job simulations, application blanks, physical agility tests, etc. With respect to the relationship between the hiring process and reasonable accommodations made to the job, it is obvious that such accommodations would impact how and whether a selection tool was used. For instance, if an employer intended to accommodate a propsective employee by not requiring driving for a job that typically required driving, the interview would not contain a question about whether the applicant has a driver’s license and a motor vehicle report would not be procured. Similarly, if an employer intended to accommodate a prospective employee by removing the customer contact component from the job, the interview would not include questions about customer service experience and the employer wouldn’t administer an assessment used to evaluate service skills.

    I trust this information is helpful.

  32. Hi:

    A few final comments regarding this matter from my perspective. At trial, in EEOC v. EchoStar Communication Corp., the plaintiffs presented evidence that a blind applicant applied for a customer service representative job at EchoStar in 1999. Prior to applying, Mr. Alton (the applicant) had completed training at the Colorado Center for the Blind for that very type of position. Blind individuals can perform the customer service representative job by using a computer program called JAWS (Job Access With Speech), which translates text into speech. A blind customer service rep uses a split headset, in which he hears the JAWS voice in one ear, and the customer conversation in the other ear. Using JAWS, people with vision impairments can process written language at 400 to 700 words per minute, which is faster than many sighted individuals read. At trial, the plaintiffs’ expert, Nelson Reiser, demonstrated to the jury how JAWS works.

    When Mr. Alton first went to EchoStar to apply, EchoStar told him it would not do him any good to put in an application because they were not set up to handle blind people. However, after receiving a copy of his charge of discrimination, EchoStar invited Mr. Alton back and put him through a sham interview process that included a Braille test, which was longer and more difficult that the test given sighted people, and a Windows skill test that consisted of a person giving him directions on how to access icons, such as ‘move to the left, move down, now click.’

    Much of the testimony related to whether, if EchoStar had tried to install JAWS in 1999, it could have worked. EchoStar asserted that JAWS could not have worked because of the complexity of the software environment. Contrary evidence presented by plaintiffs reflected that, in 1999, numerous employers in Denver such as Norwest Bank, American Express, Big Brothers/Big Sisters, and MCI had installed JAWS and employed blind customer service representatives at their call centers.

    Finally, the EEOC has a number of guideline publications that are helpful in analyzing specific situations to ascertain what type of hiring and employment accommodations are reasonable.

    I trust this information is helpful.

  33. Here is an interesting case/interpretation regarding Dog Phobia, and disability/employment .. as I was wondering the same thing..

    Okay, so we are speculating here right.. so – If I had proven that I had a phobia/allergy/asthma/anxiety from dogs that meets DSM-IV or DSM III- R criteria –

    My personal thoughts, would it be unreasonable to ask a company to A) put me in an area that would not be around pets, b) Have me work from home if applicable c)if it is a short term scenario, remove the dogs temporarily, whilst I me go to a doctor and get treatment..
    D) since dogs are not naturally in an office environment, and not a normal scenario, and heck, statistically it can be proven that their being present isn’t going to improve or reduce the quality of work, and really don’t have anything to do with the Qualification criteria of the job – unless of course it was a dog washing business or vet, and there are probably a large number of people in America who may have a fear of dogs, so this really isn’t unreasonable to ask to move them completely, or is it?

    These statements were made in the Probable context of an ADA reasonable accommodation analysis rather than in an ADA major life activity analysis

    Now, maybe I am going to extreem here.. what about this angle too, since we are going with a pet question, say if a company does ask if one owns a pet, could this also not lead to a potential for adverse/disparate impact–

    2000 and 2005 census data In America –
    Among racial demographics, European Americans/white had the countries highest homeownership rate 71.7, while those identifying as being Hispanic or Latino had the lowest homeownership rate 47.0, not far behind from Blacks 47.4 home ownership usa home owners by race

    It would be safe to assume then that the Majority of Dog owners will be home owners, as many rentals don’t want pets.. and with Whites being the Majority home owner.. so would asking about Dog Ownership, if that was a specific question on a test not also create a potential for Disparate/Adverse Impact?

    Makes me think about Griggs V Duke power and the need for highschool diplomas for blue collared jobs..

    Also let’s also say if a company asks about your subscriptions, without defining the words Business, then could that also create a potential of invasion of privacy, or even disclosing more about ones race, and affiliations??

  34. ADA also requires accommodation during the interview process. I’ll preface my response by saying that our agencies work with individuals with disabilities and disclosing the disability probably works in the applicant’s favor! Typically when we interview folks, we ask during the phone call to offer an interview whether the applicant has any special accommodation needs for their interview. We are accepting their request at face value whereas once employed, we require specific disclosure to assure we are dealing with a qualified disability.

    On the dog issue, you’d have to determine whether the level of anxiety meets the definition of a qualified disability under the ADA – there’s simply not a guidebook as to universal acceptance of this disability or that for every situation. I may be anxious but not to the point of it affecting ADLs. I may have allergies which don’t meet the qualified disability definition, but my co-worker may have a severe respiratory reaction to dogs – she’s qualified for a reasonable accommodation and the agency would have to set up something to accommodate that worker.

    Hope this helps!

  35. Thanks, David, for further definition of the case. I work with several disability services agencies. Many folks fail to realize that individuals with disabilities are truly employable or that most accommodations are made for less than a couple of hundred dollars. JAWS has been around for quite some time, and their upgrades have made it possible for many previously difficult programs to be much easier for blind or vision impaired individuals to read. Some of the difficulties that most web designers fail to consider in reaching these types of consumers or applicants – graphics, extensive tables, photos, PDF files, etc. And voice recorded programs without text files do not reach your hearing impaired consumers or applicants.

    What are some of our former blind clients doing now? Working full-time jobs as customer service representatives handling heavy computer and telephone contact, screening applications, programming, managing stores – practically any job that anyone else does, and with lower turnover. Our Industries for the Blind hires individuals who assemble mops, pillows, mattresses, t-shirts, package spices, assemble pens, etc. These items are on contract to state agencies and colleges/universities, federal goverment, military.

    Thanks for highlighting the case for employment of individuals with disabilities and the relatively inexpensive solutions for accommodations. If those folks weren’t going to require Braille in the workplace, they shouldn’t have tested him in Braille. Furthermore, it would make good sense that if you’re not sure what you can do, contact the state’s rehabilitation agency for assistance. They generally have assistive technology departments that can provide good information and JAN (Job Accommodation Network) can provide good resources for accommodations.

  36. Eric,
    It may not be an all or nothing situation, depending on how the work stations are set up. We’ve made accommodation for an individual who needed a service dog which triggered another employee’s severe asthma. We purchased air purifiers, housekeeping vacuumed nightly, and the employee with the dog was working in a location that could be blocked off away from the other indiviudal. Each situation is a unique consideration which reflects that individual’s disability, what ADLs are affected, the work environment, the essential duties of the job, what accommodations are available, and what the costs are.

Leave a Comment

Your email address will not be published. Required fields are marked *