Measuring the Quality of Those You Didn’t Hire –- Are You Missing the Best?

The quality of those not hired is the most valuable recruiting metric that you have never heard of! It informs you how often your organizations is failing to hire the highest quality applicants.

A few years back I was advising a Fortune 100 firm that had a painfully slow and somewhat arrogant hiring process. To demonstrate the negative impact of their process I had to prove to a skeptical senior manager that they were letting top candidates get away. I asked a manager hiring for an important job to rank, in order of quality, 100 applicants who had been sourced for the role. The chosen rank was discretely written on the back of paper copies of the candidate’s resumes. Months after the role had been filled, the manager was asked if they were satisfied with the hire. He was, and felt quite certain that he had successfully hired a “top 5” candidate. After hearing of his satisfaction I had him look at the initial rank he had provided the candidate who was later hired: 75.

You can imagine his shock when he realized that the hiring process had somehow let every single one of the top-ranked applicants that the firm had prided itself in hiring “every single time” slip away. Clearly the quality of the people who they didn’t hire was significantly higher than the quality of the one that they did.

Selecting HR Metrics Is Unfortunately Not a Scientific Process

Most organizations adopt metrics based on those covered by benchmark reports or that can be easily enabled via their technology providers, instead of determining what they need to discover or prove. As a result, many organizations are burdened with data and reports that offer little in the way of guidance helping them improve their effectiveness. One metric often not fully taken advantage of is quality of hire, which I estimate less than 40% of organizations even attempt to use. Even fewer use the quality of hire derivative, quality of those not hired, because it can very quickly demonstrate how poorly a process performs.

Determine Where Your Recruiting Problems Are Occurring

During an advisory conversation with a recruiting leader at a well-known social networking firm experiencing difficulty achieving hire diversity, I asked “at what step or stage is your recruiting process failing?” I wasn’t surprised when he responded “we don’t actually know, we just know that the overall recruiting process is not producing the results we need.” Like many organizations, this organization lacked well-thought-out metrics that enable both performance reporting and process diagnostics.

Recruiting processes fail because either they do not attract enough top-quality candidates up front, or they fail to accurately identify, assess, and sell those attracted on the job at later stages in the process. Most organizations focus heavily on measuring sourcing effectiveness, but ignore the later stages of the process altogether. One benefit of using a “quality of those not hired” metric is that it focuses exclusively on the back end, where I estimate at least 50% of those organizations not meeting their goals have problems. If you doubt that the problem is post-attraction, ask your favorite agency or executive recruiter what percentage of qualified candidates are lost due to slow or ineffective actions on the part of hiring managers and corporate recruiting processes.

One of the purposes of the quality-of-those-not-hired metric is to force organizations with a high percentage of quality hires slipping away to identify where in their process the talent opts out or gets dropped. There are six post application stages where firms lose top candidates, including:

  1. Resume screening process — the ATS, a recruiter, or a hiring manager mistakenly screens out top applicants.
  2. Telephone screen — top applicants rank poorly on their phone screens or their screen cannot be completed, so they are dropped from consideration.
  3. Interview scheduling — they get frustrated over the number of interviews and dropout or they cannot complete them in time because of scheduling conflicts.
  4. Interview assessment — they voluntarily drop out before the interviews can be completed, or the interview process mistakenly rates them poorly.
  5. The offer process — either the process fails to include most of the top applicants on the list of finalists, or they reject the offer.
  6. Reference checking — even though they are high-quality candidates, they somehow fail the reference/background check.

Since the goal of a good metric is to help you identify what is not working, carefully select and implement at least one metric that can point out failures occurring during the latter stages of your recruiting process.

Focus Only On the Top Candidates

A quality-of-those-not-hired metric can become cumbersome if it attempts to categorize the quality of every applicant who doesn’t get hired. In order to save time and money, narrow your focus to the strategic issue of “what happened to the cream of the crop?” Out of 50 applicants for a single job, there might only be three who were so qualified that a hiring manager would actually regret failing to hire them. I call these individuals “regrettable misses,” and it is these folks that the quality-of-those-not-hired metric aims to highlight.

Action Steps for Developing a “Quality-of-Those-Not-Hired” Metric

If you decide to implement a quality-of-those-not-hired metric, there are several action steps to consider, including:

Article Continues Below
  • Setting goals — I recommend that you set goals for the use of this metric that include: accurately identifying the top three to five “regrettable” candidates; determining what percentage of top candidates become finalists for the position, and determining what percentage of new hires came from the top candidate list.
  • Select an evaluation range — this metric should focus solely on reporting the progress of “the very top applicants” who senior managers would regret not hiring. To limit the scope of evaluation, preselect what size of candidate slate will be evaluated. For most jobs, three to five top applicants would be a sufficient number to track. You can also use a set percentage of all applicants (i.e. top 10%) to define what you mean by top.
  • Determine when to identify top applicants — identify the top applicants early on in the hiring process so that you will have time to address any issues that emerge before a final hiring decision is made. If you are conducting an audit post hire, you need to make sure that the person doing the initial selection isn’t aware of which individuals were finalists and who was hired.
  • Select methods for identifying top applicants — the best method for identifying the top applicants is to have multiple evaluators select a finalist slate that is then merged to create the sample that will be monitored. An alternative approach involves using the profile matching capabilities of your ATS to produce a listing of top applicants. A third possible list segments applicants who come from high-value benchmark firms.
  • Report the metric in percentages — the best way to report the quality of “those not hired” metric is in percentages. For example: 66% of all finalists came from the top-ranked list, and 47% of the time a top-five-ranked candidate was hired.
  • Identify the stage where top talent slips through — for high priority and mission-critical jobs, after the hiring process is complete, identify at what specific stage in the recruiting process did a top applicant opt out or get dropped from consideration. You can then use that information to improve that stage.
  • Identify cause for top candidate removal from consideration — if a significant number of top candidates opt out or are dropped from consideration without becoming finalists, follow up and find out why. If your process screened them out prematurely, recruiters and hiring managers must be questioned to identify what knockout criteria is being applied. If the candidate dropped out on their own, they need to be questioned to see if their early withdrawal could’ve been prevented.
  • Keep in touch — separate from the process of calculating the metric, the organization should keep in touch with and build a relationship with the high-quality applicants who you regret missing. Building this relationship will help to ensure that they will favorably consider another opportunity with your firm in the future. Develop an alert system so that the star applicants can automatically receive e-mail alerts whenever a relevant job opens up.

Sample “Quality of Those Not Hired” Report

Here is a sample report illustrating what a recruiting leader or hiring manager might see.

Job Family: ASIC Engineer

Report Period: Q3 2010

Hire Volume: 53

Metric Job Family Organization
Percentage of top candidates in finalist pool 53% 52%
Party responsible for removal from consideration
–candidate filtered out by recruiting process 17% 48%
–candidate opted out of recruiting process 83% 52%
Percentage of top candidates who rejected offer 95% 22%
Percentage of hires from top candidate slate 31% 42%
Process stage contributing largest slate loss Interview Scheduling

Recommended Actions

Candidates reported that more often than not interviews would need to be rescheduled because times initially proposed were no longer available upon confirmation. Many candidates reported that it took recruiting coordinators longer than a week to confirm meeting dates and times. Solution:

  1. Allocate dedicated time slots to recruiting activities that cannot be booked by other activities more than 24 hours in advance.
  2. Establish service level agreements that call for manager response to scheduling inquiries within four business hours.

Final Thoughts

If you were a competitive fisherman participating in a pro fishing tournament and you repeatedly landed prize-winning fish, you would be justifiably proud. However, if you repeatedly caught prize contenders but lost them prior to tournament completion, wouldn’t you want to know exactly where and why you kept losing them? That is exactly what the “quality-of-those-not-hired” metric tells you. It reports how often you successfully land a great applicant, but fail to convert them to employee. Your organization can’t attain the highest level of new hire on-the-job performance (quality of hire) if your process allows the highest-quality applicants to be missed.

Dr. John Sullivan, professor, author, corporate speaker, and advisor, is an internationally known HR thought-leader from the Silicon Valley who specializes in providing bold and high-business-impact talent management solutions.

He’s a prolific author with over 900 articles and 10 books covering all areas of talent management. He has written over a dozen white papers, conducted over 50 webinars, dozens of workshops, and he has been featured in over 35 videos. He is an engaging corporate speaker who has excited audiences at over 300 corporations/ organizations in 30 countries on all six continents. His ideas have appeared in every major business source including the Wall Street Journal, Fortune, BusinessWeek, Fast Company, CFO, Inc., NY Times, SmartMoney, USA Today, HBR, and the Financial Times. In addition, he writes for the WSJ Experts column. He has been interviewed on CNN and the CBS and ABC nightly news, NPR, as well many local TV and radio outlets. Fast Company called him the "Michael Jordan of Hiring," called him “the father of HR metrics,” and SHRM called him “One of the industry's most respected strategists." He was selected among HR’s “Top 10 Leading Thinkers” and he was ranked No. 8 among the top 25 online influencers in talent management. He served as the Chief Talent Officer of Agilent Technologies, the HP spinoff with 43,000 employees, and he was the CEO of the Business Development Center, a minority business consulting firm in Bakersfield, California. He is currently a Professor of Management at San Francisco State (1982 – present). His articles can be found all over the Internet and on his popular website and on He lives in Pacifica, California.



4 Comments on “Measuring the Quality of Those You Didn’t Hire –- Are You Missing the Best?

  1. John, I like it. Finally, a worthwhile and useable metric. Companies are truly missing the boat on top drawer talent (I use the term talent, rather than candidates, because the term ‘candidate’ can connote an ‘applicant’ in a lot of HR departments. If an applicant/candidate doesn’t go through all of their gates (several phone screens, different rounds of interviews, etc.), they are thrown out of consideration. Usually, someone top drawer doesn’t have the patience for this bureaucratic mindset, and they either flub something (perhaps not sounding perky enough to the junior recruiter on the initial phone screen) or they lose patience with speaking with people not empowered to make something happen. Even when it goes well, it is usually accompanied by a lack of follow through.

    A suggestion that might help matters is the following:
    After analyzing the above process flow, the recruiting/HR/line organization should put together a rapid response team, geared at tackling the top positions to the organization in an aggressive fashion. The concept here is to reduce the time from a need for someone great to getting that individual onboard and acclimated. Great people don’t just wait around. They are often happy where they are and they need a compelling reason to make a change. Work on a compelling reason and an aggressive process to close the right individual.

  2. To add to many of the excellent points Dr. John makes, I would suggest a “motivation meter” for each person in the talent pool that would measure the interest level of the contact. The person managing the hiring process would be responsible for checking the contact’s motivation gage after each step in the hiring process, and passing on the level to each person involved.

    Obviously, motivation is different for someone actively looking for work versus someone that is merely dipping their toe in the water. It is my experience that most companies forget about this once a person applies for a job. It is critical to measure motivation levels for the top 10% of the pool who are typically attracted more to the challenges than for any other reason – and providing a bit more hand holding to get them to end of the process where they still say, “Yes.”

    By managing the motivation level, top talented candidates can be reeled in more often. “Managing” motivation is no different than Dr. John pointed out for the candidates you have lost – you may just want to do it before you lose them…

  3. John, you continue to ask great questions and invite thoughtful consideration. Thinking and acting are very different, just as rating and evaluating are different.

    Having hiring managers rate quality would be an interesting exercise. If nothing else, the biases at play and anecdotal elements of work history that are valued might surface.

    The single most meaningful measure of quality in the staffing process is on-the-job performance.

    John is calling for greater analytical literacy. It is a great call. There are a wide range of resources for practitioners who want to add more metrics of meaning. The quality of not hired however may not be the best place to start. You might not like what you learn. Can you handle the truth?

  4. The January hiring numbers speak volumes about the problems in HR today: 150k+ hires might sound impressive…..but in the background of 3.7 million job postings: its pathetic.

    Not even a 10% fill rate.

    Imagine the impact on the economy if all these roles were actually filled?

    Corporate hiring is the single most broken division in nearly every company today:……I suggest because its the one segment in all firms that is still operating on policies and procedures and mentalities that are at least a decade or two behind the times.

    Until HR has actual accountability at the boardroom table for delivering revenue, there will be no urgency in finding top talent and keeping vacancies at the lowest possible rate.

    Instead HR will continue to plod along as the administrative black hole that functions at the pace of dinosaurs

    Despite every available technological tool to find and place best talent: rates to fill are actually getting worse!….while HR instead focuses on how to save recruiting dollars.

    The entire structure of most HR departments needs to be overhauled and most of the layers dismantled. It has become the bureaucracy that delivers far less value than it costs to support and the lion in the cage that most senior business executives simply want to placate and avoid….as in “HR says we cant do that”

    HR needs to grow from being purely a set of “brakes” and become a talent “engine”…….but don’t hold your breath. Its not the nature of the beast.

Leave a Comment

Your email address will not be published. Required fields are marked *