Results from the 5th Annual Rocket-Hire Online Screening and Assessment Usage Survey

Article and research by Charles Handler and Mark C. Healy

For the last five years, Rocket-Hire has surveyed talent-management professionals who use Web-based screening and assessment products to enhance and quantify their hiring processes. Our goal for this research is to document trends in the usage of online screening and assessment tools in order to help provide a clear understanding of the popularity of these tools and their perceived positive and negative attributes.

Late last year, we again asked recruiters, HR staff, entrepreneurs, and hiring managers to tell us about their use and opinions of typical hiring instruments. This article highlights our major findings.

Today’s report is just a brief summary of the results of this study. For more data and an in-depth analysis of these trends, check out your May issue of the print publication, the Journal of Corporate Recruiting Leadership.

Overall, our findings from this year, and the trends that emerge over the five-year life span of this survey, continue to be positive. We had the largest number of respondents yet this year (141 recruitment and hiring professionals completed the survey, just a tad above last year’s 136).

The increase in use of screening and assessment tools and the positive benefits of these efforts continue to notate a strong trend toward their popularity and value.

On the negative side, results clearly indicate a continued lack of proper evaluation for these tools, which is even more discouraging when one considers that lack of understanding of the benefits of screening and assessment is still a primary reason for the failure to adopt them. Thus, the need to demonstrate the value of assessment by including evaluation as a core part of implementation is crystal clear and, in our opinion, remains the No. 1 challenge related to the use of assessment.

Now on to a more detailed review of our results.

Use of Applicant Tracking Systems

As with previous surveys, a majority of respondents employ an applicant tracking system. Specifically, 79% already have or will soon have one deployed, or are considering installation. As can be seen in the table below, no matter what the hiring demand, typical ERE readers are familiar with using an ATS.

Table 1. Use of Applicant Tracking Systems by Hiring Demand

Use of Prescreening Tools

About 57% use online prescreening to sort through their job applicants. We defined prescreening tools as:

Tools that gather information about, or ask candidates to respond to questions about their experience, skills, and qualifications in order to identify if they meet minimum job requirements. These tools are typically used early on in the staffing process.

The table below captures the usage rates of each of the most prevalent forms of screening.

Table 2. Usage rates of common prescreening tools

Common tools in use include the popular qualifications screen, technical skill certifications, and analyses of “fit” with company culture. Every year, it seems that more organizations are using prescreening as part of their online recruiting strategy. But the breadth of this practice varies, with some companies using these tools for all jobs while some use it for one or two alone.

Table 3. Extent of Prescreening Deployment

Use of Assessment Tools

In contrast to screening tools, online assessment tools were defined as:

Scientifically based screening tools that look more deeply into a candidate’s abilities, interests, and skills. These tools include personality measures, cognitive tests (i.e., verbal and quantitative skills), situational judgment tests, job simulations, etc. These tools are typically used for a more in-depth evaluation later on in the staffing process.

Underscoring the growing increase in the popularity of these tools, most common forms of online assessment showed an increase in rate of adoption over previous years, as comparisons with 2002 reveal.

Table 4. Usage Rates of Common Assessment Tools (2007 vs. 2002)

Specific skills and knowledge certifications and personality inventories are common assessment strategies once the pool of one’s qualified candidates has been reduced. Indicators of cultural “fit” and various cognitive ability measures are also widely used.

For another angle on the extent of assessment implementation, we asked respondents to indicate the level of jobs for which assessment is used for evaluating candidates.

Table 5. Job Level of Assessment Implementation by Job Level

2007 appears to be the first year that our data show a general pattern of use across all levels of an organization. Nonetheless, executive-level openings tend to entail more personal and direct methods of selection and placement, and were less likely to involve the use of any online tools.

Article Continues Below

Effectiveness of Screening and Assessment Tools

This year, 65% of prescreen users and 77% of assessment users felt their tools added value to their organization. In contrast, a full 21% of prescreen users and 10% of assessment users felt these tools did not add value. But this is difficult to judge, especially if one does not take the necessary steps to adequately answer the question. In this case, only 27% of users of prescreening tools and 36% of assessment users collected metrics to judge the quality of their interventions. Moreover, users of metrics tend to report success with prescreening and assessment.

Table 6. Perceived Effectiveness of Prescreening Tools

Table 7. Perceived Effectiveness of Assessment

Clearly, collecting metrics helps to answer questions about effectiveness and, interestingly, provide a positive response to these questions.

The Future of Online Screening and Assessment

Of those companies not currently using screening or assessment instruments, 42% feel they will implement one or more in the future. For them, Table 8 summarizes the sorts of tools under consideration.

Table 8. Screening and Assessment Tools under Consideration

As with the tools current users are seeking to use more of (or expanding to include in their hiring systems), qualifications screening, assessment of cultural fit, skill/knowledge assessments, and personality inventories dominate interest. Still, the mystery surrounding the value provided by these tools, as well as a prevalent intimidation factor, keeps companies from jumping in.

Obstacles to the Use of Online Screening

As usual, we asked users of online hiring technology (as well as non-users), what they perceived to be the biggest obstacle to the adoption of prescreening and assessment in their organization. Again, we received nearly identical responses to previous surveys.

Table 9. Single Biggest Perceived Obstacle to Deployment of Online Tools.

While 18% of users feel there are no obstacles at this time, many hirers feel that one or more roadblocks prevent prescreening and assessment from being integrated into the staffing process. These reasons mostly reflect a lack of knowledge of the benefits of more in-depth hiring techniques. Moreover, there appears to be a general lack of knowledge of safeguards and metrics designed to detect problems when candidate evaluation systems are in place, even among regular users of assessment.

But do these worries represent valid concerns, or is it simply fear of the unknown? Arguably, when assessment tools are properly matched to the demands of the open position, the only relevant concern is applicant identity (e.g., the stereotypical “smart older sister” taking a math test for an applicant) and test content security. But candidate enhancement of qualifications is common on resumes, in interviews, and for some personality traits/scales. Since many of the same concerns exist for paper-based processes, this should not limit any one organization from deploying Web-based hiring technology. Overall, most companies may legitimately cite few or no substantive reasons to limit their use of online recruitment and hiring techniques.


This year’s results let us know what’s really going on, and it confirmed what most of our previous surveys have found:

  • Having an ATS installed is now nothing particularly innovative, especially if you work in a medium or large organization. For some, it’s hard to even imagine the old stacks of resumes and bulging file folders of applicant information.
  • Prescreening and assessment are continuing their gradual penetration into the mainstream of recruitment and hiring.
  • Qualifications screening, personality inventories, and skill and technical certifications continue to be the most popular online assessment tools. Assessments of cognitive abilities as well as fit with the company culture have expanded their footprint as well.
  • Respondents report a lack of understanding, weak budgets, or a general lack of support for online tools as their primary obstacles to adoption or greater use of modern prescreening and assessment technology.
  • Those organizations that formally evaluate their hiring practices tend to support the use of prescreening and assessment, but a large number of organizations are still failing to evaluate the effectiveness of their screening and assessment tools.

There clearly is a growing interest in scientifically derived hiring tools. This trend is encouraging but not particularly surprising; these numbers will continue to grow as more organizations understand the value in properly evaluating the impact of screening and assessment tools, seeing for themselves how quality hiring tools can improve a workforce.

As both authors have seen over a combined 25 years in screening and assessment consulting, skepticism about the value of these tools remains strong. We will continue to champion the use of evaluation to “close the loop” and reveal how more in-depth evaluation of candidates may positively impact the bottom line.

A word about our methodology: Respondents were evenly representative of recruiters, recruiting leaders, HR executives, business owners, and hiring managers, and represented a wide variety of organizations and hiring situations. Specifically, 59% were employed by organizations with 500 employees or more, with 34% working for a company with 5,000 or more employees. In contrast, about 20% were involved with recruiting for companies with 50 or fewer employees. In terms of employee acquisition activities, 34% hired more than 500 workers in a given year, but a majority of respondents (66%) made 500 or fewer hires per year.

Dr. Charles Handler is a thought leader, analyst, and practitioner in the talent assessment and human capital space. Throughout his career Dr. Handler has specialized in developing effective, legally defensible employee selection systems. 

Since 2001 Dr. Handler has served as the president and founder of Rocket-Hire, a vendor neutral consultancy dedicated to creating and driving innovation in talent assessment.  Dr. Handler has helped companies such as Intuit, Wells Fargo, KPMG, Scotia Bank, Hilton Worldwide, and Humana to design, implement, and measure impactful employee selection processes.

Through his prolific writing for media outlets such as, his work as a pre-hire assessment analyst for Bersin by Deloitte, and worldwide public speaking, Dr. Handler is a highly visible futurist and evangelist for the talent assessment space. Throughout his career, Dr. Handler has been on the forefront of innovation in the talent assessment space, applying his sound foundation in psychometrics to helping drive innovation in assessments through the use of gaming, social media, big data, and other advanced technologies.

Dr. Handler holds a M.S. and Ph.D. in Industrial/Organizational Psychology from Louisiana State University.







3 Comments on “Results from the 5th Annual Rocket-Hire Online Screening and Assessment Usage Survey

  1. As a user of on-line assessment tools, one of the key components in my evaluation is that of ‘validation’. The empirical backup for any tool is really key… an employment agency recently told me they do ‘in house validation studies’ – translated they informed me it meant if their customers ‘liked’ the tool. Granted, I’m biased towards the Profile XT — because of the extensive validation and reliability studies,.. combned with post use metrics validity can be powerful uses of empirical evidence to evaluate these instruments.

  2. The kind of assessment I use in my consulting measures judgment and decision-making ability. Specifically it measures the ability of the person to perceive clearly and to translate that perception into a decision that is appropriate to the situation. Basically, that is the core talent a company “buys’ in an individual, at all levels.

    Naturally you can’t use the usual self-report method for this, so we use an objective mental task, analyzing it to track the process used. I’m not sure where this fits in the listing above, so I can’t tell how many are using this methodology. (It’s heartily predictive.) It’s not technically psychological, nor is it what is traditionally considered a cognitive assessment. Any ideas? I suspect it’s a completely different category. The discipline behind it is philosophy/symbolic logic.

Leave a Comment

Your email address will not be published. Required fields are marked *