Major Findings From The Annual Rocket-Hire Online Screening and Assessment Usage Survey

Article and research by Charles Handler and Mark C. Healy

For the last three years, Rocket-Hire has surveyed ERE readers who use Web-based screening and assessment products as part of their hiring process. The information we have been able to gather from this research has been a valuable part of our efforts to track important trends in the use of screening and assessment tools.

Thus, earlier this year, we again asked people professionals to tell us about their use and opinions of these tools. This brief report highlights the major findings of our fourth annual survey.

Please visit the ERE article archives to view the results of the previous three years’ surveys. The full version of our survey results will be available in mid-November; contact us at info@rocket-hire.com and we will be sure to email you a copy.

The information below provides an overview of the data we used to achieve our objectives, which included the following three goals:

  1. Understand the general rate of usage of Web-based screening and assessment
  2. Get information on usage rates for various types of tools
  3. Determine whether these hiring tools are perceived as effective for the organization

Who Responded?

This year, 136 people professionals completed the survey, representing an increase of 51% over last year. This increase is indicative of the fact that screening and assessment is continuing to gain momentum as a best practice among hiring professionals.

As we have seen in past surveys, respondents were evenly representative of recruiters, HR executives, and hiring managers, and represented a wide variety of organizations and hiring situations. Thirty-four percent hired more than 500 employees in a given year, but a majority of respondents (66%) made 500 or fewer hires per year.

ATS Usage

As with previous surveys, a majority of respondents use an applicant tracking system. Specifically, 72% already use one, with 8% in the process of installing applicant-tracking technology. As can be seen in the tables below, companies of all sizes are taking advantage of applicant tracking technology, and high rates of adoption indicate that use is fairly mainstream.

Table 1
Size of Company % using or intending to use ATS
1-50 56%
51-500 71%
501-2,500 82%
2,501-5,000 85%
More than 5,000 100%

This is important because the use of screening and assessment tools is often the next step in the creation of a technology-based hiring process. Thus, the higher the adoption rate for ATS, the greater the usage of screening and assessment tools will be.

Screening Tools

Strictly an administrative process, an applicant tracking system alone does not usually mean any in-depth evaluation of candidates. However, the following data on use of prescreening and assessment tools does underscore the growing importance of judging job applicants using systematic tools. For purposes of this survey, screening tools are defined as:

Tools that gather information about, or ask candidates to respond to questions about, their experience, skills, and qualifications in order to identify if they meet minimum job requirements. These tools are typically used early on in the staffing process.

Sixty-two percent of respondents indicated that they use at least one online screening tool to sort job candidates early in the hiring process. The table below captures the usage rates of each of the most common forms of screening.

Table 2
Type of Prescreen % using as a primary automated prescreen
Resume-scanning systems 15%
Biodata or personality questionnaires 10%
Basic qualifications 58%
Assessment of “fit” with company 14%
Other tools 3%

As can be seen above, qualifications screening was the most common form of automated prescreen in use. Over the last several years, usage rates for each of these tools are quite consistent. Part of this trend seems to be the relative simplicity with which basic job (or stated “minimum”) qualifications (such as education or years of experience) may be collected and evaluated. As can be seen in Table 3, the extent of the deployment of prescreening technology varied quite a bit as well, with some companies using these tools for all jobs while some using it for one or two alone.

Table 3
Extent of Prescreening %
All jobs within a business unit, but not all business units 25%
All domestic jobs 27%
All worldwide jobs 19%
Specific local jobs only 18%
Not sure 11%

Assessment Tools

In contrast to screening tools, online assessment tools were defined as:

Scientifically based screening tools that look more deeply into a candidate’s abilities, interests, and skills. These tools include personality measures, cognitive tests (i.e., verbal and quantitative skills), situational judgment tests, job simulations, etc. These tools are typically used for a more in-depth evaluation later on in the staffing process.

The overall usage rate of in-depth candidate assessment tools by organizations has grown when compared to the last few years. Although our current sample of people professionals represents a largely different group of individuals compared to last year, the substantial increase in the adoption of most types of tools is difficult to put aside. This trend alone is quite significant as it shows the growing increase in the popularity of these tools.

Table 4
Type of Assessment 2006 2005 2003-04 2002
Personality measures 65% 34% 30% 21%
Assessment of “fit” with company 53% 35% 27% 29%
Cognitive ability 42% 46% 27% 26%
Biodata 16% 15% 10% 14%
Skills/Knowledge 66% 53% 40% 12%
Background investigations 65% 54% 30% 31%
Simulations 21% 18% 14% 10%
Online interviews 7% 15% 4% 19%

In line with previous surveys, checks of criminal background, specific skills and knowledge certifications, and personality inventories are becoming routine. However, this year’s results signify a substantial jump in the level of adoption of these instruments. Online interviews and simulations continue to be less often used. But how extensive are deployments of assessment tools across organizations?

Table 5
Variation in Deployment %
All jobs within a business unit, but not all business units 24%
All domestic jobs 28%
All worldwide jobs 8%
Specific local jobs only 27%
Not sure 13%

As with prescreening, the variation in the deployment of tests and assessments is wide, with a total of 36% of users indicating that assessment is utilized in the placement of individuals into either all domestic (28%) or global jobs (8%). For another angle on the extent of assessment implementation, we asked respondents to indicate the level of jobs for which assessment is used for evaluating candidates.

Table 6
Use Across an Organization %
Entry level/hourly 43%
Lower-level management/professional 55%
Middle-level management/professional 53%
Higher-level management (director) 41%
Executive/Vice President 26%

Table 6 suggests that this aspect of online assessment has evolved. 2006 appears to be the first year that our data show a general pattern of use across all levels of an organization. But unlike job level, rates of use for different types of job settings appear to vary quite a bit. Table 7 displays these broad differences.

Table 7
Rates of Use for Different Types of Job Settings %
Customer service 41%
Manufacturing/Labor 12%
Skilled trades 24%
Account management 32%
Call centers 30%
Managerial/Supervisory 53%
Administrative 41%
Information technology 38%
Retail 12%
Consulting/Advising 16%
Sales 41%
Professional 45%
Other 22%

As we noted last year, Web-based managerial, sales, IT, administrative, and customer-service assessments are common, in part, because so many different instruments and recruiting programs have been developed specifically for these sorts of jobs. Moreover, non-online, validated assessments of key skills and competencies required for success in these roles have been used for decades.

Effectiveness of Screening and Assessment Tools

A key goal of this series of studies has been to document the feelings people professionals have about the effectiveness of online recruiting and hiring systems. This year, 37% of prescreen users and 63% of assessment users felt their tools added value to their organization. The rest were evenly divided between either not having an opinion or feeling that Web-based hiring tools did not add value.

Understanding how these users have formed their opinions is of even greater interest. Apparently, these opinions have been formed through general impressions as opposed to actual evaluation, as only a minority of companies in our survey actually evaluated the use of hiring technology. Specifically, only 31% of users of prescreening tools and 30% of assessment users collected metrics to judge the quality of their interventions. Moreover, users of metrics tend to distinguish themselves from the majority. Tables 8 and 9 reveal this contrast:

Table 8
Are Prescreening Tools Effective? Organizations collecting metrics Organizations that do not collect metrics
Yes 59% 24%
No 19% 31%
Not sure 22% 31%
Did not answer 0% 14%
Table 9
Does assessment have a positive impact on your organization? Organizations collecting metrics Organizations that do not collect metrics
Yes 89% 50%
No 7% 27%
Not sure 4% 23%

Clearly, collecting metrics helps to answer the question of effectiveness and also appears to lead to impressions of success in the use of Web-based hiring systems. A variety of different metrics and measures are in use. Our respondents mentioned comparisons of assessment scores versus tenure and turnover; validation studies; diversity indicators; standard HR metrics (e.g., time-to-fill, cost-per-hire); and impressions and opinions of managers and job candidates.

Article Continues Below

The Future of Online Screening and Assessment

Of those companies not currently using screening or assessment instruments, 62% feel they will implement one or more in the future. For them, Table 10 summarizes the sorts of tools under consideration.

Table 10
Type of Screening/Assessment % considering use
Resume-scanning tools 24%
Qualifications – experience, education, etc. 48%
Personality measures 25%
Assessment of “fit” with company 25%
Cognitive ability 21%
Biodata 5%
Skills/knowledge assessment 32%
Background investigations 22%
Simulations 8%
Online interviews 11%
Don’t know 11%

Consistent with experienced users, qualifications screening, assessment of cultural fit, skill/knowledge assessments, and personality inventories dominate the interest of organizations seeking to implement web-based hiring tools. But with such an optimistic outlook ahead, what is keeping organizations from adopting online screening and assessment tools?

Obstacles to the Use of Online Screening

Of those seeking to enter the world of online hiring, what is single biggest obstacle to the adoption of prescreening and assessment in their organization?

Table 11
Single-biggest obstacle to the adoption of online screening and assessment? %
No obstacles right now 22%
Lack of knowledge in organization 17%
Too costly / lack of budget resources 16%
Skepticism about the ability of screening to provide results 13%
Decision-makers do not believe it is worth the cost 10%
HR not interested in innovation, reluctance to change 6%
Hesitation due to legal issues 6%
Technology is still too new 4%
Tools will negatively impact the candidate experience 3%
Hesitation due to security issues 3%

The data in Table 11 are both encouraging and discouraging at the same time. It is very encouraging that the largest percentage of respondents indicated there are no obstacles at this time. This is a first and clearly shows an increasingly positive climate for the use of screening and assessment tools.

At the same time, it’s the same-old song and dance. Data suggest that typical concerns center around value of spending money and time on more advanced assessment, candidate reactions to hiring tools, and lack of knowledge regarding the positive implications of assessment tools.

Thus, in terms of skepticism about benefits and a lack of budget, companies continue to be penny-rich and dollar-poor when it comes to dedicating monetary and human resources to their recruitment and hiring processes. Cost of entry is low, and the value of the better assessment tools has been established through hundreds of studies.

Survey respondents also tended to mention candidate distortion of qualifications as the biggest threat to the success of their online hiring tools. But inflation of job skills has always been with us. Overall, most companies have few or no substantive reasons to limit their use of online recruitment and hiring techniques.

Summary

We offer the following as key findings that help clarify today’s trends in online hiring:

  • Most medium-sized and large organizations have adopted or will adopt an applicant tracking system, but the use of an applicant tracking system does not relate to perceptions of hiring effectiveness.
  • Some hiring tools, like qualifications screening, skills testing/certification and personality tests, are becoming standard features of a strategic hiring process. The use of some methods (e.g., online interviewing) is rare.
  • Few companies formally assess their prescreening tools or use metrics of any kind as a decision making aid. Those who do tend to be aware of the effectiveness of their hiring systems and perceive greater value in them.
  • People professionals report a lack of understanding, weak budgets, or a general lack of support for online tools as their primary obstacles to adoption or greater use of modern prescreening and assessment technology.

Overall, the results of this fourth version of our survey are the most encouraging yet. They clearly show a growing interest in these tools. Valid, Web-based approaches to recruiting, screening, and placing job candidates are fast becoming the norm at leading edge, people-focused organizations.

This trend is encouraging and, in our experience, not surprising. We feel these numbers will continue to grow as more organizations understand the value in properly evaluating the impact of screening and assessment tools.

Unfortunately, a major takeaway for us is that there is still a very widespread failure to adopt the business intelligence mindset required to evaluate the effectiveness of screening and assessment tools. We found a lack of understanding of the value these tools can return as well as budgeting issues the primary obstacles.

As long as this deficiency exists, it is not surprising that skepticism about the value of these tools remains strong. We will continue to champion the use of evaluation to “close the loop” and help ensure an increase in the understanding of how properly implemented screening and assessment tools can impact the bottom line.

(See next month’s Journal of Corporate Recruiting Leadership for expanded coverage of findings from the 4th Annual Rocket-Hire Online Usage Survey.)

Mark C. Healy also contributed to this article. Mark is a consultant, teacher, and business writer based in Oakland, California. He assists companies with the development of their human capital strategy and designs hiring, leadership development, and training programs. Mark teaches Industrial/Organizational Psychology at De Anza College in Cupertino, California.

Dr. Charles Handler is a thought leader, analyst, and practitioner in the talent assessment and human capital space. Throughout his career Dr. Handler has specialized in developing effective, legally defensible employee selection systems. 

Since 2001 Dr. Handler has served as the president and founder of Rocket-Hire, a vendor neutral consultancy dedicated to creating and driving innovation in talent assessment.  Dr. Handler has helped companies such as Intuit, Wells Fargo, KPMG, Scotia Bank, Hilton Worldwide, and Humana to design, implement, and measure impactful employee selection processes.

Through his prolific writing for media outlets such as ERE.net, his work as a pre-hire assessment analyst for Bersin by Deloitte, and worldwide public speaking, Dr. Handler is a highly visible futurist and evangelist for the talent assessment space. Throughout his career, Dr. Handler has been on the forefront of innovation in the talent assessment space, applying his sound foundation in psychometrics to helping drive innovation in assessments through the use of gaming, social media, big data, and other advanced technologies.

Dr. Handler holds a M.S. and Ph.D. in Industrial/Organizational Psychology from Louisiana State University.

LinkedIn: https://www.linkedin.com/in/drcharleshandler

 

 

 

 

Topics

Leave a Comment

Your email address will not be published. Required fields are marked *