Results from the 6th Annual Rocket-Hire Online Assessment Usage Survey

Article and research by Charles Handler and Mark C. Healy

For the last seven years, Rocket-Hire has surveyed users of web-based pre-employment assessment tools, so we again asked members of the ERE community to tell us about their usage of typical pre-employment screening, testing, and assessment programs. As with years past, we zeroed in on the pulse of pre-employment assessment usage. And in an increasing climate of legal scrutiny for testing, and the hoopla surrounding the Ricci case, we decided to focus the content of today’s article on two issues that are inexorably linked: Implications of evaluating one’s assessment strategy, and attention to relevant legal issues.

Those interested in obtaining a copy of our full report can email us ( and we will be sure to send you a full copy once it has been completed.Or, check out an upcoming Journal of Corporate Recruiting Leadership, probably the October 2009 issue, where we’ll have an in-depth analysis.

A Word About our Methodology
This year, 148 recruitment and hiring professionals completed the Rocket-Hire Online Assessment Usage Survey. Respondents were evenly representative of recruiters, recruiting leaders, HR executives, business owners, and hiring managers, and featured a wide variety of organizations and hiring situations.

Use of Assessment Tools
Overall usage of assessment tools was generally about the same as in past years — roughly two-thirds of respondents. Of that two-thirds, 54% are deploying both paper-based and online assessment, and 30% are using exclusively online assessment. The remaining employ only paper-based tools. Most use a variety of different assessment methods, with the majority using between one and three different types. The table below reveals the proportion of firms using various common tools.

Usage rates of common assessment tools

Skills and knowledge certifications and personality assessments continue to be popular, with indicators of cultural “fit” and various cognitive ability measures also widely used. In addition, we observed a surge in the adoption of web-based background investigation strategies.

Interestingly, 2009 highlights a trend emerging over the last few years: It is now just as common to find organizations deploying assessment company-wide as it is to find it used for only one or two jobs. These findings reflect the fragmented use of assessment that we have experienced in our client work. To be sure, there are tons of ways assessment can be used, but there is no one major trend in how or how much it is implemented.

Effectiveness of Screening and Assessment Tools

This year, 80% of assessment users felt that assessment had a “positive impact” on their organization. In contrast, only 3% of assessment users felt these tools did not add value. However, only 64% of assessment users collected metrics to judge the quality of their interventions, and 79% judged ROI using one or many indicators. However, definitions of “ROI” vary widely, with some approaches considering a broad, cursory opinion from managers, whereas only a few use a true scientifically derived method such as a validation study.

Wide variation in the real-world evaluation of ROI clearly reveals that, while many companies do collect metrics, few of them are going about this in a way that reflects best practices. This is cause for concern given the fact that these same best practices create the foundation for demonstrating the legal defensibility of assessment measures.

Article Continues Below

Legal Issues

This year’s Supreme Court decision in favor of firefighters who challenged the City of New Haven, Connecticut, has once again focused attention on the legal aspects of pre-employment assessment. In anticipation of the High Court hearing this case, we wondered how much typical users paid attention to legal and regulatory issues related to using assessment to evaluate candidates.

Percent indicating legal issues are “central to this activity” or “very prominent”

These results express both positive and negative connotations. On the positive side, it seems that a good number of companies pay attention to legal issues related to testing and assessment. On the other hand, perhaps more firms should consider devoting resources to demonstrating and documenting legal defensibility. The lack of understanding of what is actually required for ensuring compliance and defensibility may create problems as both assessment usage and legal scrutiny increase.

The energy spent on legal considerations does not relate to the extent of assessment deployed, or size of the organization, nor number of hires. As with overall usage of assessments, legal concerns do not follow any pattern across survey respondent demographics or assessment strategies.

Legal defensibility is not just another set of meaningless hoops that must be jumped through and checked off. Proper evaluation of ROI via validation and legal defensibility are actually two sides of the same coin, the common goal being to document the job-relatedness of all tools and processes used to select employees.

Though the continued economic turmoil has put a damper on hiring, the adoption of in-depth assessment tools continues to grow, especially as more organizations devote resources to find out if testing and assessment is worth it. The bottom line: Quality hiring tools can improve performance and employee engagement (i.e., provide ROI) while also ensuring the legal defensibility of hiring practices, but it is essential to demonstrate how much they actually benefit your organization.

Dr. Charles Handler is a thought leader, analyst, and practitioner in the talent assessment and human capital space. Throughout his career Dr. Handler has specialized in developing effective, legally defensible employee selection systems. 

Since 2001 Dr. Handler has served as the president and founder of Rocket-Hire, a vendor neutral consultancy dedicated to creating and driving innovation in talent assessment.  Dr. Handler has helped companies such as Intuit, Wells Fargo, KPMG, Scotia Bank, Hilton Worldwide, and Humana to design, implement, and measure impactful employee selection processes.

Through his prolific writing for media outlets such as, his work as a pre-hire assessment analyst for Bersin by Deloitte, and worldwide public speaking, Dr. Handler is a highly visible futurist and evangelist for the talent assessment space. Throughout his career, Dr. Handler has been on the forefront of innovation in the talent assessment space, applying his sound foundation in psychometrics to helping drive innovation in assessments through the use of gaming, social media, big data, and other advanced technologies.

Dr. Handler holds a M.S. and Ph.D. in Industrial/Organizational Psychology from Louisiana State University.







7 Comments on “Results from the 6th Annual Rocket-Hire Online Assessment Usage Survey

  1. What’s the desire for companies to 1) use web-based simulations with video clips or avatars to show a vignette, and then have candidates choose the most appropriate response (typically in multiple choice format)vs. 2) have a phone-based simulation presenting a live situation in which the candidate must interact with a role player/assessor? If the costs were not too dissimilar (say 300 vs. 500), would companies find value in the live simulation to pay more?

  2. Good question Amy. I think that the best answer is, it depends… Both options have value depending on the situation at hand. Companies make use of both of these tools as a part of their hiring process. The use of web based simulations is just getting started but you should expect to see more and more options for this kind of assessment in the near future. These assessments are an excellent way to place candidates in realistic job situations and can be way more engaging then traditional tests. Phone based simulations are less common and I think their use is moving in the opposite direction. The exception to this would be a behavioral interview, which functions somewhat like a simulation as the candidate is asked to discuss how they would handle various scenarios that are relevant to the job. Phone based assessments are not automated so their cost is likely to be higher due to the fact that each phone call requires more resources. On the other hand, there is more richness of information from a one on one dialogue then from a more static simulation.

    When it comes to pure simulations, not including interviews, I think the trend will be an increase in the use of web based simulations and fewer phone based simulations. This makes the most sense from an economic perspective.

Leave a Comment

Your email address will not be published. Required fields are marked *