10 Screening and Assessment Trends for 2008

Every year, I take a few minutes to reflect on the things I have noticed while working with both producers and consumers of screening and assessment tools. Overall, I am very encouraged by what I have been seeing. The market for screening and assessment tools continues to grow. This makes me extremely happy because we I/O psychologists know the value that is to be had via the use of quality assessment tools.

The science geek in me is also very happy to see strong investment in innovation. I am really pleased to see the ways in which quality content is being combined with technology to collect the mountains of data that are required to uncover underlying truths about the relationship between human traits and job performance.

One of the most valuable means of insight into assessment trends over the past five years has been the annual ERE/Rocket-Hire Screening and Assessment Usage Survey. If you have not taken the time to complete this survey yet, please consider taking five minutes to help us out. Your input really does make a difference, and we greatly appreciate your time. A link to the survey is provided at the end of this article.

The major theme for 2008 will be increased integration of assessment into carefully designed products. Much of the rationale behind these products is the reduction of the work required to implement assessment via a combination of data and technology that results in a turnkey product that is both relevant and easy to use. This trend is going to continue to increase the accessibility of assessment and lower barriers to entry that have long been limiting the use of assessments. A quick look at the trends outlined below clearly demonstrates support for this point. In 2008, I believe we will see the following trends continue to unfold:

Article Continues Below
  1. Continued Interest and Use in Assessment. The market for assessment tools will continue to increase as it has every year for the past decade. This will be the result of a number of factors, many of which are discussed in more detail below. In my mind, however, the number one reason for increased adoption is still the fact that properly used screening and assessment is one of the most important ways to increase the quality of hiring decisions.
  2. More Vendors to Choose From. As interest increases, so shall the number of options available to consumers. For the most part, this is a good thing, as the bulk of new companies and offerings are being created based on solid best practices. As always, there will continue to be less sophisticated or technically-sound options available. With so many options to choose from, extracting the most value out of assessment tools will continue to require a solid game plan and due diligence as the foundation of the vendor selection process.
  3. Increase in Acquisitions for Content. As companies continue to work toward developing products and increasing use of assessment tools, many are discovering that it is easier to purchase content that has a strong history behind it than it is to create their own. This has also been playing out as existing assessment companies continue to purchase smaller companies in order to increase their product range. Recruitment Process Outsourcing (RPO) is also a huge driver of this trend. RPO will continue to be a major force in the future of hiring, and assessment should end up as a key part of the better RPO offerings.
  4. Increased Integration and Packaging of Assessment into Off-the-Shelf Products. Directly related to the trends mentioned above, there has been a continued push towards the development of product suites that provide a hiring process into which assessment is embedded. The goal is to create a more turnkey product that can be lightly customized and quickly implemented. The more quality assessment that is baked into a well thought-out process that can be implemented relatively easily, the more quickly assessment can begin to return value. This equation also lessens the need for professional services and makes the product a bit more “foolproof.” The more turnkey the product, the less assurance one has that it is “dialed in” to a particular position or job. The decision to use a more turnkey product as opposed to a more customized approach should be treated as a business decision driven by the contingencies of one’s particular situation.
  5. Increased Attention to Products/Sales into Vertical Markets. Again, as the trends noted above unfold, we are seeing an effort to package assessment products that have been developed specifically for certain vertical markets. This is driven partly by the fact that vendors are finding that it makes sense to promote and sell products in a more targeted manner. Health care is one of the hottest examples of this trend, but I have also noticed increases in product creation and promotion in manufacturing and financial services as well.
  6. More Products Designed for the Middle Market. The labor-intensive nature of assessment tools has not traditionally lent itself to use by smaller or mid-sized companies. An increasing number of vendors are creating turnkey products designed to overcome the need for professional services and large-scale data collection exercises. As interest in assessment continues to grow, these products will be well positioned to provide value for a wider range of companies.
  7. Movement into Job-Search Process. An increasing number of job boards/career portals are discovering that adding assessment can provide much better search/match functions than do traditional methods, such as keyword search. I have been pushing this idea for the past five years, but this year will provide a noticeable increase in the use of assessment tools as part of the DNA of many job boards.

While the above represent what I feel to be very positive trends, I also see the perpetuation of a few trends that I am not so wild about. These things include:

  1. Continued Fragmentation. Most users of assessment seem to still take a test-focused viewpoint in which the emphasis is on the use of one specific test rather than on the result of a coherent process. While some situations warrant this type of approach, it is best to build a process in which assessment data is tightly integrated into other decision-making data. The good news is that all of the trends outlined above represent movement in the direction of less fragmentation.
  2. Tactical Focus. Assessment is still being used in a highly tactical manner. It is most often considered as a way to fight fires, as opposed to a preventative measure. As consumers experience more success with the use of assessment tools, a more strategic focus will result. It will be some time before we see this type of usage occur, but we are headed in the right direction.
  3. Continued Lack of Proper Evaluation. My head hurts from repeatedly banging it against the wall over this issue. Yes, it is hard to create the buy-in needed to properly evaluate the impact of assessment tools. However, without proper evaluation, it is very difficult to clearly demonstrate ROI and build a solid business case for the use of assessment tools. While vendors are trying to help design products that can assist in this area, proper evaluation requires commitment from the consumer. The consumer is the one who needs to collect the data relating to the business impact of assessment. While this is not an easy task, if we can put a man on the moon, we should be able to collect relevant performance data.

This promises to be a great year for screening and assessment. Continued interest by consumers has created a buzz amongst vendors who are, in turn, creating products designed to eliminate the headaches often associated with using assessment. Of course, there are always tradeoffs. The more turnkey a product is, the less it will be optimized for a particular local situation. The degree of customization required is a business decision that should be made based on a thorough needs and cost/benefit analysis. Some things never change!

Want to help us learn more about what is going on with the use of screening and assessment? Please take five minutes to complete our survey. We will be sharing the results with the ERE community this spring.

Dr. Charles Handler is a thought leader, analyst, and practitioner in the talent assessment and human capital space. Throughout his career Dr. Handler has specialized in developing effective, legally defensible employee selection systems. 

Since 2001 Dr. Handler has served as the president and founder of Rocket-Hire, a vendor neutral consultancy dedicated to creating and driving innovation in talent assessment.  Dr. Handler has helped companies such as Intuit, Wells Fargo, KPMG, Scotia Bank, Hilton Worldwide, and Humana to design, implement, and measure impactful employee selection processes.

Through his prolific writing for media outlets such as ERE.net, his work as a pre-hire assessment analyst for Bersin by Deloitte, and worldwide public speaking, Dr. Handler is a highly visible futurist and evangelist for the talent assessment space. Throughout his career, Dr. Handler has been on the forefront of innovation in the talent assessment space, applying his sound foundation in psychometrics to helping drive innovation in assessments through the use of gaming, social media, big data, and other advanced technologies.

Dr. Handler holds a M.S. and Ph.D. in Industrial/Organizational Psychology from Louisiana State University.

LinkedIn: https://www.linkedin.com/in/drcharleshandler

 

 

 

 

Topics

12 Comments on “10 Screening and Assessment Trends for 2008

  1. First off, compliments to Dr. Handler for an informative, well-presented summation. I learned from it, and believe the knowledge I gained will benefit our agency.

    However, I would like to know how the Dr. proposes we handle a situation I am seeing all too often. In 2007, we had seven cases of ‘final cut’ candidates who took a required written assessment, following the final interview, and as a pre-cursor to an offer. Of these seven, three did not meet the basic assessment qualification, and were subsequently not hired. Again, these were candidates who had completed the hiring process and were pending offers following the ‘assessment’. In other words, excellent qualifications, and all interview findings, were dismissed based on the assessment.

    Of course, the candidates were never made aware that mere failure to pass the assessment closed the opportunity. I did speak with the Hiring Authority regarding what I perceived to be an over-reliance on the assessment score. He concurred with my view, yet told me there was nothing that could be done because ‘HR is highly vested in the assessment tool, and will not bend’. I recommended the assessment be given earlier in the process, but the HA told me that had been discussed and dismissed because the cost of the assessment warranted that it be given only to people being considered for employment. And, based on the HA’s concerns, I did not take this to the HR Director, which likely would cause us to mysteriously fall through the cracks when it came time to hand out search assignments.

    This is NOT an isolated event. Too often, HR management has a tendency to rely more on tools and metrics than HA recommendations. Understandably, after selling it to decision-makers, HR is hesitant to buck the new program, lest they be held accountable for a hiring mistake that might occur.

    What is the answer, Dr. Handler?

  2. Jim – Regarding your comment:

    ‘HR is highly vested in the assessment tool, and will not bend’ …

    I too had this same encounter with a company executive last year who was dismissing candidate after candidate based on test results alone.

    For the record, I have personally managed close to one billion dollars in search activity/actual offers/acceptances as measured in salaries/net comps over my 21 years in recruiting. A large portion of this was direct hands on involvement without factoring in other recruiters reporting to me.

    There are companies earning hundreds of millions in annual revenue today, from the management talent we’ve positioned. I can rant for pages but I’ve made my point.

    Bottom line: I have far more intuitive experience .. and actual life experience … than any testing service and have even yanked candidates out of the interview process mid-stream when I detected subtle behavior that was contrary to what the position required (or to what they originally portrayed themselves to be).

    What such companies are saying when they tell you and I ‘we have too much invested to change our minds’ is that someone’s ego will be brutally bruised and damaged should they have to admit the tests are BOGUS and FAULTY (MOST I have observed ARE).

    I took one of these tests once. Had to pull strings to get my hands on it. It rated me as a 6 out of 10 on sales capability. R-i-g-h-t. And here I am enjoying a livlihood that’s better than 90% of most workers worldwide based on what? Sales! According to the ‘test’ I’m not supposed to be successful at what I DO !!

    Tests are only as good as input. Garbage in -> garbage out. Many of those questions are ellusive, vauge, nebulous and subject to easy misinterpretation and multiple ‘correct’ replies.
    Hence – garbage in.

    If you read the Book EGONOMICS it brilliantly describes how egos, not facts but EGOS … rule the roost when it comes to such decisions.

    Someone made a case for the test. More likely, a brilliant salesman/woman (better at selling the test than you and I were at presenting our candidates and service) convinced someone to buy into the test process.

    That ‘someone’ would have to admit they are now WRONG if they go by ‘their actual interview process’ about spending hundreds of thousands of wasted money on a test process.

    No one is going to easily give up something they fought for, and convinced management to write a several hundred thousand dollar check against.

    Its as dumb and stupid as that: Egonomics. Where egos make the call even though common sense dictates otherwise.

    Oh by the way – Those that DID GET HIRED and made it through our client’s ‘TEST’ — were fired within 12 months after a company president decided to reorg. The whole process was an exercise in utter futility.

    I ask you: Where was the ‘TEST’ that could have forecasted how futile all that hiring would be?

    – Frank Risalvato
    http://www.searchwizardry.com

  3. Employers commonly utilize a variety of hiring tools (e.g., criminal background checks, interviews, assessments, job simulations, educational requirements) in an attempt to get a complete picture of an applicant’s job-related characteristics. Various tools are used because each one may provide an evaluation that is inconsistent with that provided by the other tools–if the information provided at each step of the hiring process always provided a concurring evaluation, there would be little reason to have a multi-step process. Thus, it is typical for all prior positive evaluations to be dismissed on the basis of the most recent step (or last step) of the hiring process. It is somewhat axiomatic that all competitive candidates need to have good qualifications and perform well in at least one interview in order to move forward in the selection process. However, throughout a diligent hiring process the strong qualifications and initial interview results of many candidates will be completely disregarded due to information gleaned from additional interviews, job simulations, assessments, criminal background checks, medical examinations, references, etc.

    In conclusion, if an employer is using a job-related tool, it is an appropriate hurdle that must be cleared during the hiring process. Taking such an approach in a consistent manner, is a legally defensible and sound HR practice and not an over reliance on a particular tool. As an aside, except in cases where the law mandates the timing of a selection procedure (e.g., the Americans with Disabilities Act mandates that all medical tests be administered post conditional offer), employers are well served by using the least expensive hiring tools up front and the most expensive ones towards the end of the process.

  4. Jim,

    You bring to light a very good point. And I recognize that your question was posed to Dr. Handler, so I will keep this brief – in hopes that he will provide even more insight as the subject matter expert.

    However, what many business leaders – and even HR professionals – fail to realize is that there are laws and processes that must be followed when utilizing selection assessments. In addition to validation, there are parameters surrounding how the assessment should be used for selection purposes. Often, a selection assessment should NOT be a final determining factor in someone being hired for a position. It should be one of the factors – as per the Uniform Guidelines for Employee Selection Procedures (the regulation that governs the use of assessments in the selection process). I would be curious to see how Dr. Handler approaches educating hiring managers and HR professionals on these guidelines and the parameters within which they should be operating.

    Just my two cents…..

  5. Yes, this is a problem I have encountered. There is a certain inevitability once HR gets completely sold on psychometric testing. Yet I know of one recent example where the candidate was hired into a senior finance role in spite of having scored just 6% (not a typo, six percent) on the math portion of his assessment. So the point of that test was …? Go figure.

    I do see some value in these tests. I have taken some myself that I believe did a good job of analysing me. But companies tend to see the results as black or white, which is rubbish. Now, if the exec team would agree to undergo the tests, just to confirm that they really do throw up the best hires and don’t exclude some that could just be outstanding, I will buy it. Maybe.

    I don’t have the answer, but personally I’d like to see a total separation of Recruitment from HR, each having its own direct report to the CEO/MD. In my humble opinion, recruitment is not a function of HR. It’s a function of the entire company, and should be seen and valued as such. In the same way that sales is not marketing, and vice versa. As things stand, we are fed information in a need to know, after-the-fact fashion. We will be told this is the screening we’re adopting, and these are our minimum standards.

    Done well, recruitment is of enormous strategic and bottom line value across the company. Its ability to perform well impacts every department in a most profound way. Once that is recognized and reflected in the organization, I dare say assessments would be just as stringent, but far more empirical in the way they are applied.

    Any takers?

  6. As counterintuitive as it may seem, when faced with a client who insists on doing formal testing, it may be best to do it quite late in the process.

    We have run into situations where clients have insisted on doing quite expensive testing for every candidate PRESENTED.

    This proved to be so expensive that it was necessary to drop these clients entirely.

  7. Dionne Mack referenced the U.S. Department of Labor’s Uniform Guidelines on Employee Selection Procedures. These Guidelines do govern ‘assessments’, but they also govern resume reads, interviews, application forms, work samples, skills tests, job simulations, physical requirements — i.e. anything used to make employee selection decisions. Very simply, the Guidelines require that any selection procedure be both job-related and valid. Most resume reads and interviews, as practiced, do not meet these standards. It would take rigorously developed, job-related, well documented, structured interviews, qualified interviewers and consistent scoring methods to have any hope of demonstrating validity. That’s because a selection procedure’s reliability caps its validity; unreliable procedures cannot be valid within the meaning of the Guidelines. Resume reads and interviews generally lack reliability (and therefore validity) because, even under the best of circumstances, applicants polish the inputs, interviewers add subjectivity and, from there, the processes that winnow applicants and lead to final selections typically take twists and turns that make each recruitment rather ‘unique’.

    Best-in-class assessment instruments can meet the Uniform Guidelines (i.e. keep employers out of EEOC trouble) and underwrite consistently and significantly better hiring and promotion decisions, across the board. As with most disciplines the best results come when capable people use the right tools, properly. Recruiters (internal or external) would do well to learn how assessments can help make their candidates, their hiring managers and themselves more successful.

    As Dr. Handler points out, some assessments and vendors are much better than others. Check out Dr. Handler?s previous ERE articles; his writings on ?what to look for in an assessment? and ?how to choose and assessment vendor? cover these two important topics quite thoroughly.

    Companies that do not have a strategic hiring process that includes formal job analyses to determine what it takes to perform well in each position will have a hard time identifying people who have what it takes. Job analyses identify the important job-related factors; capable assessments measure those factors reliably and objectively. Better information and better processes lead to better decisions.

    Today?s best-in class assessment instruments can address a variety of strategic hiring process needs and quickly pay for themselves. Powerful off-the-shelf solutions deploy quickly and easily without organizational upheaval. There?s no hardware or software to buy, learn or maintain and no consultants or experts to pay either. Web-based assessments and reports make the implementation very cost-effective and provide 24/7/365 real-time availability, anywhere in the world.

    Hiring managers, recruiters, HR and the C-suite have a lot to gain by, as Dr. Handler put it, ?extracting the most value out of assessment tools?.

  8. Thanks for all the good discussion on this topic
    The original issue noted is a common problem with assessment and one that often makes my job harder. I have seen many cases where temporary seasonal employees hired with no testing perform the job well and are offered permanent status pending passing through the standard hiring process. When one of these folks fails the test it really causes hiring managers to turn blue in the face and revile against testing using the scenario as an example to show that testing does not work.

    Here are my usual replies: First of all testing is never 100% accurate, not even close. Humans have too much variation to allow for complete accuracy. Assessment still has value despite this fact and it is important that personel using testing are aware of this fact and do not over-rely on test scores as an absolute/final index.

    Secondly, the process is as important as the test in which it is embedded. I strongly discourage reliance on one test score as a pass/no pass unless there are conditions that warrant such a strategy. Rather, hiring should be the result of a series of evaluations that are designed to collect a range of data that can be used to support decision making. Hurdles are commonplace but I prefer them to be earlier in the process and focus on a few key determinants of success.

    Finally, if applicants have proven they can do the job or for whom all evidence in the hiring process indicates they would be a good hire are not passing the test, I would immediately examine the job relatedness of the test in question. Perhaps the test is missing good performers because it is not really measuring the key drivers of success. I often find companies using tests that are not ideal for the job in question. This situation causes increases in errors, truncates ROI, and can lead to legal issues.

    I hope everyone finds this commentary useful.

    thanks

    Charles Handler

  9. First of all wonderful article Charles.

    You are hitting the nail on the head when you mention that assessments should be used to see if people match what a job is asking for. I would say that with the explosive growth of net based jobsearch sites, a large percentage of them employ very “frustrated” filter techniques to identify prospects. Due to the often low quality at this stage, one requires more refined tests for those that pass the “telephone repetition interviews”. As many of the tests are not normally created by the person offering the job role, they can lack in substance.

    This can be annoying for the job manager and often is a major bottleneck.

    What is needed with the event of the newer technologies out there is fully custom made testing by the actual guy or girl offering the job role – as they know exactly what they are looking for. Add factors like your own video testing questions such as in supercandidate and you will get an indepth view to the knowhow of the prospect.

    The tests should be user friendly with no programming.

    Several comments talk about cost, so the cost should be heavily reduced to allow for earlier testing – before the interview stage.

    Finally a great feature such as cell phone notification when several people pass the recruiter’s threshold of quality, would allow mobility and thanks by many recruiters.

    A technology like that would eliminate downtime and really make it feasible to find a job prospect that meets the match on a higher level of credibility.

  10. This is a great discussion! A lot of diverse opinions. Let me first disclose that I am a consultant and my clients use assessment tools for hiring and development that I provide to them. These assessment tools directly produce the report…I am not the interpreter.

    On the hiring side, I make sure they know not to use the results of the assessment for more than 1/3 of their hiring decision. It’s not because the results are faulty…it’s because they only offer up the objective information. As important are the subjective pieces such as how we feel in our ‘gut’ about the person and whatever intelligence we can gather from reference and background checks. Also, if certain skills are critical to the job and the company is not in a position to train the individual, this needs to be factored in.

    In defense of pre-employment assessment tools–I am talking about those that are continuously tested for pre-employment validity and reliability–their results can be trusted if they include profiling the best performers currently in a given job and using that profile to compare candidates against. For instance, we all know that not all sales jobs can be filled by anyone who is a star salesperson elsewhere…it depends on what’s being sold, the sales cycle, and the culture of the company among other things.

    I have proven this over and over again by analyzing the data after the assessments have been in use for at least a year. All we have to do is see who the top performers are vs. the bottom performers vs. their assessment scores. The correlation is extremely close between the top performers and those whose overall assessment scores are the highest.

    It gets even more interesting when you then compare the assessment attributes being measured, looking for those attributes where the highest performers’ scores are essentially the same and are significantly different than those of the lowest performers for the same attributes. This tells you that these attributes are the most important ones to focus on when selecting candidates for this position.

Leave a Comment

Your email address will not be published. Required fields are marked *