When Testing Goes Wrong

Some readers react negatively to testing. They think of testing as a hard-hearted, diabolical attempt to remove humanity from the hiring process. Some even put it in the same class as not washing you hands after going to the bathroom, scratching ones armpits in public, or forgetting to floss. But nothing could be further from the truth. Let’s get a few things straight. Even rational people:

  • Argue interviews are not “really” tests ó yet they only hire applicants who give impressive answers to questions.
  • Say they never hire “dud” employees ó yet they somehow allow duds to end up on the payroll.
  • Know workshops rarely turn performance duds into performance stars ó yet they continue trying to repair “broken” employees by sending them to workshops.
  • Recognize that star performers continue to be stars regardless of working conditions.
  • Know bad managers can do more to squelch good performance than develop bad performance.

Virtually everyone hired by an organization undergoes pre-employment testing. In some cases, it includes filling out an application form. In others, it involves having drinks with the boss. In still others, it involves being picked out of a line up. Testing is everywhere! Anyone who argues that selection is not the single most important function in an organization is not seeing the big picture. Think about it: How long would a production manager be employed if he or she knew as little about measuring raw material specifications as many people in the recruiting profession know about measuring human performance? Let’s examine a few major job families where interviews and unvalidated tests frequently fail to assess critical failure areas. C-Level Executives C-level executives have worked long and hard to achieve senior positions; they demonstrated sufficient skills and job accomplishments to move up the ladder. But now that they have achieved a position of power and influence at the top of the pyramid, where do their failures occur?

Article Continues Below
  • C-Level executives must be visionaries. They have to separate the trees from the forest. They must make decisions based on little or no data. And they have to rely on an incredible depth of “expert” knowledge to make decisions that would elude most other people. This level of decision-making requires considerable abstract information processing.
  • C-Level people are also very tempted by power. Look around at the number of executives in the public eye who feel perfectly justified lying, cheating, manipulating, and stealing. As they sit at the top of the pyramid, subordinate employees quickly learn that square pegs do, indeed, fit round holes. Uncovering these kinds of dark-side motivations requires special instrumentation and interpretation.

What kind of questions do C-level applicants get asked? Generally they’re about past accomplishments and future visions. But why do C-level executives most often fail? Because of dark-side motivations and a lack of abstract decision-making ability. Technical/Knowledge Workers These folks are generally drawn to occupations that emphasize mental ability more than interpersonal skills. These include occupations such as information technology, engineering, science, medicine, accounting, mathematics, and so forth. It should come as no surprise that people drawn to mental occupations are not very good at interpersonal relationships. So where do they tend to fail? Yes, you guessed it: They can be downright abrasive and insensitive. It is not because they want to be socially awkward, it is just in their natures. They tend to be more comfortable with the tangibility of things than they are with the intangibility of people. Evaluating this job family often includes interpersonal simulations that involve people effectiveness. For example, I once worked with a public utility where the safety inspectors had the interpersonal skills of bridge trolls. Even their HR representative was psychotic. Imagine the kind of communication problems they had ó and it involved radioactive materials! What kind questions do technical applicants get asked? Generally they are about the level of technical ability they possess. But why do they most often fail? Because of dysfunctional interpersonal skills. Salespeople Salespeople are often all clustered in the same pot, but sales is actually among the most diverse occupations. For example, selling computer systems takes entirely different skills than selling furniture. In most cases, though, salespeople tend to emphasize personal persuasion. Salespeople like to talk and be social, and they often put a substantial amount of energy into getting people to like them. But all this social schmoozing has a down side: salespeople often underestimate the value of learning their product thoroughly. Salespeople tend to forget that people buy solutions, not sales pitches. Hiring within the sales family requires an analysis of very diverse sets of skills. It can involve evaluating everything from mental prowess and organization abilities to motivations and interpersonal skills. What kind of questions do sales applicants get asked? Generally they take the form of “show me your W-2” and “sell me this pencil.” But why do they most often fail? Because of a lack of planning and follow-through, and an unwillingness to learn the product line and ask prospects the right questions. Customer Service Customer service people must have done something very bad in a past life. They work long hours, take abundant verbal abuse, have their days scheduled by automated machines, and are often poorly paid. Like the others, this job family can also be subdivided into several types: outbound sales, inbound sales, customer service problems, marketing surveys, collections, technical support, web-based support, etc. Like the sales job, each customer service job takes a different set of skills . The nature of the customer service job makes it either a lifetime position (for sainthood candidates) or a transitional one. For example, I worked with a company that provided technical customer service for computers and the like. Management’s objective was to keep customer service people on the job for two years. Any more and employee raises would escalate their costs. Any less and the cost of training would be excessive. Customer service people tend to turnover when they discover the job they do is not as glamorous as the job they expected it to be, or when the required amount of problem-solving does not match the employee’s ability or motivation. What kind of questions do customer service people get asked? Usually it’s nothing more than, “Can you fog this mirror and promise you do not have a contagious disease that resists modern antibiotics?” Where do they most often fail? When they learn what the customer service job is really like. In addition to the above, here are some other important things we need to keep in mind about assessment:

  1. Training never could and never will change a sow’s ear into a silk purse. Good management never could and never will change a sow’s ear into a silk purse.
  2. Bad management can effectively squelch the best performer.
  3. Human resource systems are frequently irrational and disconnected: People are hired based on one set of standards, managed on another, reviewed under a third, trained on a fourth, and rewarded on a fifth. Organizational confusion is the norm, not the exception.

The single most important function in an organization is staffing. An organization can NEVER be better than the talent it employs, yet staffing is usually the most misunderstood and poorly implemented function in an organization. But the greater the problem, the greater the opportunity.


7 Comments on “When Testing Goes Wrong

  1. I am a great believer in personality testing but providing it is used in the right way. Most ineffective testing I have seen is because of the quality of the test itself and the reasons and way the tests are administered.

    In my opinion, when used in the selection process, they should only be used to give an unbiased opinion between several candidates to help differentiate them for a particular position.

    They should not be used to make a decision but rather as a tool to help the employer make their own decision based upon the information given.

    To use them individually and for everyone because it is company policy to, is a waste of time and money and simply devalues the process and results.

    You might as well use horoscopes.

    Dr Williams, depending on your definition of course of a sow’s ear and a silk purse, are you saying that you believe we are born with our skills and cannot be taught?

    For example a born salesman?

  2. I enjoyed the article by Dr. Williams. Unfortunately, I agree with what he says. I say unfortunately because we so often are unwilling to work on a total hiring solution that would require effort, planning and investment of dollars.
    There is another problem though. Let’s take hiring sales people as an example. Is it possible to hire a top performer who also has the motivation and interests to learn product, create a sales plan and stick with it?

    I would say yes it is, but these people are very hard to find. We should be benchmarking this type of salesperson and hiring against the benchmark. Of course this would be a specific benchmark that would be created for your business and would also reflect the person as they relate to the culture of your business. This type of in-depth assessment would certainly enhance the hiring process.

    Our business is new and has been developed to offer a complete online solution for recruiting within the Retail Automotive Industry. The vision was and is to give Dealers an opportunity to find and hire new people based on the top performers within the culture of their business. We offer a job board, screening tools, benchmarked assessments, interviewing plans and active and passive recruiting capabilities.

    All of these tools are capable of creating a better hiring process based on culture.

    Back to the original question. If I want sales staff that are motivated to sell. Who look at everything they do in terms of return on investment of time and/or effort, I may not get someone who is particularly motivated to build long term loyalty, through follow-up. Why, because they don’t see an immediate return on time and effort invested. But they are a top producer because they are tenacious and extremely customer oriented when the ROI is within forseeable reach.

    If this is the type of salesperson you have as a top performer, perhaps you can do a few things to motivate change:
    Show them the payoff in customer loyalty in terms that make sense to their primary motivators.
    Give them small bits of the ‘unpleasant’ work and coach them actively.
    Give the work to a Customer Relation Management Team and let them concentrate on what they do best…’Sell’.

    I find that a lot of businesses don’t want to change the characteristics of their successful people because they like the results. If this is the case then the alternative is to have the loyalty development work done by others.

    We can only change that which wants to be changed. If the company is not ready for a change then our mandate should be to provide them with the kind of person they want.

  3. Bruce, I always remember the story about a large chemical company that decided to test their sales people so they can create the perfect profile to test new recruits against, and match against their best sales people.

    The only problem was the results of the test.

    The results showed that their most successful sales people had a profile of cutting corners, being non team players, unethical thinking, would stitch their own mother up attitude to win a deal and so on. Needless to say they dropped the idea as they did not like to think that this profile represented their best sales people.

    With testing you are only measuring the characteristics of someone based upon the information they give you. You can never predict the success of a salesperson with them because there is no ideal profile and a lot of what makes someone successful in sales cannot be measured or predicted.

    All you can do is to say that based upon the information you have, this person shows the characteristics that you would expect to see in a successful salesperson. I have seen the exception to the rule too many times to even trust that.

    Being successful in sales is more about individualism than being the same as the next person. It is also about the person you work for and the company you represent. Ask anyone from IBM, Cisco or Microsoft.

    I’ve met and worked for Managers that can make anyone successful in selling. I have seen Managers that couldn’t motivate themselves let alone their sales team.

    Interesting subject area though.

  4. I enjoyed the article by Dr. Williams but as with most things there is an element of truth in what he says and an element of falsehoods.

    Anthony has already touched on most of the counter-points that needed to be brought to light. So let me just say that I agree that some form of ‘testing’ is always done AND that while the results of certain testing should not be totally discounted, they should likewise not be looked at as totally foolproof. If one gave 100% credence to testing, I would not have been hired for several of my previous positions and I’ve been successful in all them – some of which had nothing to do with recruiting or career counseling. Testing results also showed that I would not make a good recruiter – and I’ve shown that to be false as well.

    If testing had been a requirement at the time, Moses probably would never have been chosen to lead his people out of Egypt. David wouldn’t have been chosen to be King over his brothers that appeared to be a ‘better fit’. How many of us recruiters have had Hiring Managers say to us that they would have never hired someone had the resume just came in, or had the ‘unseen’ benefits of a certain person not been brought to light? How many burned out light bulbs have shown brightly in a different environment?

    It is for those reasons that good recruiters will always be in demand. We bring what no resume can and, as far as I’m concerned, what no test can reveal either.

  5. Good discussion…May I elaborate?

    1) Interviews are TESTS…after all, people ‘pass’ of ‘fail’ based on their answers, right? Show me an organization that does not test and I will show you an organization that hires everyone who applies. The only question we have is ‘how accurate’ we want our tests to be.

    2) Nothing is 100% accurate. We play the odds. There will ALWAYS be outliers, but we have to focus on the NORM result…the rest will take care of itself.

    3) Yes, top producers cut corners. That’s why job standards are set on ‘salt of the earth’ producers (those who are just above the mid-line). A thorough job analysis MUST include both reasons for failure AND for success. Any vendor who sets standards exclusively based on their best people seriously misunderstands testing.

    4) Testing is not limited to personality…it includes EVERYTHING critical related to job performance…that includes mental abillity, planning skills, interpersonal skills, and AIMS.

    5) There is a good reason why professional sports teams rely on skilled talent scouts…they supply the raw materials.

  6. Royce said it well, with every thing in this world there is an element of truth and of course an element of wow, I can?t believe you really believe that!!!

    Thanks for Bringing up Sports Dr. William, using your analogy we can take a good look at the Lakers, which was once a great team, not good, but pretty darn awesome.

    Excellent and Raw talent was brought in, Coby Bryant what an excellent player, great moves, but what a lousy team player both on and off the court (which by the way helped get him in trouble as he decided to go to CO, instead of being with the team). Gee they even lost Shaq because he could not stand it any more. This team has gone from great to mediocre in less than 2 years.. Is it because they don?t have talent, they do!

    ?Now let us take a look at the Angels and the Cardinal? gee who could think they could finally win a world series ? Tim McCarver said to Joe Buck during Game 2. ?It?s almost unbelievable that a team led by a $20 million right fielder, a $12 million ace and an $18 million #2 starter could pull this off. There was talk about them not having the talent, but they surprised a lot of people so. Who knew?? Never in a billion years could anyone have ever anticipated that the Red Sox, a team that runs on spirit and camaraderie alone, could advance past the New York Yankee – they’re World Series winners because of their never say die attitude and their blue collar ways, ?It would have been a shame if players like Manny Ramirez, David Ortiz, Pedro Martinez and Curt Schilling would have had to play in obscurity the rest of the
    lives. Hopefully, this World Series has introduced these guys to people outside of Boston? Their New Young manager and Trainer also had a lot of input

    My point, yes testing is good, in some aspects testing is great, but when it comes down to it, testing only goes so far. Hire a talented arrogant fool, who takes great tests, and your team will become a mess. Promote, train, and encourage a staff to do well and winners will be found and created from within.

  7. Everyone on the teams you mentioned first passed extensive tryouts (i.e., tryouts=tests)…

    They were not amateurs, they were not everyday folks with a dream, they were not ‘made from old cloth’, they were the best of the best.

    If they could not play like pro’s, they were not on the field.

    ‘Want to staff an organization with winners? Hire only fully skilled people and don’t screw them up.

    ‘Want a mediocre organization? Forget about job analysis; forget about validation and try to teach/train/coach/incent employees into becoming something they are not.

Leave a Comment

Your email address will not be published. Required fields are marked *