Who and What Can You Trust These Days?

A few days ago, John Hollon wrote a short blurb about the growing population of HR “experts.” I could not agree more. I blame it on public exposure from the Internet. The web has made it possible for almost anyone with a computer and an opinion to claim expertise. So how do we separate expertise from strong opinion? It’s not easy. In my case, it took studying jobs and developing selection tests to discover the clues. It’s embarrassing to admit they were there all the time …I just never thought about it until I had to measure them.

Rungs of Expertise

Expertise is ladder-like. The first rung is a pair of hands; i.e., people who make a living doing what the client asks. Usually they have some practical experience with the subject (e.g., they know slightly more than their clients); but, they are actually just skilled individual contributors. You might think of them as knowing how to use the most common Word features.

The next type of consultant is a facilitator. He or she is not necessarily more knowledgeable than a pair of hands, but knows how to manage groups. Facilitators usually start life as a pair of hands but learn little more from every client engagement. Eventually they learn to help people solve their own problems and keep them on track through group processes.

This next rung is where the herd really thins out. This is where we find people who are subject-matter experts. Not only do they have all the practical experience possible, they understand the theory that supports it. You could think of them as knowing how to use all the functions of Word as well as being able to teach others how to use any function.

Lastly, we have the experts who not only know all the functions of Word, but they can tell you what’s working, what’s not, and why. Given the opportunity, they are capable of actually redesigning the software to make it more useful and efficient. These folks are few and far between. The developer of Internet protocol and the people who developed the concept of integrated office software belong to this group.

As a side note, clients should know clearly when they need a pair of hands or facilitator who can help them move from A to B; or, an expert who can move them from A to Z. For example, people who develop their own selection or competency systems often become very upset when a subject matter expert suggests major changes. This is unfortunate because in most cases a wrongly-designed system will inevitably fail within a few years. There is really only one best-practice way to select managers or individual contributor; like it or not, the rest are flawed in one way or another.

Careers, Degrees, and Experience

Moving up the expertise ladder involves a combination of practical experience (i.e., actually performing the work), mastering its theory and technical aspects, and working with a variety of clients to broaden exposure to multiple situations. In my profession, for example, one has to be thoroughly proficient in job analysis, competency measurement, ADA, validation, multi-trait-multi-method measurement practices, 1978 DOL selection guidelines, and APA test-development protocol. If someone does not master these basics, they are not qualified to do the work.

Another indication of a non-expert is when someone tries to enter the field with an unrelated degree. For example, all test developers are expected to complete graduate courses in test design, job analysis, assessment, validation, experimental design, statistics, and so forth. The objective of these programs is to minimize error and maximize accuracy. If your consultant’s qualifications and experience are limited to recruiting, training, or an SPHR designation, their expertise is probably limited to being a pair of hands or facilitator. Don’t misunderstand. There is nothing wrong with hiring a pair of hands or facilitator if that is all you need; but, if you want to test candidates for job skills, build an integrated competency-based system that actually works, or head off legal challenges at the pass, it is not enough.

Article Continues Below

People are also usually unaware that it takes more than a psychology degree to be a qualified job psychometrician. Out of a 72-hour graduate program, for example, industrial, clinical, and counseling psychologists have only about 25% of the courses in common. The other 75% are specialized. So, while psychologists might have the same letters on their diploma, counseling and clinical psychologists are trained to help people function in society, not predict job skills for the workplace. If your hiring expert provides you with a report that looks like a mental health evaluation, it’s a clue you are dealing with the wrong kind of psychologist. Even within my field (i.e., the practical application of psychological principles to solve business problems) only a small percentage of graduates are true test experts.

As someone who both worked in the business school of a large urban university and earned two business degrees, I have also found that many business professors are better at theory than application (e.g., with the notable exception of accounting and computer science). For example, while my management professors often treated the MBTI, Hawthorne studies, and Maslow as sacred cows, my psychology professors conducted controlled studies and looked for proof. It was an eye-opening experience to read study after study debunking many business theories I thought were rock-solid.

As a case in point, you might recall Professor Mike Hammer’s popular book on redesigning jobs. Hammer’s career expertise and education included engineering and computer science. His co-author, Jim Champy, was an engineer and lawyer. It does not take a rocket scientist to realize that when jobs change, so do the skills necessary to perform them, yet these authors only devoted a page or two selecting people who could do the job (e.g., a field industrial psychologists spent the last 100 years studying). If you share Hammer and Champy’s assumptions that people are sufficiently plastic to do almost any job; ask a few bankers how much time and effort they invest getting tellers to cross-sell, or how many successful individual contributors you know who are successful managers.

Blogger Alert!

Bloggers are cropping up everywhere with opinions that are generally poorly informed. For example, one person commented on an article I wrote some time ago inferring they said as much the same thing in a presentation. I had never heard of the blogger and was curious about their qualifications to make this statement. I did a little background research and within about five minutes learned the blogger’s competency qualifications were limited to working in HR and being a trainer. Sorry, folks, being trainer or working in an HR department does not qualify one to be an expert. Be wary of bloggers who have an ALL CAPS opinion, but lower-case expertise. That’s why you will never see my name on an article about organizational development, reengineering, recruiting, compensation, or neurosurgery.

In Conclusion

I would like to offer a few thoughts I have learned along the way: one cannot identify another technical expert unless they already are one (that means we are all pretty dumb about a lot of things); the more expert one becomes, the more they realize what they don’t know (yep, it’s OK to feel really stupid most of the time); an expert will have a combination of advanced academic education (to master theory) and practical experience ( to separate theory from reality); experts belong to professional associations where expertise is a condition of membership; an expert in one subject is usually painfully ignorant of another; real experts are the first to admit they don’t know much; the more certain someone is about his or her expertise, the dumber they usually are; experts can always produce legitimate 3rd-party proof of their claims; strong (or loud) opinions are no indication of expertise; everyone has an opinion about something; and, be VERY wary of the person who does not know what he or she does not know (sometimes you need an expert to know an expert).

Topics

34 Comments on “Who and What Can You Trust These Days?

  1. A couple things:
    “There is really only one best-practice way to select managers or individual contributor;”
    If it is so, what is the way, and why isn’t it always used? Would it work for all types of organizations under all circumstances? Also, what is “best”? Is it fastest, cheapest, highest caliber-individual for the current position, highest caliber-individual for a career with the company, best organizational fit? I haven’t seen evidence that there is only one best way, and would appreciate seeing some.

    As there is little advanced academic education related to recruiting, and there seem to be no universally-regarded professional recruiting associations (comparable in prestige/clout to the AMA or American Bar Association), it therefore appears we have very few experts on recruiting, with notable exceptions such as yourself. That certainly bears out my impressions. (But what do I know?)

    Thanks for your articles,

    Keith

  2. I don’t know if you meant it tongue on cheek, but there are many experts in assessment and measurement…every year hundreds of them graduate from great schools…I even know of one big temp staffing firm that has a few…And, they are on almost every Fortune 1000 payroll…However, I have only met one (1) professional recruiter willing to apply best practices. Why? I suppose it’s because recruiters tell me they measure success by who survives the guarantee period.

    Professional organizations? Check out the APA (APA.org) and SIOP (SIOP.org).

    As to one best way… I have been writing about that for years: job analysis and validated tools (NOT unstructured interviews, magic questions, or silly tests).

    Does it work for all organizations? Yes. Under all circumstances? Yes. Why isn’t it always used? It takes work, skill, and time.

    Why is it the best? A better question would be what part of best practice do you disagree with: thoroughly understanding the job, basing tests on business necessity and job requirements, using using tools that are trustworthy, hard to fake, have proven job predictability and conform with the EEOC intent?

    I may be one of the few selection and assessment voices on ERE, but I am not the only one in the field. I cannot help you find people, but I sure as hell can tell you whether they can do the job.

  3. I don’t know if you meant it tongue in cheek, but there are many experts in assessment and measurement…every year hundreds of them graduate from great schools…I even know of one big temp staffing firm that has a few…And, they are on almost every Fortune 1000 payroll…However, I have only met one (1) professional recruiter willing to apply best practices. Why? I suppose it’s because recruiters tell me they measure success by who survives the guarantee period.

    Thank you, Dr. Williams. I think in my case it was more “hoof and mouth” or “cheek by by jowl” as opposed to “tongue in cheek”. If you mean assessment experts as opposed to recruitment experts, there are many fine assessment experts such as yourself who meet the criteria. However, I was speaking about recruiting as opposed to assessment. I would like to meet that recruiter – s/he and I can start working on developing what I’ve called “Generally Accepted Recruiting Practices” or at least find out if such things are possible/feasible. (I’m not sure that they are.)

    Professional organizations? Check out the APA (APA.org) and SIOP (SIOP.org).

    Again, there are many professional associations for recruiting and its specialties, at the same time, they do not have the power to restrict my entry or hiring into the field if I don’t belong, or increase my income if I do.
    As to one best way… I have been writing about that for years: job analysis and validated tools (NOT unstructured interviews, magic questions, or silly tests).

    It seems to me that a single method that could equally hire the best dishwasher in a mom-and-pop restaurant, a non-lead actor in a play for a professional regional theater, a mid-level teacher in a tough urban school, a starting quarterback for an NFL team, an entry-level administrator, a neurosurgeon, and a CEO of a Fortune 20 firm would receive much more acceptance and general knowledge of its existence.

    Does it work for all organizations? Yes. Under all circumstances? Yes. Why isn’t it always used? It takes work, skill, and time.

    IMHO, that’s like saying a Stradivarius can always produce all types of beautiful music, but only in the hands of a virtuoso. In the real world, we rarely simultaneously possess “work, skill, and time”- “two out of three ain’t bad”.

    Why is it the best? A better question would be what part of best practice do you disagree with: thoroughly understanding the job, basing tests on business necessity and job requirements, using using tools that are trustworthy, hard to fake, have proven job predictability and conform with the EEOC intent?

    These all seem like they could be part of best practices, but “the devil’s in the details”- how are these terms defined, in what context do they operate (very carefully defined parameters or loose ones), is this a snapshot of “now” or a prediction of the future, does it take into account cognitive biases (both the company’s and the candidate’s), group dynamics, and plain-old likeability?

    I may be one of the few selection and assessment voices on ERE, but I am not the only one in the field. I cannot help you find people, but I sure as hell can tell you whether they can do the job.

    I value your and at least one other assessment professionals’ contribution to Recruiting. If I may add my own contribution to what an expert is:
    “An expert is someone who says they have AN answer, not someone who says they have THE answer.”

    Cheers,

    Keith

  4. As usual WW, you’re mostly right and make lots of good points, but you also throw down some provocative claptrap guaranteed to raise a bit of controversy.

    “Bloggers are cropping up everywhere with opinions that are generally poorly informed”

    Really?

    Here is a famous rant, now laughable in its utter smugness and failure to recognize a major sea-change: http://www.cbsnews.com/stories/2004/11/08/opinion/main654285.shtml

    Bloggers have hammered and disintermediated many an old tyme journalist, as they no doubt have displaced some demand in the HR consulting industry.

    “An expert will have a combination of advanced academic education”

    Across all domains of human knowledge and achievement? Or simply in the narrow areas of I/O Psych that you happen to work in (and similar guilded professions?) I would not want an uneducated surgeon working on me or mine, but I know there are plenty of great engineers, designers, backroom lawyers, farmers, and an endless list of people out there without the credentials to do what they are actually doing at a superb level.

    Credentialism is sometimes critical, and other times just a means of class and social division. Blanket statements regarding the development and recognition of expertise are really no more useful than the stylings of newbie bloggers.

    To be sure, I agree that education and experience are needed to create quality pre-hire assessments that have a hope of conforming to objective reality.

  5. Well M. Gladwell in “Outliers: The Story of Success” says that it’s 10,000 hours of focused practice, with widely varying initial educational experiences. Education can mean many things- if you mean a formal, actual degree, I don’t think its a strict requirement for expertise in many areas of life.

    OTH, for some areas, the degree is the sine qua non, so you can’t just make blanket statements about it. I think expertise is in the doing- the ability to produce the performance. Sometimes the performance is an unforgiving hard fact (life or death) and others a matter of art or feeling. Consulting can be some of both, especially HR consulting, because humans are complex and cause and effect can be very difficult to identify or act upon.

  6. Unfortunately (and fortunately, in some areas), the world has changed. Most of what is read and thereby shared, commented upon, tweeted, etc., is opinion. People love op-ed, hence the explosion of the blogosphere.

    Example: Launch a whitepaper and a blog rant on the same morning, and see which one generates the most buzz and is most amplified through social media channels.

    It’s typically more enjoyable to discuss/debate opinion than read statistical analyses or peer-reviewed research. I’m not saying it’s right; it’s just human nature.

    Anyone that wants to exert broad influence, particularly true industry Experts, would be well served to begin blogging themselves.

    Simply take your insights and inject them with some humor, candor, etc., within a blog or vlog. In the end, your influence will expand farther and more rapidly.

  7. So Dr. Williams you raise the question that has plagued employment practice more than any other and could be one of the leading factors to the stagnation of American Business Growth (OK, maybe a bit over the top…)- the ability to repeatedly put people in jobs that will glean their highest level of performance.

    I am no I/O Psyche expert, but I do try to surround myself with those that are (we have one of your esteemed colleagues on our Advisory Board for example), yet no one has ever been able to break down the selection process into an affordable, easy to use method where the predictive results are better than a coin flip (50%). No question that to get a validated result, an assessment needs to be crafted for the specific job, company and industry – and no two jobs are alike. You mention that it takes, “work, skill and time,” I totally get it, and that’s the real point actually – to get a validated result is too much work for the thousands of companies with 100-500 employees that fuel our economic engine (and many of the larger ones don’t use the resources available and seem willing to carry 20-30% underperorming employees on their pay rolls). We would LOVE to change that – but the truth is that we don’t have the tools the market is willing to use…

    Martin certainly makes your point using Gladwell’s assertion of 10,000 hours of focused effort combined with a base of concentrated learning (academia, mentors, peer network, etc.) to have the “goods” to be an expert. As an expert in this space, can you please help us figure out the age old answer of how to assess using validated methods that won’t require a small team of experts ($$, time, difficult to use) to craft a new testing instrument each time we want to evaluate a person’s ability for a role.

  8. It is always interesting reading an article detailing the true qualifications of an expert by an author who just so happens to fall into all of the categories necessary… 😉

    That conflict of interest aside, I think there are two things “experts” need to come to grips with: First, the democratization of information on the web means anyone can comment, off-handedly analyze, reference or blog about any topic. You must let go of the notion that the general population and HR community will ever scientifically discern what a true expert is going forward…and that is a good thing (and something you may want to blog about). Which brings me to my second item. The audience will forever decide going forward what is useful expertise and what is useless. I find that to be ruthlessly efficient. No checklist of degrees, required years of experience or certifications will equate to an expert designation in the current opinion driven climate. The usefulness and insight that even a lowly blogger might offer will be determined by the industry audience. They are the ones relating it to their current work experience and what if any applicability it has to their business. That is a determinant of best practice and expertise as much as an assessment tool developed by a degreed academic/industry vet (if not moreso).

    After all, you never said if the HR trainer who commented long ago made a valid point. Who cares if it did not come with a complete expert ‘checklist’ behind their name.

  9. Joshua – excellent points, totally agree. There is a responsibility on the part of true experts to share the wealth.

    K.C. – please see the following site which shows how validated assessments ADD to a coin flip chance: http://www.gatelyconsulting.com/STATSORC.HTM

    I think maybe some of this debate stems from different ways of looking at the problem. There is the “ideal” where every selection mechanism is crafted and reviewed by an I/O psychologist, and the “real” where those organizations with time and resources (as K.C. points out) are able to do things like this, but most organizations simply cannot.

    But what isn’t being acknowledged is that the most common types of assessment are resume screening and interviews. And there is much that can be done–without great expense–to add validity to those processes (particularly the latter). There are many true experts that have written very accessibly about this topic, for example: http://maamodt.asp.radford.edu/Research%20-%20IO/2005-August-Interviews.pdf

    I think to some extent people like Wendell and Charles H. are reacting to the lack of respect given to assessment, which is infuriating to those of us who understand how important they are.

  10. It seems we need a Yelp- or Glassdoor-type evaluation system for professional experts.

    Meanwhile, some quotes on experts:

    “An expert is someone who knows more and more about less and less”

    “Make three correct guesses consecutively and you will establish a reputation as an expert.” – Laurence J. Peter

    “An expert is someone who has succeeded in making decisions and judgements simpler through knowing what to pay attention to and what to ignore.”
    Edward de Bono

    “An ordinary man away from home giving advice.” – Oscar Wilde

    “An expert is someone who knows some of the worst mistakes, which can be made, in a very narrow field.” – Niels Bohr

    My favorites:

    “What a delightful thing is the conversation of specialists! One understands absolutely nothing and it’s charming.” – Edgar Degas

    “Do not be bullied out of your common sense by the specialist; two to one, he is a pedant.” – Oliver Wendell Holmes

    “An expert is a person who avoids the small errors while sweeping on to the grand fallacy.” – Steven Weinberg

    “My definition of an expert in any field is a person who knows enough about what’s really going on to be scared.” – P. J. Plauger

    Cheers,

    Keith

  11. Thanks Bryan – I am familiar with the graphs on hte Gately link…and that is surely an esteemed group of Phsyche folks – but I will stand by what Wendell and Charles have been saying all these years since they really play in the real world of talent management and recruiting and don’t sit at a desk in a professors office…

    I have to disagree with you wholeheartedly about resume and interviews as a good tool for assessment – in fact I would say that together they are one of the biggest reasons for the high annul turnover and underperforming employees typically found in large American companies…

    Validated job performance testing is the best tool that actually can predict an employment success or failure. The problem is the difficulty in doing it…

  12. Just to clarify, I didn’t say resumes and interviews are good tools, I said they are the most common ones. I agree that they aren’t typically done well.

    The problem I see is that “validated assessment” is often treated as voodoo or some unobtainable concept pushed by ivory-tower-dwelling-never-set-foot-in-an-actual-business-academics (of which in my experience there are few). In fact, good assessments are very much within our grasp.

    Oh, and by the way, a lot of the research done by ivory tower dwellers helps us understand what we should be doing.

  13. May I suggest you check out Mr Gately’s credentials to sell assessment systems…I believe Profiles International licenses a lot of independent distributors. I do not know what their licensing criteria is, but the last time I heard, Mr. Gately was a (retired?) professional engineer…not a selection and assessment expert. The many charts and graphs on that site are taken out of context from old studies…It goes to prove my point about people entering the market without expertise.

  14. @Dr. Williams,

    I took your comments as a hint and emailed Yelp (http://www.yelp.com/contact) suggesting the value of a “Business Consultants” category- companies would find it valuable to rate and review how various consultants and experts work for them. I don’t expect much to come from it, but if a number of our “Gentle Readers” did likewise, who knows?

    Can anyone suggest an alternative site for this?

    Happy Friday,

    Keith “Appreciates Helpful Suggestions” Halperin

  15. Keith I’m not sure the ratings provided by proles like the everyday users of Yelp would be helpful to establish expertise-

    WW, how do you feel about crowdsourcing for validation of expert opinion ?

    We count on it for elections and stock markets(in other words, not so effective!)

  16. @ Martin:
    1) That was condescending to the many Yelp users. (BTW, that doesn’t include me- I don’t use Yelp.)
    2) I am not suggesting crowd-sourcing to verify expertise(ALTHOUGH SEE: http://en.wikipedia.org/wiki/Wisdom_of_crowds), I am suggesting posted customer feedback to evaluate service providers, who in this case would be business consultants.
    It would also be useful for a similar site for business and other consultants to evaluate clients…

    Cheers,

    Keith

  17. Im my experience, it takes an expert to recognize an expert..Non-experts usually are impressed by immediate outcomes (at the expense of long term costs). For example, one of my clients started using personality tests to select employees based on teamwork. After a few years they woke up to discover they an assembly plant staffed with “bleeding hearts” (their term, not mine)…That is, their employees were so relationship oriented, they would not confront production problems for fear of hurting someone’s feelings.

    When the problem popped up, the “expert” who recommended selection based primarily on teamwork was long gone.

  18. Keith, the closest thing I can think of is honestly.com, which is more targeted to co-worker relationships.

    The problem–as we’re seeing–is how to identify experts. And honestly WW’s suggestion of looking at formal credentials is–if not perfect–the fastest way of at least getting someone in the ballpark.

    In some ways this is no different, coincidentally, than a hiring scenario: how do you find the right expert? Why…you assess them of course.

  19. @John Rorick, re: “The audience will forever decide going forward what is useful expertise and what is useless. I find that to be ruthlessly efficient. No checklist of degrees, required years of experience or certifications will equate to an expert designation in the current opinion driven climate…”

    I would rather deal with someone who has the certifications, education, and work experience than to rely on the public court of opinion, or some blogger…after all you know what they say about the masses and about opinions.

  20. @Marcelo – I’m not saying I would not rather deal with someone with credentials, etc, but let me ask you – How often do you read online reviews regarding cars (edmunds.com) or general products (cnet, amazon) etc and draw conclusions based on themes and opinions that are being shared by the crowd? My guess, most of the time. If 10 bloggers/reviewers share one opinion about a car but one “expert” who was a mechanic for many years has a contrary opinion which viewpoint wins??

    The fact of the matter is this will be the context of expertise offered from here forward (or the validation of expertise). Argue with its value all you want, but that in my opinion is the world we now live in, and I see some benefit. If something resonates with the crowd as effective expertise then it will likely be listened to…It is dangerous to also presume that somebody with a PhD knows anything. It could stand for “piled high and deep” or “guru in their discipline”. Just because somebody paid for a certification class and passed does not mean they are an expert in anything. It means they took the class. Only when they share their experience and OPINION do I begin to draw conclusions regarding their expertise. I can not imagine I am alone in having experienced as many worthless consultants as valuable. Degreed or otherwise.

  21. Maybe I can clarify a few things…expertise = all the relevant research on the topic + practical experience. One without the other is bad news. The experienced-based person who knows nothing about the research is acting on limited knowledge. The egghead who operates without experience is equally clueless.

    My suggestion is to look for both. When I was a trainer-practitioner I was told the MBTI and DISC were phenomenal…When I actually read at the research supporting the two instruments, I learned otherwise. In B-school, I bought into Maslow’s theory. When I read the research on the subject, I learned otherwise. The story goes on.

    My suggestion when it comes to testing and interviewing, learn all there is on the subject before voicing an opinion, or least treat claims with considerable caution. Sometimes a simple question like, “Was your test designed to predict job performance?” is enough. Be especially cautious of tests that were developed for training and the vendor is attempting to enter the hiring marketplace. Training tests illustrate differences. Selection tests written and behavioral interviews) predict job performance.

    Caveat Testor!

  22. @John Rorick – “How often do you read online reviews regarding cars (edmunds.com) or general products (cnet, amazon) etc and draw conclusions based on themes and opinions that are being shared by the crowd? My guess, most of the time. If 10 bloggers/reviewers share one opinion about a car but one “expert” who was a mechanic for many years has a contrary opinion which viewpoint wins??”

    Once again we disagree. In your example I would likely listen to the expert, rather than the crowd, but that person needs to be more than just a mechanic to be called “expert”. As for the “listening to the crowd” approach, too often these days the person who makes the most noise is declared “the winner” in the public court of opinion, without much attention given to actual substance. The masses are generally too lazy to do the work needed to be informed/educated, so instead they react to noise or slick marketing…which is why I’d rather go with the experts so long as you’re not using the term loosely…just my opinion.

  23. @Marcelo – So if in my example that person needs to be more than just a mechanic for you to be called expert, we actually agree 🙂 (and I do believe for a second you would read reviews on edmunds by posters all saying they have problems with the transmission and throw them out when a “expert” claims something to the contrary…even if they are MORE THAN a mechanic). But I think we are arguing different points.

    Your comment “As for the “listening to the crowd” approach, too often these days the person who makes the most noise is declared “the winner” in the public court of opinion, without much attention given to actual substance. The masses are generally too lazy to do the work needed to be informed/educated, so instead they react to noise or slick marketing…which is why I’d rather go with the experts so long as you’re not using the term loosely…just my opinion.”

    We agree as my general point is this is the world we live in, and it is NOT changing. You say as much in the above comments. I understand you are trying to make the point that you are not lazy and are well-informed, but there is something efficient about letting the crowd (assuming it is the right audience) decide for themselves wha

  24. @Marcelo – you are actually agreeing with me. This is the world we live in, and it is not going to change. Nowhere did I say I want to follow the recommendations of the lazy uninformed crowd, but I do see value in the current climate in the democratization of information on the web and letting the online audience (assuming it is the right audience) determine if it is worthwhile or not. You seem to want to couch my earlier comments as somebody who determines expertise based on some American Idol voting approach, without thought. So be it. But my main point is this is how things are and I do see some value in it.

    Regarding your disagreement on the mechanic example, I do not believe you for a second… 🙂 If ten people write they had problems with a cars transmission and one “expert” (even more certified than a mechanic) says something to the contrary I doubt you would disregard crowd opinion. I can only assume you are a car expert yourself and if so, I will make sure to consult you during my next purchase (that was meant in jest) Enjoy the weekend all.

  25. I subsequently realized there are at least two sites where clients review contractors/consultants for their work oDesk (www.odesk.com) and e-Lance (www.elance.com).
    ODesk lists 48,760 contractors/consultants in Business Services and e-Lance lists 13,588 under Finance and Management. (Not many recruiters are listed, though.) I think I may go there to see who clients think the good experts are….

    Cheers,

    Keith

  26. Great debate here on an important issue. Beyond opening the can of worms regarding how well assessment works, the bigger picture for me is that there are a lot of mediocre products out there peddled by so called experts. It is something I see on a daily basis. Of course there is more than one way to skin a cat and there are many great products that take different approaches that support this. But, vendors are salesfolks and they behave as such. As a consumer of assessments, it pays to ask the right questions and to look for the red flags. The hardest part of this is that many of these mediocre products are not worthless or shameful, rather they just leave value on the table for the companys that use them.

    We all need to work to be, skeptical , educated consumers of tests or anything else for that manner.

    Thanks WW.

  27. @Keith, my comment about yelp users being proles was /.

    @Charles, the article headline was “Who and What can You Trust These Days” and WW seemed to be riffing on the idea of expertise in general in the Internet age, rather than merely his area of pre-hire assessment. Perhaps a theory of knowledge which thesis seemed to be that only people who have formal education can really be experts?

    In contrast, the meme of the age seems to be that crowdsourced opinions can simulate or extend individual expertise, which may implicate group intelligence, which MAY say something about pre hire assessment.

    Jay Cross had an interesting (and maybe pertinent) little essay on Sumser’s site the other day: http://www.hrexaminer.com/getting-the-work-done

Leave a Comment

Your email address will not be published. Required fields are marked *