Online Personality Tests Are they really all they’re cracked up to be?
Personality Test … Psychometric Test …”Online Aptitude” Test … Profile Test …
They may be called different names but all accomplish the same thing:
They often stop potentially rock-solid candidates dead in their interview tracks if the scores don’t add up to what the company has decided is a Passing Grade.
If you’re a recruiter with any number of years in the business you may be experiencing this infatuation with personality tests more than ever.
Here’s how it often unfolds:
You are given a job search set of instructions.
You invest weeks sourcing, recruiting, networking and following leads until you find two highly qualified candidates.
Both come highly recommended by former bosses, co-workers, and nearly everyone you have conducted formal or informal reference checking with. Maybe you even know the candidate from first hand business relationships.
Things go very well … First interview with Human Resources … and a second interview. But at some point someone tells you the candidate must score well on their “Profile Test we require everyone in this unit to take.”
You gulp. And your apprehension leads to outright anger when you find out both candidates “flunked” the test and despite all your “Due Diligence” that went well beyond what the test is capable of doing … a scoring system and rating criteria unbeknownst to you (or the candidate) has knocked your candidate clear out of the ballpark.
For recruiters working professional discipline areas such as Customer Service, Client Relationship Management, Sales, Business Development, Marketing … “Personality Tests” have become increasingly popular as companies attempt to toss whatever they possibly can out in order to gain some appearance of screening and filtering for retention.
Other factors contributing to the Profile Testing Stampede is that testing prices have dropped considerably (you get what you pay for). This coupled with the fact that they are now more easily administered instantly, via online web portals, has increased their popularity and usage.
This does not always bode well for the hiring company or their recruiting representatives however.
I have found that too often, such tests produce confusing and outright conflicting results. I have even had some candidates read the questions to me asking me for explanations to multiple choice questions which can be very tricky even for an experienced pro such as myself that has administered scores of such tests in my previous corporate life and now in recruiting.
I remember the time someone tested “HIGH” for competitiveness as a Personnel Recruiter here at IRES … but couldn’t debate, talk or negotiate their way out of a paper bag despite weeks of different types of training.
On the other hand, one of our current best producers scored 55% (barely above average) in terms of her sales ability on the “test” … yet went on to be our most consistently productive account executive. Having been convinced by twenty years of hands on experience that such Profile Tests may be no better than tossing a coin and choosing heads from tails, I decided to dig deeper and get my facts straight to see if my hunch was right.
I began by calling the nation’s best. I contacted Dr. Wendell Williams of www.scientificselection.com in Atlanta. With a Phd. in industrial psychology, and a member of numerous organizations including the American Psychological Association, Society for Industrial/Organizational Psychology and others, Dr. Williams has authored many recruiting articles and is known as the “Guru” of corporate psychological tests. His firm’s services are frequently used by many Fortune 50 corporations to validate existing tests and he is hired to actually create custom tests from scratch.
Following are excerpts of our telephone interviews and discussions, which spanned late summer through fall this year:
Risalvato: “Are ‘Profile Tests’ such as those frequently administered online by companies such as Profiles International, Omnia, Caliper and the likes actually useful in identifying a candidates’ ability to perform a job? Are they reliable?”
Dr. Williams: “First we must define what a “TEST” is. A test is any portion of the process that screens out. Unless you are hiring EVERYONE that applies, you are screening and when screening you are applying some form of test.”
Risalvato: “So in other words when I pay attention to someone’s energy level, tone of voice, ability to enunciate and articulate … that is a test in itself?”
Williams: “Yes. And if you decide to throw the resume or application out that person failed the resume matching or application-matching portion of the “test” in a sense.”
Risalvato: “So what would you call the popular Online Tests I just described which we are finding companies use with greater frequency?”
Dr. Williams: “Some are great examples of junk-science. They might measure personality preferences in a classroom, but were never designed to predict job performance. For example, do all of your sales people have the same personality? Of course not! But vendors who sell personality tests would like you to think so.”
Dr. William added that companies generally know that “Having Great Impact”…(something that usually is code word for appearance, height, good looks, poise and other attributes individuals may be helpless to change about themselves unless they undergo an extreme makeover) is highly subjective: However, what they often don’t realize is that online personality profiles can be just as unpredictable and subjective.
His web page www.scientificselection.com gets into this phenomenon more in depth.
Another web page on his website clearly illustrates the low credibility that should be placed on such pre-packaged, one-size-fits-all Online Profile Tests. In fact, if you click here www.scientifselection.com you will see a summary of the predictability of all hiring tests. This table summarizes a broad range of studies (i.e.. a meta-analysis) done by independent scientists of hiring tests and their effectiveness. The table shows personality test scores actually predict performance about 4% of the time. In other words 96% of the time they are no better than a coin toss.
I never thought personality tests were great but was surprised at how poorly these studies rated them. Here’s the table he has listed on this web page:
Selection Method "Predictability"
Adapted from a meta-analysis conducted by Hunter and Hunter, Psychological Bulletin, Vol. 96, 1984. Percentages have been rounded. Predictability refers to the explained variance.
But why are they so unreliable?
As Dr. Williams and I continued our discussion, he added “lying and deceiving on such tests” is one reason while another is that “the test has little or no relationship to the actual job being performed”.
In other words you can be a “hunter” or “farmer” (as some tests classify personality traits) … or an “I, N, T, P” as rated by Meyers-Briggs and still be able to perform the job regardless of fitting into different ends of a personality spectrum.
Whoa. This gets deep. The purpose of this article is not to get bogged down in statistics and analysis, but to provide a broad overview. So let me back up.
The only test that is truly credible according to Dr. Williams, is one that tests critical aspects of the job such as driving a car. These are described as Content Valid Simulations. Such tests are reliable 64% of the time.
What is “Content Valid Simulation?” Simply put … if you are applying for an airline pilot you ought to do well in the flight simulator test, because that specific test precisely simulates what flying a plane will be like. In other words the “content” of the test simulates an important aspect of the job. There’s no way to fake such a test unless you really know how to fly.
If you were applying for the Olympics you would be asked to throw the javelin and be measured for distance. If you were applying for a tennis team you would be asked to serve, be rated on your serve and then be rated on your backhand and forehand. It would be hard if not impossible to fake a backhand as you will actually have to execute it.
For Recruiters, you would be placed on a telephone and asked to make a cold call to a president and read from a sample script. End of test. You either, pull it off or not. Or you may be placed on a mock interview and asked how to initiate the interview questions. If you can truly make a sales call, you should have little difficulty performing such a “Content Valid Test”.
For a MS Excel Spreadsheet Analyst: You would have to create a 6-column by 6-row spreadsheet and demonstrate how you would apply formulas for subtotals – Again, hard to fake this.
If you are applying for a Field Marketing Rep or Territorial Sales Professional, where you’d have to drive distances each week and interact with clientele … well … you get the idea.
As Dr. Wendell Williams describes it … anything less than “Content Valid Simulation” is junk-science and filled with errors and misinformation. This much I can agree with. In fact, according to the Meta-Analysis table he was explaining, Traditional Interviewing (which is most of what corporate internal recruiters utilize) also rates a 4% predictability...no better than a test. This could also be said about external recruiters, but then again 90% of our job focuses on FINDING qualified candidates and not predicting how long they remain on the job which can be impacted by many factors beyond our control. This is also why having our candidates de-bunked by a ridiculous test is so frustrating. The test has no idea of the weeks and months (as well as a lifetime of human experience) that went into the search process which is for the most part more in-depth and hands on than a test.
Dr. Williams also points out that the 1984 date listed in the footnote of his meta-analysis chart is a representative summary. Every individual study would be slightly different, but he stated the averages and rankings have tended to remain constant up through the present.
Unfortunately there is so much information about testing he was able to provide not all can be incorporated into this article (I suggest companies interested in his testing expertise call him direct at 770-792-6857.)
The points Dr. Williams did make which are important to the recruiting profession and recruiting process are as follows:
I’d like to add my own thoughts after learning quite a bit about this frustrating subject, which often rears its ugly head to lay to waste to many weeks or months of exhaustive recruiting efforts. Just this summer and early fall alone, such Profile Tests have cost our firm well into the tens of thousands of dollars in lost placements and unrecoverable time. In at least four incidents the candidates were precluded from being invited to their first face to face interview based exclusively to their marginal test score result! (Despite months of recruiting time required!)
Here are my thoughts:
I recently spoke to Joe Smith (name changed to protect his identity), a candidate our firm actually placed into a position this summer and asked him “How did you manage to do so well on a test that struck down the last 4 individuals that took it before you?” His answer: “Simple Frank. I knew exactly where each question was going and gave them what I knew they wanted to hear. Not rocket science to figure out.”
I have received calls late at night by panicked candidates asking me if I’d help them as they found the questions so confusing. This presents a dilemma as on one hand I know how to score very highly if not perfectly on such tests (I’ve administered nearly every variety and have seen the answer templates countless times) … yet I must maintain professional distance so as to not influence the process … I usually do not provide answers but help them come to their own conclusion that they were on the right track all along.
So if you have been frustrated and angered by “Personality Profile Tests” killing your star candidates just as they were about to reach the Offer “Touch Down Zone” … you now have Dr. Williams and Risalvato’s findings you can make your client hiring managers aware of.
By the way, according to the Meyers-Briggs-Jung test (one of the oldest originally conceived around 1930’s) I’m an INFP. That means a) Introverted b) Moderately Intuitive c) Slightly expressed feelings and d) Slightly able to express my perceptions.
Oh Brother! Most people are unable to suppress my perceptions, feelings and opinions and find me too opinionated!
If you’d like to have some fun this holiday season and discover your own Meyers-Briggs-Jung Typology go visit www.humanmetrics.com right now for a simple, free, instant profile test requiring just 5 minutes of your time. You can even pass this around and have your family members take it for your own affirmation.
So there you have it. 96% of all standardized On-Line Personality Profile tests are no better than a coin toss as far as their ability to correlate to actual job performance and not worth the electronic space they occupy according to Williams and his cited studies.
By the way, I sent several requests to a few of the testing firms to give them an opportunity to reply to some questions. Only one replied (Profiles International of Texas). In sum I was told “…A hiring manager or company should only use the assessments as 1/3 of the decision making process. They should not make a decision on whether or not to meet with a candidate based on scores alone.”
I have used Profiles International tests myself for internal recruiter hiring for nearly ten years now. Every single recruiter must take a Profile Test ( www.profilesinternational.com ) before a contract is written. At the very least, the Sales Test we use tells us what areas we should focus our training on.
I don’t believe such tests are entirely junk as was suggested. I do believe if used they should be used judiciously and with caution as indicated by the quote above. You should have one eye on the test result and the other on the candidate sitting in front of you. What aggravates me is when I see companies allowing paper/pencil (or mouse/web) tests preclude an interview from ever taking place.
- Frank G. Risalvato, CPC CEO IRES, Inc.
Risalvato has appeared on numerous radio and TV business segments and has been considered an authority on hiring and staffing trends since 1987. He is frequently invited by Federal, State and Trade associations as a key and dynamic speaker and motivator --- Visit www.iresinc.com for free and immediately downloadable material · Main Tel: (973) 300-1010 · Email: firstname.lastname@example.org
Author/CEO/Founder of IRES, Inc., a national recruiting and staffing firm. Risalvato has appeared on numerous radio and TV business segments and has been considered an authority on hiring and staffing trends since 1987 --- Visit www.iresinc.com for free and immediately downloadable training material · Main Tel: (973) 300-1010 · Email: email@example.com