All hat, no cattle!

October 3, 2017
Hang around Texas long enough and you’re sure to hear this saying. I suppose it’s a bit of an insult, but in general it’s a useful saying to describe something, or someone, more concerned with appearing to be something than with actually being that thing. Watch out, the assessment industry is full of concrete cowboys!

Let’s start with why a company would choose to use a behavioral assessment as a part of their hiring process. Simply put, the answer will be some form of “to make better hires.” Going a little deeper, this typically means selecting new employees who perform at a high level and stay employed with the company for a long period of time. “Performing better” can mean different things in different industries or jobs, but the concept applies across the board. Making better hires is the “Cattle” but everywhere you look you are bombarded with “Hats.”

The Ten-Gallon Hat: One of the newest talking points in the market is around using artificial intelligence (AI) to analyze facial expressions, words, and voice patterns from video interviews to try predicting a candidate’s likelihood of being a good hire. Putting aside the obvious concern about being the first employer to get hauled into court over the use of this questionable technology, there are plenty of reasons why this approach is highly flawed.

First, there is limited usability of AI technology in a high volume hiring process. The candidate completion rates of these sort of tools used early in the application process is untested and is most likely going to be extremely low. While the concept sounds interesting, you will inadvertently be reducing your applicant pool. A large number of candidates simply will not complete the process that you have laid out.

Along with the obvious technology challenges (even though you may have the latest iPhone, it doesn’t mean that all of your candidates do) consider the applicant who is applying for a job while taking the train to work, or the applicant who is applying for jobs from the comfort of his bed on a Saturday morning. The logistics of asking this person to complete the very unnatural task of holding their phone in front of them and talking to their own picture are daunting. You couldn’t think of a more intrusive, frustrating, and uncomfortable experience than that. How many of us would feel the need to get redressed into our work clothes and get to a quiet and professional environment for this part of the application process?

Beyond these challenges, the underlying theory and technology is highly questionable when it comes to results. There has been ZERO independent analysis showing that this approach will produce better hires than a traditional selection process. The idea has been debated over the past few years by Industrial Psychologists and there is no consensus on the validity of this approach. While you will have the benefit of reviewing a video interview for the few candidates who do complete the process, you will not be making better hires and you will be losing a large portion of your applicant pool simply because they will not complete your process. That’s an awfully big hat for as few cattle as you’re going to get.

The Stetson: Playing games or participating in simulations to help uncover a job candidate’s behavioral preferences has also been getting a lot of discussion recently. While this approach does attempt to capitalize on the proven theory of behavioral fit being an important predictor of job success, it fails in a few respects.

With the number of job applicants using their mobile device for their job search, the differing experiences that a candidate would have with the game/simulation on their phone versus on their computer is a major challenge. For legal defensibility, you need to ensure the process shows no difference when completed on different devices. This is not possible with most of these games and simulations. You end up having to force all of your candidates to use a full computer to finish their application process and nothing screams “2005” like not making your full application process available on a mobile device. And don’t fall for the line, “We have an app for that.” Having your application process scream “2010” is no better than having it scream “2005.” It is not acceptable to force a candidate to download an app just to apply for a job at your company, and many of your candidates will simply move on to an employer who offers a more modern and flexible application process.

Another major issue to be considered is the disparate impact that these approaches are bound to show. When asked to play a game to see how quickly you can control your keyboard, it is highly unlikely that an older applicant is going to have a similar result to a millennial who grew up with their hands attached to an electronic device since they were 10 years old. You will often hear vendors tout the fact that there is no gender or racial bias in their process. However, they will be careful not to mention age bias. Unless you’ve been living under a rock, you already know that you could get sued for that. Vendors will try to deflect their age bias by saying that they adjust their models to attempt to control this sort of bias, but in reality they are simply minimizing the predictive weight of the attributes in their models that show this sort of disparate impact against a legally protected class. Imagine getting hauled into court over this one….

Additionally, games and simulations are incredibly long and expensive to build/maintain/adjust. Once you finally build a model that you think will work, you will likely be asking your candidates to spend an inordinate amount of time playing these games to measure a broad enough set of behavioral attributes to be useful. Ultimately, you will likely have to shorten the experience, which means only measuring a few attributes. This makes for very unstable and error-prone results.

The tools employed in this methodology do not reliably and accurately measure work related behaviors used to predict employee performance. They sure sound good and in many cases appear to be fun or engaging, but ultimately don’t produce the outcome that is desired (unless your desired outcome is to entertain your job applicants for a long period of time). Just another example of “all hat and no cattle.”

Big Hat, Lots of Cattle: Over the past 17 years Infor Talent Science has delivered the actual outcomes that matter when a company chooses to use a behavioral assessment in their hiring process. We continue to advance the predictability of our product as well as incorporating innovative ways to make the process easy for applicants to complete.

The most dangerous reason to listen to these concrete cowboys who spend their time and money on chasing the latest marketing buzzwords is the actual predictability of their models. The Ten-Gallon Hat and the Stetson spend most of their time on the front of the assessment process which collects data from job candidates. But the true predictability of an assessment process comes from building accurate predictive models against which to compare candidates. For the past 17 years, companies have been chasing the dream of building models that are as predictive as the ones built by Infor Talent Science. No one has even come close. Our patented process of building these models is why we are able to talk about documented, statistically significant results like reductions in employee attrition and increases in employee performance while others speak of their results in generalities.

In fact, for any company that is interested, we would be happy to do a full analysis of your turnover and performance data to show you exactly what is going on at your company, all at our cost. Now that’s a lot of Cattle to go along with our big Hat.

Make sure you click the Follow button on this page and also visit our website at Infor Talent Science for more info on how we are delivering on the promise of hiring better people.

Patrick McKittrick, VP and General Manager, Infor HCM
  • North America
Let’s Connect

Contact us and we'll have a Business Development Representative contact you within 24 business hours

Infor values your privacy.