This is a guest post by Meredith Wan and Julian Shoun from Talent Data Labs. TDL was founded in 2017 when it became clear that Nova’s 20+ years of Top Talent experience meant we had one of the markets biggest datasets on top talent. This data and the knowledge it contained led to our first matching algorithm, and Talent Data Labs was founded to continue the development and to ensure that Nova always has top-of-the-line matching capabilities.

According to Bersin by Deloitte, over the past few years, we have seen incredible growth of HR analytics tools and their adoption rates. In 2018, Bersin reports that there is plenty of extra space for growth but almost a third of companies have reached the highest levels of maturity, becoming “truly digital innovators”.

We at Talent Data Labs [Nova spin-off company] wanted to look into these findings to see if, indeed, we are moving towards highly effective talent analytic capabilities or whether we are still throwing mud at a wall and seeing if anything sticks. To do so we researched three key topics:

  1. According to decades of research, we know that there are four consistently strong predictors of performance (see Table 1). But how frequently do we measure them all, and how well do we actually predict them?
  2. Do the algorithms out there use a single, multiple, or all four predictive traits of performance (see Figure 2)?
  3. How valid are the algorithms out there? How many suppliers clearly post and report findings or results similar to what is found in the research?

If you, like us, are curious to learn more about the state of HR analytics, and its flaws, then please continue reading.

The dreadful state of HR analytics

The majority of recruitment processes predominantly consider the skill- and job-fit of potential hires, whereas most talent analytics processes account for a variety of other factors as well – such as personality, organizational commitment, cultural fit, and other preferences. This has created a mismatch in available data and the required data, which typically prevents us from predicting long-term hiring efficacy. In an ideal scenario, by hiring for the long-term through data and talent analytics we can witness deeper relationships with longer tenure, reduced churn rates, and better employee productivity.

However, it is incredibly laborious and cumbersome to employ data entry or data gathering processes to improve talent analytics competencies. In the post-Big Data era that we live in today, it comes as no surprise that companies are increasingly turning to job matching algorithms to improve the effectiveness and efficiency of their talent acquisition processes. To understand how far we have come in terms of talent analytics capacities we carried out a detailed comparison and analysis of 20 competitors that utilize job matching algorithms.

1. Out of the four most predictive traits of work performance below, “skills” is most frequently considered.

Barring exactly measured scores such as General Mental Ability (GMA), the four traits that we, and over 100 years worth of research (Schmidt & Hunter, 1998), deem most predictive of performance are:

Table 1. Predictive traits: Description and rationale of traits most predictive of performance. 

Hence, we wanted to investigate the percentage of job matching algorithms that consider each factor when calculating the Talent Match scores for a role in a company.

As seen in Figure 1, 95% of algorithms take the skills that potential hires have into account. The second most frequently considered trait is the job and workplace preferences of job candidates (75% of algorithms). 25% of the algorithms incorporate the personality traits of applicants into the matching score. Finally, it is alarming that only 20% of the algorithms that we analyzed consider the cultural preferences of candidates since, according to the Society for Human Resource Management (SHRM), hiring someone who is a poor cultural fit can result in turnover costs of 50% to 60% of her annual salary.

Figure 1. Trait integration: Percentage of algorithms that consider the respective predictive traits of performance. Schmidt, Oh & Hunter, 2016

2. Most algorithms consider 2 out of the 4 predictive traits of performance.

Ideally, job matching algorithms should account for all 4 predictive traits. We were curious to find out the percentage of algorithms that considered the varying number of predictive traits in the process of generating a Talent Match score. As seen in Figure 2, we found that 70% of the algorithms accounted for 2 predictive traits, while 10% and 15% of algorithms accounted for 1 and 3 traits, respectively. Only a single algorithm took all 4 predictive traits into consideration.

Figure 2. The number of traits integrated: Percentage of algorithms that consider varying number of predictive traits.

3. None of the job matching companies provide access to validation studies

As demonstrated in Table 1, it is clear that the right traits – when measured effectively – are able to predict talent-organization fit to a great extent. However, what we asked ourselves was whether or not the methodology and findings of the algorithms were supported by an explanation or any sort of transparency. This would make them more reliable and would enable the client to assess the validity of the findings themselves, instead of trusting blindly. We decided to look into this further and were shocked to find that none of the companies investigated provided any access to validation studies. This means that for each Talent Match score provided by these companies the client is simply relying on the word of the company and nothing else.

So what makes Talent Data Labs and Nova different?

Here at Talent Data Labs, our tools incorporate all four predictive traits of performance in our Talent Match scores, as we believe that a complete picture should be painted in order to maximize the job and organizational fit of the hire.

Through Nova, we work with clients all over the world. Our products are designed with user experience in mind, and we automate as many features as possible in order to improve the convenience for our clients.

We offer full transparency for our Talent Match scores so that you’ll be able to understand why certain candidates are more compatible with your organization. For example, as part of our cultural matching report, we highlight how much a candidate matches or diverges from your company’s culture. Through an interview, you’ll be able to dig further to assess whether the identified diverging factors warrant concern.

We ensure that our talent matching products are backed by science. Our Cultural Matching score is carefully validated with employee self-reported match, satisfaction and commitment scores, which is supported by studies from Kristof & Brown (2005) and Schmidt and Hunter (1998).

How far have we come?

In conclusion, the state of talent analytics is evolving but may not be as rose-tinted as Bersin makes us believe. In ignoring valuable information, we may create a tendency to rely excessively on overly simplistic algorithms that help in making predictions but with that fail to truly satisfy our deep desires to eliminate biases and successfully mitigate risk.

  1. Due to the difficult and complex nature of the human mind, we find that it is hard to use the same practices to measure all predictive aspects of performance. Therefore, we focus a lot on metrics that are more widely available and easier to execute through massively available software. For example, we often turn to LinkedIn when we search for candidates with the desired skill sets.
  2. We tend to ignore a substantial number of indicators of performance and focus exceedingly narrowly on one or two traits that we deem to be important.
  3. If we truly did talent analytics right we would expect reports from all over the world of people now more accurately predicting tenure, performance, promotions, growth, and other outcome variables. Unfortunately, these findings are rare and usually rely on face validity, thus ignoring our desire to eliminate hiring bias.

While reading the reports of Bersin, we might believe that Talent Analytics has reached a certain state of maturity. However, through our research, we find that evidence to support such a statement is sorely missing. Luckily, there is hope! Thanks to all the new tools and great products that are being developed, the path to widely deployed improved predictions are quite clear.

We at Talent Data Labs and Nova aim to help all of you navigate these roads effectively. So if you found this analysis compelling, and want to work on a solution together, contact Nova!