Stay ahead of crisis: How to *really* use AI to Predict Employee Turnover [+ 3 tips from experts]

Machine learning is transforming the way companies are anticipating employee turnover and thus managing retention and engagement. We will explore this transformation along with various insights from experts in the field who will help us to understand the benefits, challenges, and best practices that you should consider when using AI to forecast employee turnover and drive engagement.

More than just employee turnover

It is almost impossible to talk about turnover without talking about employee retention and engagement. For instance, even if your employee turnover rate (the people that leave your organization) is different from your employee retention rate (the people that stay in your organization), these two variables represent the opposite sides of the same coin.

Likewise, just like there is a direct correlation between poor employee engagement and high turnover, there is also a direct correlation between solid engagement and high retention rates.

We invite you to keep this association in mind because when companies use AI to predict employee turnover (in order to find out who is leaving and why), they are ultimately trying to improve their employee retention and engagement strategies.

How can AI be used to predict employee turnover

Foreseeing employee turnover is far from new. According to Ian Cook, Vice President, People Analytics at Visier, “predicting turnover using AI has been in practice for close to a decade and has settled to some core models, specifically machine learning.” AI-driven methods and techniques offer several advantages over traditional ones. Let’s see some of them and the kinds of solutions that companies are starting to adopt nowadays.

What are the benefits of using AI to manage employee turnover

The benefits of using AI for anticipating employee turnover are enormous especially when you look at the capabilities that AI-driven technology offers when compared to traditional methods.

“Machine learning algorithms can analyze large datasets, identify patterns, and make predictions with greater accuracy than traditional statistical methods,” argue UNSW Business School’s Andrew Dhaenens, Mary-Anne Williams and Karin Sanders. The following are some of the features you need to keep in mind.

Large and complex data

AI methods can efficiently process and analyze large volumes of data while handling complex relationships and patterns that may be challenging for traditional methods to uncover.

Higher accuracy

AI-driven technology leverages advanced algorithms (e.g. decision tree, random forest, logistic regression), machine learning, and predictive analytics to provide more accurate predictions.

Real-time analysis

Machine learning processes can analyze data in real-time, allowing companies to monitor ongoing trends and take proactive measures.

Deeper insights

AI models can identify complex patterns and relationships within the data. They can also uncover non-linear correlations and interactions between variables, providing deeper insights into turnover risks.

Higher customization

By leveraging individual employee data, AI models can provide tailored insights and retention strategies, enhancing the effectiveness of interventions.

Scalability and efficiency

AI methods can scale effortlessly to handle large organizations or datasets, reducing manual efforts and increasing efficiency. They can process and analyze vast amounts of data within a shorter time frame compared to traditional methods.

Ongoing learning and improvement

AI models can continuously learn and adapt as they receive new data. They can refine their predictions and accuracy over time, incorporating the latest information and trends in turnover patterns.

As you can see, AI-driven people analytics can leverage employee turnover prediction in a powerful way. It is important to remember, however, that the real benefits that AI offers, as a tool for anticipating employee attrition, lie beyond all the capabilities we just mentioned.

“Success using people analytics tools isn’t measured by attaching a percentage rate to the number of people identified as being at risk for leaving the company. Instead, success occurs when the technology identifies workers who are likely to leave and action is taken that keeps the employee, which in turn saves the company the money it spent developing that worker’s skills and experience,” explains Cook.

In other words, success occurs when companies are able to implement actions that allow them to improve their overall talent management. When that occurs, organizations are able to enjoy some of the following benefits:

• Boost employee engagement
• Increase employee retention
• Save financial resources
• Maintain productivity levels
• Enhance decision making processes
• Reduce employee turnover
• Optimize strategic planning
• Retain top talent

AI methods for foreseeing employee attrition

The following are some of the most popular AI methods and techniques that companies are using today to anticipate employee turnover:

• Machine learning models using historical employee data.
• Natural Language Processing (NLP) that allows companies to analyze employee feedback and sentiment.
• Predictive analytics algorithms that can be used to identify patterns and trends.
• Data mining techniques that reveal hidden variables influencing employee turnover.
• Social network analysis that helps companies to identify influential employees and their potential impact on turnover.
• Sentiment analysis of employee communication channels such as emails and chat logs that help companies to measure job satisfaction among employees.
• Deep learning models for anticipating employee attrition.
• Chatbots that allow companies to check the pulse of employee sentiment at work.
• Explainable artificial intelligence (XAI) techniques (e.g. LIME, PDP) that help companies to understand the outputs from complex employee data.
• Recommendation systems to provide targeted retention strategies for high-risk employees.
• Predictive modeling of employee behavior using real-time data such as work patterns, social media activities, and collaboration tools (e.g. Asana, Slack activity).

The predictive employee turnover market

Even if AI models offer big advantages over traditional models, the incorporation of AI technology into workforce management roles is still in the early stages. According to “The Use of People Analytics in Human Resources” report elaborated by SHRM, “only 9% of HR professionals whose organizations use people analytics use an AI-driven form.”

This is, of course, only a temporary issue. Considering the level of accuracy and performance that AI can bring into the field of people analytics, the adoption of this kind of technology in regards to most workforce management functions is just a matter of time.

In the following image, we can see the different levels of the people analytics maturity model, the percentage of companies at each level, and some of the potential uses that AI can bring into each level:

In spite of being in its early stages, the application of machine learning technology to employee turnover analysis has enabled the development of a whole range of predictive attrition companies. “Companies such as IBM and Ultimate Software Group have paved the way for a new crop of “predictive attrition” companies such as Retrain AI, Eightfold AI, and HR Signal—that is, companies that inform employers when an employee is at risk of quitting.”

According to Andrew Spott, cofounder and president of HR Signal, the predictive attrition field is currently divided into the following four categories:

1. Predictive analytics companies that use public big data to draw conclusions.
2. Analysts that interpret internal data provided by employers.
3. Surveyors that ask employees directly if they are considering quitting.
4. Surveillance-focused organizations that employ spyware (such as keylogging and mouse movement tracking).

Similarly, the market of AI software and third-party solutions that companies use to improve their overall talent management strategies is growing rapidly. Some of the companies that are already providing these kinds of solutions include IBM (Watson Studio), Microsoft Viva Glint, Visier, RetainTalent, and Erudit, among many more.

Forecasting employee turnover (with or without AI) comes with five challenges

Considering the ongoing attrition in the workplace and the level of performance that AI brings into the HR function of employee turnover, many companies are starting to think about implementing some sort of AI-driven people analytics system into their organizations.

“Recently, we have seen many of our clients express significant interest in developing predictive models to anticipate and address employee attrition and look for ways to stay ahead of their competitors through more effective talent retention,” explain Deloitee executives Eric Lesser and Gary Parilis.

This interest is not only the result of the ongoing attrition across all industries but also of the ongoing frustration and the poor results that companies are getting with their current data analyses.

In fact, many companies gather extensive data using all kinds of means (e.g. ENPS surveys, tracking reports, chat tools for checking employee sentiment) without being able to bring together all this information in a way that can generate useful insights for the organization.

In this context, AI-driven people analytics seems to be the best thing since sliced bread for anticipating employee attrition. However, machine learning isn’t magic and there are several challenges you need to consider when implementing AI models across your organization.

Challenge 1: Quality data and literacy

As we mentioned before, there are lots of organizations that can’t get anything valuable from the data they collect. This is mainly because of the following two reasons:

1. They tend to rely on surveys and data related to demographics and compensation that don’t provide any value to the organization.
2. They don’t have the necessary literacy to collect, analyze and interpret the data.

Sometimes this is also the result of HR Information Systems (HRIS) that are often overlooked. According to Rohit Punnoose, Senior Director, Data Sciences at Target India, and Pankaj Ajit, Data Science and Engineering Manager at Shopify, these HRIS are “typically under-funded compared to the Information Systems of other domains in the organization which are directly related to its priorities. This leads to the prevalence of noise in the data that renders predictive models prone to overfitting and hence inaccurate.”

This is particularly critical when you don’t have well structured, clean and ethical data that covers a long period of time. According to Ian Cook, “core HR systems hold this information but not in the shape or quality that allows for an AI prediction to be generated.”

There are, indeed, lots of companies that are already investing in machine learning models but are finding it difficult to consolidate their data. “We often see these organizations encountering challenges in designing and assembling these models and using them to act within their organization,” explain Lesser and Parilis.

This is also because they don’t have the necessary literacy to interpret the data. “If managers do not understand how and why responses are generated by AI models based on the input datasets, it is unlikely to augment data-driven decision-making and bring value to the organizations,” argues a paper published by Taylor & Francis Online.

Challenge 2: The complexity of human behavior

Human behavior is complex and algorithms may have problems identifying some unique behaviors. Furthermore, they can even flag people for doing something that has nothing to do with a serious intention to leave the company.

“What if an employee is only casually considering quitting, but after getting flagged by some form of AI as a quit risk, is now hounded by skeptical, or even angry, managers? What if these pressures cause a worker, who ultimately would have stayed if they had been left alone, to quit?,” asks Aj Hess, staff editor at Fast Company.

Challenge 3: Lack of trust in AI

According to an HBR article written by Jessica Kim-Schmid, Manager of MBA Programs at Moderna, and Roshni Raveendhran, Assistant Professor of Business Administration at the University of Virginia Darden School of Business, people have a very hard time trusting and accepting AI-driven decisions.

“People often mistrust AI because they don’t understand how AI works, it takes decision control out of their hands, and they perceive algorithmic decisions as impersonal and reductionistic,” explain Kim-Schmid and Roshni Raveendhran. This is, of course, a big challenge that many companies need to face when using AI models for foreseeing employee attrition.

Challenge 4: Privacy issues

Privacy concerns are one of the biggest challenges that AI poses across all industries and functions. In fact, AI models can uncover insights about employees that were not initially intended for prediction, leading to potential breaches of privacy.

These concerns are even bigger when third-party companies are involved in the process. For instance, AI models often require large amounts of data for training, and when third-party companies handle this data, there’s a risk of data breaches, unauthorized access, or misuse of personal information.

This is particularly true for companies that are incorporating social media data into their predictive analyses. AI models can analyze social media posts and interactions to infer personal information, such as an employee’s political views, religious beliefs, or lifestyle choices. In this context, employees may feel that their privacy is violated when their personal preferences are used to predict turnover risk.

Challenge 5: Bias and ethics

This is one of the most important challenges you need to face, especially if you are relying on historical data that has been subject to bias. As pointed out by Kim-Schmid and Raveendhran, AI models aren’t entirely bias-free because they are typically trained using existing datasets, which may reflect historical biases.

In fact, machine learning models can incorporate bias in many different ways. For example, ​​if the model uses proxy variables that are correlated with sensitive attributes (e.g. age, gender, race) to predict turnover, it can indirectly perpetuate bias by making predictions based on these protected attributes.

Three tips from experts on how to embrace AI for anticipating employee attrition

We had the opportunity to talk about this topic with various experts including Ian Cook from Visier, Irma Doze from AnalitiQs, and Professor David Allen from TCU.

We asked them to give us a piece of advice for managers and business professionals who may be considering AI for foreseeing employee attrition. The following are the three main points from the feedback we gathered.

1. Start answering basic questions before bringing AI into the process

When we asked Irma Doze, Managing Director at AnalitiQs, about using AI for foreseeing employee turnover, her immediate reply was far from what we expected to hear: “it depends on what you mean by AI”. This short answer led to something deeper:

The first question you need to answer is this: do we have a problem?

Irma Doze

According to Doze, your initial analyses need to target the following basic questions one by one:

• What is the actual turnover?
• Do we have a problem?
• Why is it a problem?
• Is turnover everywhere?
• What if we do nothing? Is it going to be worse or is it going to get better?

Answering these questions implies carrying out analyses, which don’t necessarily need to incorporate any AI-driven technology. “I wouldn’t call those analyses AI yet, for me that’s more like descriptive and diagnostic analytics,” argues Doze.

2. Bring expertise and get a solution that fits your organization

Once you know that there is a problem, you need to solve it. How? The experts we interviewed gave us a mix of answers here. For Irma Doze, the solution should be a step by step process that only later brings AI into play. “It always starts with a brainstorm,” argues Doze. In the following image, we can see her rationale for dealing with employee turnover.

Regardless of your selected approach to the problem, you need to bring qualified data analysts on board if you want to do it right. “HR professionals are not data scientists and for these kinds of analyses, you really need data scientists,” argues Doze.

If you want to achieve this, “you could recruit a data analyst or hire an external data scientist. If you are small, you don’t need a full time consultant,” states Doze. Furthermore, recruiting this kind of talent is definitely something to consider. “Many firms are of course hiring data analysts, which ultimately is probably the most sustainable solution if the firm is large enough to absorb it,” explains Professor Allen.

Of course, that’s not the only option available. “I would hesitate to recommend anything specific, but there are firms large and small offering custom technology-enabled solutions. I would recommend doing significant due diligence about scientific background, track record, all the things one would normally do hiring a consultant. There are independent contractors that could do it much less expensively (e.g., university professors with the right training),” suggests Allen.

As far as the tools you need to get the job done, Irma Doze thinks that it depends on the size of your company, the available budget, the potential resources you already have, and the specific circumstances and characteristics of your organization. Of course, there are some solutions that have already implemented algorithms and you can use them. However, designing your own company specific algorithm can be much more effective and once you have built the algorithm you can implement it into your existing HR system.

Ian Cook, on the other hand, firmly believes that if your company is having a turnover problem and is considering an AI solution to tackle it, “the best approach is to look for a specialist people analytics application to solve this issue.”

According to Cook, this kind of application will be less expensive and risky than hiring people to do the work. “Modern technologies solve for both the insight and the action, which is hard, slow and risky for a person to achieve on their own with simple BI tools,” argues Cook.

Furthermore, anticipating employee attrition is a complex process and taking care on your own of all the most important aspects associated with it, could be quite challenging. “The complexities related to data preparation, legal frameworks, testing, validation and maintenance make it almost impossible for a company to manage all of these processes on their own.” explains Cook.

Apart from all of the above reasons, Cook also believes that the most valuable element that AI brings into the process comes from being able to predict future turnover, rather than just understand existing turnover. “Predictions are most effective as you need to stop turnover before it happens. This is why it makes sense to use AI capabilities, such as machine learning, to develop a predictive model,” explains Cook.

3. Empower managers with guidance and accountability

The success of any strategy dealing with foreseeing employee attrition depends on the actions you take to either keep the employee at the company or put in motion a strategic plan for a smooth replacement. As stated by Ian Cook, “predicting that an employee will leave is simply a curiosity unless that data gets into the hands of a manager or leader who can take some action.”

Because of the above, it is important that you provide guidance to managers in terms of coaching and strategic planning. For example, if managers know that Robert, a solid performer in a critical role, is at higher risk of leaving the company, it is crucial that they understand the kinds of actions they can take.

According to Professor Allen, “it would be extremely valuable to know that he [Robert] is at higher risk of leaving so strategies could be developed, e.g., retention strategies targeted at Robert, redesigning work so the departure isn’t so disruptive, or building a network of possible replacements.”

Companies need to provide guidance to managers so they know what kinds of actions they can implement with the information they get from their predictions. “Going back to Robert, what should the manager do? Offer money? Ignore it? Be nicer? Talk about career paths? Have a conversation about goals? I think firms need to be much more thoughtful and perhaps get some assistance in figuring out how to best use the information the technology provides,” argues Allen.

Along those lines, Ian Cook says that it is vital that organizations think through the application and action that will come from the analysis before they start to work with AI and its predictive capabilities. “Without this, the work can become an interesting, technical project that delivers little to no value to the business,” warns Cook.

This particular aspect can benefit a lot from Generative AI. According to Cook, “the introduction of Generative AI increases the ability for solutions to provide more specific guidance to managers on how they might respond to employees who are at risk of leaving.”

Providing guidance to managers is essential. However, if you really want to reinforce their role into the whole process, you should also think about accountability. As suggested by Professor Allen, companies should task managers with being accountable for employee retention.

Final thoughts on using AI to predict employee turnover and drive engagement

There’s little doubt about the potential benefits that AI-driven people analytics can bring into your employee turnover analysis. In spite of that, implementing AI models across your organization is a process that should start with a brainstorming where you need to think about two things:

1. The nature of your problem (if you have one).
2. The actions you will take to tackle the problem.

As far as the nature of your problem goes, you should be careful about the data you use. While the quality of it is crucial, you should always keep in mind the complexity of human behavior as well as issues of trust, privacy, and bias.

In terms of the actions that you will implement to tackle the problem, you need to bring expertise to the process and find a solution that fits your business. At the same time, you need to empower your managers with guidance and accountability so they can make the right decisions with the insights they get.

The whole point of anticipating employee attrition relies on the fact that you should use your employee turnover analyses as a tool aimed at leveraging your talent management strategy whether that means keeping the employee through engagement and retention initiatives or replacing him/her in a way that doesn’t disrupt the productivity of your business.

AI-driven people analytics can act as a great assistant to your organization. However, at the end of the day, the conversations you have with your employees and the strategies you implement to engage and retain your top performers will define the success of your business.

Comments Cancel reply

Δ