What Should Every Recruiter Know About the HR AI They Deploy?

In the past year, the human resource technology market has seen growing activity in investment volumes and investor interest. Several recruiting AI startups have reportedly hit unicorn status while deploying AI-based technologies to drive next-generation recruiting. CB Insights reports over 250 different commercial AI-based HR tools out in the market as AI is being routinely used in HR processes from advertising job ads, collecting candidate data, screening, profiling and prioritizing candidates, to predicting worker performance. As ESG is gaining growing attention in board rooms, more recruiters turn to AI also for driving the enterprise diversity initiatives.

AI technologies carry a promise of making HR processes less impacted by human biases, faster and more consistent, and of higher quality – but the potential for amplifying past human biases and limiting people’s job opportunities worries a growing number of recruiters. The examples introduced by Amazon, Hirevue, and Uber remind us that when applied without proper consideration of the impacts, AI-driven automatization of HR can also lead to serious risks for job seekers, employees as well as the deploying brands.

So it is no surprise that the EU’s upcoming AI regulation considers recruitment and worker management as high-risk applications of AI along with AI applied in health, education, law enforcement, various other public services, and more.

The EU regulation introduces new quality management and transparency requirements for the providers and users of HR AI technologies with an aim to protect people of the potential adverse effects on fundamental rights such as non-discrimination. In New York, the new automated hiring law will require bias audits from any automated employment decision tool before being used by recruiters and demands candidates and employees to be notified about the use of such tools in the assessment or evaluation for hire or promotion. The US Equal Employment Opportunity Commission (EEOC) recently launched their AI initiative to ensure AI used in hiring and other employment decisions comply with existing federal civil rights laws.

Not only are regulators awake, but the risks are well recognized also by the industry. In the US, a group of America’s largest recruiters including CVS Health, Deloitte, General Motors, Humana, IBM, Mastercard, Meta, Nike and Walmart formed a Data & Trust Alliance, to adopt responsible data and AI practices. Positioning itself as a “do tank”, the group promises to develop and share practices that can be adopted across industries to facilitate faster learning. The group’s first focus, Algorithmic Bias Safeguards for Workforce, underlines the importance of employment-related AI risks for companies, and the urge to act.

According to a European enterprise survey, European enterprises use procurement as the most common way of sourcing AI-based systems. While 60% of enterprises report buying third-party software, only 20% develop AI fully in-house.

For HR the true figure is arguably much higher; the US alliance of major recruiters goes even further by saying that most of the algorithmic systems used to support workforce decisions are introduced and maintained by third-party vendors from software providers, professional networking sites, and consultants, to recruiting firms. Considering these sourcing strategies, in HR, responsible AI is first and foremost about responsible deployment and use of third-party technologies.

So HR professionals must start asking AI vendors questions that allow them to hold accountability for the technology they deploy. Informed by such transparency, recruiters can use AI-based products responsibly or, when needed, decide not to take into use products that conflict with the enterprise’s ethical principles. EU’s new AI regulation introduces obligations for AI technology providers to provide transparent instructions of use for the deployers of their technology and to keep them updated throughout the system lifecycle. Ultimately, it is about delivering what is needed for establishing accountability, and that is what we need for HR AI.

As the first step, recruiters should demand greater transparency from their HR AI technology providers.

Building on the upcoming compliance requirements and emerging industry practices, we compiled a template for systematic transparency exchange between the technology providers and recruiters. Here's a summary of the key questions:

• What are the intended purposes and use contexts of your product, and what is the business value created?
• What is the accuracy of your product, and how do you ensure its consistent performance over time?
• What data have you used to train, test and operate your product? How have the datasets been examined in view of potential biases?
• What potential risks might be caused by your product, and how do you mitigate the risks?
• What measures have you put in place to test, detect, mitigate and monitor potential biases across the product lifecycle from design to data selection, to model training, to deployment and monitoring?
• Which tools and education do you provide to ensure people overseeing your product can understand its capacities and limitations, monitor its operations, interpret the system output and when needed, intervene or interrupt the system?
• What changes have you done or plan to implement to your product and how do those changes impact the behavior of your product?
• What is the expected lifetime of your product and the required maintenance measures to ensure its proper functioning?
• What are the deployer responsibilities in ensuring your product performs as intended?
• Has the product been audited, by whom and how? Which actions have beentaken to address the recommendations raised?

HR AI technology providers must prepare to serve transparent information about their AI products to their customers throughout the product lifecycle. By doing so, AI technology providers not only demonstrate proactivity in addressing the ethical risks, such as AI biases, and regulative requirements but also establish a new feedback channel when comes to monitoring their system and its use cases on the market.

A better understanding of the use of the technology and potential ethical problems observed by customers are key in monitoring AI risks and driving product quality in an increasingly competitive AI marketplace.

While third-party AI risks become a new source of risk for brands, procurement establishes itself as the most critical process where the AI alignment with ethical principles and regulations is operationalized in HR. Saidot’s AI governance platform connects HR AI technology providers and recruiters around transparency. With our platform, HR AI technology providers can maintain their AI product transparency effortlessly and reach enterprises with systematic transparency data from one place in comparable format preferred by major recruiters.

‍