Gartner Predicts 75% of Large Organizations Will Hire AI Behavior Forensic Experts to Reduce Brand and Reputation Risk by 2023

Users’ trust in artificial intelligence (AI) and machine learning (ML) solutions is plummeting as incidents of irresponsible privacy breaches and data misuse keeps occurring. Despite rising regulatory scrutiny to combat these breaches, Gartner Inc. predicts that, by 2023, 75% of large organizations will hire AI behavior forensic, privacy, and customer trust specialists to reduce brand and reputation risk.

Bias based on race, gender, age, location, and bias based on a specific data structure has long-standing risks in training AI models. Besides, opaque algorithms such as deep learning can incorporate many implicit, highly variable interactions into their predictions that can be difficult to interpret.
“New tools and skills are needed to help organizations identify these and other potential sources of bias, build more trust in using AI models, and reduce corporate brand and reputation risk,” said Jim Hare, research vice president at Gartner. “More and more data and analytics leaders and chief data officers (CDOs) are hiring ML forensic and ethics investigators.”

Increasingly, sectors like finance and technology are deploying combinations of AI governance and risk management tools and techniques to manage reputation and security risks. Besides, organizations such as Facebook, Google, Bank of America, MassMutual, and NASA are hiring or have already appointed AI behavior forensic specialists who primarily focus on uncovering undesired AI models’ bias before they are deployed.

These specialists validate models during the development phase and continue to monitor them once they are released into production, as unexpected bias can be introduced because of the divergence between training and real-world data.

“While the number of organizations hiring ML forensic and ethics investigators remains small today, that number will accelerate in the next five years,” added Mr. Hare.

On the one hand, consulting service providers will launch new services to audit and certify that the ML models are explainable and meet specific standards before models are moved into production. On the other, open-source and commercial tools specifically designed to help ML investigators identify and reduce bias are emerging.

Some organizations have launched dedicated AI explainability tools to help their customers identify and fix AI algorithms’ bias. Commercial AI and ML platform vendors are adding capabilities to generate model explanations in natural language automatically. Open-source technologies such as Local Interpretable Model-Agnostic Explanations (LIME) can look for unintended discrimination before it gets baked into models.

These and other tools can help ML investigators examine the “data influence” of sensitive variables — such as age, gender, or race — on other variables in a model. “They can measure how much of a correlation the variables have with each other to see whether they are skewing the model and its outcomes,” said Mr. Hare.

Data and analytics leaders and CDOs are not immune to a lack of governance and AI missteps. “They must make ethics and governance part of AI initiatives and build a culture of responsible use, trust, and transparency. Promoting diversity in AI teams, data, and algorithms, and promoting people skills is a great start,” said Mr. Hare. “Data and analytics leaders must also establish accountability for determining and implementing the levels of trust and transparency of data, algorithms, and output for each use case. They must include an assessment of AI explainability features when assessing analytics, business intelligence, data science, and ML platforms.”

Gartner clients can read more in the reports: “Predicts 2019: Digital Ethics, Policy, and Governance Are Key to Success With Artificial Intelligence” and “Top 10 Data and Analytics Technology Trends That Will Change Your Business.”Note to Editors*
“Predicts 2019: Leadership Means Expanding Options, Not Limiting Them — A Gartner Trend Insight Report” is a collection of research aimed at helping CIOs and IT leaders focus on how the landscape is shifting for individuals, businesses, and IT organizations.

Gartner Data & Analytics SummitGartner analysts will provide additional analysis on data and analytics trends at the Gartner Data & Analytics Summit 2019, taking place June 10-11 in Mumbai. Follow news and updates from the events on Twitter using #GartnerDA.