AI can help reduce inequity in credit access, but banks will have to trade off fairness for accuracy – for now

• Traditional credit decisioning can result in bias, both intentional and unintentional.
• AI-based lenders say that their models can eliminate discrimination.
• But there are risks to these models, including the quality of the data we feed them.
• This article is part of a series called "The Cost of Inequity," examining the hurdles that marginalized and disenfranchised groups face across a range of sectors.

Unequal access to credit has long been an issue in the US. And while there are legal protections in place to fight discrimination in lending, instances of redlining and other less blatant bias have persisted, which have locked many out of the mainstream credit ecosystem.

Historically, loan applications were processed by humans, introducing the risk of bias, both intentional and unintentional.

As lending becomes increasingly digital, firms are looking to computers to help make credit decisions based on historical data. Beyond just analyzing data, some digital lenders employ artificial intelligence and machine-learning algorithms to make faster, more efficient credit decisions.

Many argue that AI-based lending can help solve for bias, both intentional and unintentional, in credit decisioning. By eliminating human bias, incorporating alternative data, and training the models in real-time, AI proponents say they're able to make more accurate predictions of a consumers' creditworthiness, regardless of factors like race and sex.

But some high-profile cases have ignited debates over the efficacy of so-called black-box algorithms.

In 2019, the web programmer and author David Heinemeier Hansson said on Twitter that when he applied for the Apple Card, he was offered 20 times the credit limit of his wife, even though they file taxes jointly.

The tweet went viral, and resulted in the New York Department of Financial services conducting an investigation into whether there was bias baked into Apple's credit-underwriting process, which is managed by Goldman Sachs.

The review found no evidence of intentional bias in Goldman's underwriting for the Apple Card. However, the DFS noted that "even when credit scoring is done in compliance with the law, it can reflect and perpetuate societal inequality."

While making credit decisions based on income, assets, and credit histories is legal, those factors "often reflect the nation's long history of racial and gender discrimination," the DFS said in its report.

While the algorithms behind the Apple Card didn't present any intentional biases, the underlying credit decisioning data is still a concern for regulators.

"The use of credit scoring in its current form and laws and regulations barring discrimination in lending are in need of strengthening and modernization to improve access to credit," the DFS said.

While AI-based lending has the potential to decrease discrimination in credit decisioning, companies need to consider how they train their models. If lenders don't scrub and analyze historical credit data for bias, their models could continue to make unequal credit decisions on the basis of race and sex.

"Racism is encoded in the data," Mike de Vere, the CEO of machine-learning fintech Zest AI, told Insider.

How credit scores can result in discrimination

Credit decisioning is complex, and increasingly involves several factors beyond an applicant's income and credit history. But credit scores are still front-and-center of the consumer-credit market.

FICO and Vantage scores range from 300 to 850. Typically, consumers with scores above 660 are considered prime or super-prime, and are able to access mainstream credit at average rates. For subprime and near-prime borrowers, with scores below 660, access to affordable credit is limited.

But credit-score averages aren't consistent across race. About 54% of Black consumers reported having no credit or credit scores below 640, according to a January survey by Credit Sesame. Meanwhile, 41% percent of Hispanic consumers, 37% of white consumers, and 18% of Asian consumers reported the same.

Those discrepancies can lead to different outcomes when traditional lenders heavily rely on credit scores.

To be sure, several major banks, including JPMorgan Chase and Wells Fargo, are reportedly planning to offer credit cards to people with no credit as part of a government-backed plan to improve access to credit.

But some AI-based lenders have found that their models, which often incorporate alternative data like education, income, and employment history, result in more approvals. A 2017 case study by the CFPB with AI lender Upstart found that its models increased credit access for many.

Upstart's model approved 27% more applicants than a baseline traditional model. Average APRs were also 16% lower, the analysis found. That expansion of credit access was consistent across variables including race and sex. And consumers with scores between 620 and 660 were approved twice as much by Upstart's model.

Proxy discrimination

One potential pitfall of using AI-based lending models is that they'll identify proxies for race in the same way financial institutions stopped serving predominantly Black neighborhoods, a process known as redlining.

Anya Prince, an associate law professor at the University of Iowa and one of the authors of a 2019 paper on proxy discrimination, told Insider AI techniques run the risk of "unintentional proxy discrimination" if it's not controlled for properly.

"If you say, 'Here's all this data, just don't use race. Inevitably, to find the best piece of data, in some situations, we'll find the next best proxy for race, because that's the most predictive thing,'" Prince said.

"You add AI into it, and all of a sudden we might have a situation where it's effectively proxy. We might not even be able to exactly see that connection if we don't go and look for it," she continued.

That said, there are ways to address the risk of disparate impact via proxy discrimination. Prince's paper, for example, lists four broad themes that could serve to counter the effect.

One approach is to expand the pool of relevant data used in AI decision-making. Another would be to flip "the default approach of anti-discrimination law," allowing models to only use preapproved variables when making decisions, rather than deciding which variables to exclude.

AI can help, but there are trade-offs

For lenders looking to deploy AI-based lending, they have two options: Scrub historical data or design the model from scratch.

Both options, however, have their downsides.

"The issue with correcting the data on the way in is you have to explain how you did that and why you did that," de Vere said. "That can be problematic for a lender."

Meanwhile, the second approach, focused on the model itself rather than the data, is costly.

"The tools that most financial institutions have are just woefully inadequate," de Vere said. And what's more, if a bank drops a variable from its model that could eliminate bias but that results in worse economics for the bank, they're likely to keep that variable.

Zest AI, which provides machine-learning models for lenders, has developed decisioning models that operate simultaneously. One might be a risk predictor, the other a race predictor. If the second model identifies a variable that might be a proxy for race, the risk model can tune down that variable to eliminate proxy bias.

Zest AI's clients include small community banks as well as large institutions like Freddie Mac and Discover.

"We have AI that's able to create fair, optional models for that financial institution, where they can understand the trade-off between the accuracy of the model, or the economics, as well as the fairness of the model," de Vere said.

To be sure, many lenders do see financial effects when adjusting for these proxy biases. For now, there's a trade-off between slightly less accurate models and slightly more inclusive ones.

But as these models become more fair and more common, the economics can improve, too.

"You could be so much more inclusive and still deliver better economics, but these organizations have got to be purposeful about it," de Vere said. "When I think about our customer base or people that we talk to in the industry, there is an interest in making that change."