Economist and data scientist Avi Goldfarb reached outside of his academic disciplines to collaborate with a legal scholar, exploring how machine learning could reveal the uncertainty inherent in refugee claim decisions—and improve outcomes for vulnerable populations. — *Economist and data scientist* ***Avi Goldfarb*** reached outside of his academic disciplines to collaborate with a legal scholar, exploring how machine learning could reveal the uncertainty inherent in refugee claim decisions—and improve outcomes for vulnerable populations.

Avi Goldfarb is an economist and data scientist specializing in marketing. So how is it that he come to publish a paper on reducing false denials of refugee claims through artificial intelligence?

Goldfarb, a professor at the Rotman School of Management at the University of Toronto and a Schwartz Reisman faculty affiliate, read Refugee Law's Fact-Finding Crisis: Truth, Risk, and the Wrong Mistake (2019), a book by Hilary Evans Cameron, an assistant professor at Ryerson University’s Faculty of Law.

He found some remarkable overlaps with his own work, and particularly the methodology he employs in his 2018 book, Prediction Machines: The Simple Economics of Artificial Intelligence.

It just so happened that Evans Cameron had read Goldfarb’s book too.

Refugee law expert Hilary Evans Cameron is an assistant professor at Ryerson University’s Faculty of Law. — *Refugee law expert* *Hilary Evans Cameron* *is an assistant professor at Ryerson University’s Faculty of Law.*

“It turned out we effectively had the same classic decision theoretic framework,” says Goldfarb, “although hers applied to refugee law and problems with fact-finding in the Canadian refugee system, and mine applied to implementing AI in business.”

Decision theory is a methodology often used in economics and some corners of philosophy—in particular, the branch of philosophy known as formal epistemology. Its concern is figuring out how and why an “agent” (usually a person) evaluates and makes certain choices.

The main idea around which Evans Cameron’s and Goldfarb’s thoughts coalesced was this: human decision-makers who approve or deny refugee claims “are often unjustifiably certain in their beliefs,” said Goldfarb in his research presentation at the Schwartz Reisman weekly seminar on October 7, 2020.

In other words: people who make decisions about claimants seeking refugee status are more confident about the accuracy of their decisions than they should potentially be.

Why?

Because “refugee claims are inherently uncertain,” says Goldfarb. “If you’re a decision-maker in a refugee case, you have no real way of knowing whether your decision was the right one.” If a refugee claim is denied and the refugee is sent back to their home country where they may face persecution, there is often no monitoring or recording of that information.

Goldfarb was particularly struck by the opening lines of Evans Cameron’s book: “Which mistake is worse?” That is, denying a legitimate refugee claim or approving an unjustified one?

The answer is clear: sending a legitimate refugee back to their home country is a much greater harm than granting refugee status to someone who may not be eligible for it. This is what Goldfarb refers to as “the wrong mistake.”

So, from Goldfarb’s perspective as an economist and data scientist with specialization in the unique powers of AI and machine learning (ML), he started to wonder: could ML’s well-known ability to reduce uncertainty help reduce incidences of “the wrong mistake”?

Goldfarb’s and Evans Cameron’s collaboration reflects the intersections between the four “conversations” that guide the Schwartz Reisman Institute’s mission and vision. Their work asks not only how information is generated, but also who it benefits, and to what extent it aligns—or fails to align—with human norms and values.

“ML has the ability to make uncertainty visible,” says Goldfarb. “Human refugee claim adjudicators may think they know the right answer, but if you can communicate the level of uncertainty [to them], it might reduce their overconfidence.”

Here, Goldfarb is careful to note that shedding light on “the wrong mistake” is only part of the battle. “Using AI to reduce confidence would only work in the way described if accompanied by the changes to the law and legal reasoning that Evans Cameron recommends,” he says.

“When uncertainty is large,” says Goldfarb, “that does not excuse you from being callous or making a decision at all. Uncertainty should help you make a better-informed decision by helping you recognize that all sorts of things could happen as a result.”

So, what can AI do to help people realize the vast and varied consequences of their decisions—reducing their overconfidence and helping them make better decisions?

“AI prediction technology already provides decision support in all sorts of applications, from health to entertainment,” says Goldfarb. But he’s careful to outline AI’s very real limitations: it can often introduce and perpetuate bias, it lacks transparency, and so on.

What Goldfarb and Evans Cameron advocate for is a kind of assistive role played by AI, one in which the necessary statistical predictions of evaluating refugee claim decisions could be done better.

“Fundamentally this is a point about data science and stats. Yes, we’re talking about AI, but really the point is that statistical prediction tools can give us the ability to recognize uncertainty, reduce human overconfidence, and increase protection of vulnerable populations.”

So, how would AI work in this context? Goldfarb is careful to specify that this doesn’t mean an individual decision-maker would immediately be informed whether they made a poor decision, and given the chance to reverse it. That level of precision and individual-level insight is not possible. While we may not solve “the wrong mistake” overnight, AI could at least help us understand what shortfalls and data gaps we’re working with.

Of course, many challenges to implementing Goldfarb’s and Evans Cameron’s ideas remain. Implementation would involve designing an effective user interface, changing legal infrastructure to conform with the information these new tools produce, ensuring accurate data-gathering and processing, and firing up the political mechanisms necessary for incorporating these processes into existing refugee claim assessment frameworks.

While we may be far from implementing AI to reduce incidences of “the wrong mistake” in refugee claim decisions, Goldfarb highlights the interdisciplinary collaboration with Evans Cameron as an especially promising start to exploring what the future could bring.

“It was really a fun process to work with someone in another field,” he says, “something which the Schwartz Reisman Institute is really working hard to facilitate between academic disciplines, and which will be crucial for solving the kinds of complex and tough problems we face in today’s world.”