Dissecting Racial Bias in an Algorithm that Guides Health Decisions for Millions

Опубликовано 7 июня 2019, 15:15

For millions of patients across the US, hospitals use commercial risk scores to target those needing extra help with complex health needs. We examine a widely used commercial algorithm for racial bias. Thanks to a unique dataset, we also study the algorithm’s construction, gaining a rare window into the mechanisms of bias. We find significant racial bias: at the same risk score, blacks are considerably sicker than whites. Removing bias would double the number of high-risk blacks auto-identified for extra help, from 17.7% to 46.5%. We isolate the problem to the algorithm’s objective function: it predicts costs, and since blacks incur lower costs than whites conditional on health, accurate cost predictions produce racially biased health predictions. We find suggestive evidence of a “problem formulation error”: as algorithmic prediction is in a nascent stage, convenient choices of proxy labels to predict (in this case, cost) can inadvertently produce biases at scale.

See more at microsoft.com/en-us/research/v...