Review of Weapons of Math Destruction By Cathy O'Neil
June 22, 2021
Weapons of Math Destruction explains the dangers of outsourcing jobs to poorly trained algorithms that do not use feedback to improve themselves. The book explains that most of the issues occur when the application area is poorly understood and a company or government agency uses an algorithm in order to cut costs.
One of the most interesting examples of an algorithm impacting the real world involves the US News college rankings. This list ranks universities in the United States and the rest of the world. The rankings are based on a variety of measures ranging from alumni gifts to acceptance rate. The list has become very popular and many prospective college students rely on it during their college search. As a result, it has become important for universities to increase their standing in the US News ranking in order to attract students. The problem is that the factors considered by US News do not always coincide with educational outcomes. As a result, schools attempting to increase their ranking have to do so by accepting more legacy students, rather than trying to educate middle and low income students. This creates a system where universities prioritize the wrong areas while continually increasing their cost of attendance to offset their spending.
The US News university ranking list is considered a weapon of math destruction (WMD) because it is a poorly thought out mathematical formula that has negative real world impacts. This is the case because the parameters of the model are arbitrary and are not updated in response to problems. This creates a feedback loop that reinforces unhelpful behavior. Weapons of Math Destruction gives many similar examples on topics ranging from policing to credit scores.
Before reading Weapons of Math Destruction I did not know how pervasive flawed algorithms are. Reading the various case studies made me realize that those creating computational systems should always keep in mind the impact their work will have on others. Many of the worst consequences discussed in the book could have been avoided if those developing the algorithms had more foresight in their work. The impacts could have also been minimized if the creators of the algorithms would have constantly updated their systems, rather than leaving them static. Overall, Weapons of Math Destruction taught me how complex and destructive the effects of an engineering project can be.