WHY IS THIS IMPORTANT? WHY DOES IT MATTER IF WE LOOK INSIDE THE BLACK BOX?
You may be asking yourself, why does it matter how these things work? Why does what's inside of them matter? Do I really need to know how my TV works in order to watch it? Does it really matter what's inside my smartphone or who programmed TikTok, Instagram, or Twitter as long as I can access them?
Many of us have used public restrooms. Many of us have used public restrooms with automatic sensors on the sinks, the ones where you put your hands in front of the light sensors and it activates the water to wash your hands. My partner and I have two completely different experiences when we encounter these, and it is no accident. I also have different experiences depending on the time of year. As a Mexican woman, if it is the middle of summer, and my skin is much darker, it is almost impossible for me to activate one. I ALWAYS have to use the palm of my hands. My partner, who is white, has little difficulty activating the sensors.
I know what you're thinking, there's no way that the sink I'm trying to use has any sort of racial bias. This sink can't possibly be racist, it's in inanimate object, a harmless piece of technology, dare I say, neutral. But is it?
“Whites Only?” Post by TeejMaximus September 02, 2015. Retreived from on YouTube December 02, 2020.
In 2015, this idea made headlines when a viral video made its rounds on social media after two friends at DragonCon were in a Marriott hotel bathroom (Plenke). One white, who had no problem making the light sensor work and triggering the faucet, and the other, a black man, who could not get the sensor to activate. The video was titled "Whites Only," a throw-back to Jim Crow segregation, and raised the question of whether or not we need a closer look at who is behind, and what is inside our technology.
Biometrics: These are physical characteristics like fingerprints, facial recognition, voice, and eye or retinal recognition that can be used to identify who you are. They can be used to unlock your phone, to access bank accounts, open doors, even identify you for crimes (Gillis).
This may seem frivolous, it's just a soap dispenser, but it becomes even more important when these technologies are unable to recognize people. No longer a thing of the future, every major city in the United States and countries all over the world use biometrics; from tracking citizens on city streets to using it in law enforcement.
However, these technologies have been shown, time and again, to not work, and they have high inaccuracies for people of color and women. This technology has been massively rolled out world wide in countries like the United States, the United Kingdom, and China.
Facial Recognition
Facial recognition is being widely rolled out in major cities across the world with little oversight. Many studies have shown that facial recognition technology is not only biased, but that in many circumstances it inaccurately identifies people of color, especially women (Al Jazeera, Rae).
AlJazeeraEnglish, & Rae, A. (2019, July 03). Do Biometrics Protect or Compromise Our Security? | All Hail the Algorithm. Retrieved from YouTube on December 02, 2020.
Stephanie Hare: Ethics of Biometrics & Vulnerable Populations
AlJazeeraEnglish, & Rae, A. (2019, July 03). Do Biometrics Protect or Compromise Our Security? | All Hail the Algorithm. Retrieved from YouTube on December 02, 2020.
Biometric Technology & Vulnerable Populations
In areas where people have minimal rights, like refugee camps, biometric technology is being used to collect data. How should this technology be used in these spaces? Stephanie Hare discusses whether or not it is ethical to use it with groups of people who have limited rights, or are not in a position to refuse the use of this technology and collection of such intimate data.
HOW IS THIS POSSIBLE?
HOW DOES TECHNOLOGY SHOW RACIAL BIAS?
These are just a few examples of the more obvious racial bias that are present in technology. We can begin to see how these technologies directly impact how we behave (or perform) in the real world, but who or what is dictating our behavior from inside the black box?
How did this racial bias get there, or maybe you already know, or have some idea. When thinking about racial bias in technology we need to think about who is making the tech.
WHO MAKES THE TECHNOLOGY?
The tech industry is predominantly male and white. This isn't to say that the technology is intentionally made not to recognize darker skin, this is to say that the people writing the code, the people making the product, testing the product, and doing quality control are predominantly white men. There aren't people of color to step in, to find racial blind spots, so racial bias gets programmed into the system.
When a technology like facial recognition is being tested, if it is only being trained on white, male faces, a bias gets programmed (intentionally or not) into the system.
Representation matters, from coding to quality control. Representation matters every step of the way, especially when that technology is helping to put people behind bars.
Bias in the Justice System
More and more, algorithms are being used in the criminal justice system, from sentencing to parole boards. It has been assumed that technology would help remove the bias from the system, to help bring a sense of neutrality, but is that what has happened? Bias in the criminal justice system is already apparent. However, with the implementation of algorithms to help the system, the technology that was intended to help the system has its own bias. These technologies are also created for efficiency and to expedite processes, this means issues like bias also become more efficient.
Another important question to consider, especially when it comes to the Justice System is: Does not being able to question the algorithm or look inside the black box violate your right to Due Process? If we can’t see the algorithm because the company who owns it doesn’t have to show it to us, how do we know we’re getting a fair trial by our peers? How do we know if our constitutional rights aren’t being violated?
Julia Angwin: Disparity in Error Rates
VPROinternational, & Kieft, M. (2018, October 26). Algorithms Rule Us All | VPRO documentary - 2018. Retrieved from YouTube on December 02, 2020.
COMPAS: Violation of Due Process?
AlJazeeraEnglish, & Rae, A. (2019, July 03). Can We Trust Algorithms? | All Hail The Algorithm. Retrieved from YouTube on December 02, 2020.
Glen Rodriguez: Lack of Transparency
VPROinternational, & Kieft, M. (2018, October 26). Algorithms Rule Us All | VPRO documentary - 2018. Retrieved from YouTube on December 02, 2020.
Digital Poorhouse
Virginia Eubanks, author of “Automating Technology: How High-Tech Tools Profile, Police and Punish the Poor,” speaks on how our new automated systems are modern Digital Poorhouses and the need to understand America’s history with race, and poverty to see how automated systems makes services difficult to access for those in need.
In the second clip, she shares the how politics, social stigma and technology converge create these modern-day conditions.
AlJazeeraEnglish, & Rae, A. (2019, July 03). Can We Trust Algorithms? | All Hail The Algorithm. Retrieved from YouTube on December 02, 2020.
CUNY TV75. (2018, January 21). Automating Inequality - Virginia Eubanks | The Open Mind [Video]. YouTube. https://www.youtube.com/watch?v=Avxm7JYjk8M
DISPARITY INCREASES
As previously mentioned, because systems are made to be more efficient, systems that already have some disparity only become more efficient at disparity and discrimination. This can happen because discrimination is seen as a problem with the individual and not as systemic and structural (as explained in the left clip below). Unfortunately, the more efficient the system the easier it is for people to no longer be seen as human, but as commodities, as author & data scientist Cathy O’Neill explains (right clip below).
Virginia Eubanks: Discrimination built into the Algorithm
AlJazeeraEnglish, & Rae, A. (2019, July 03). Can We Trust Algorithms? | All Hail The Algorithm. Retrieved from YouTube December 02, 2020.
Cathy O’Neill: Algorithms Cause the Future
VPROinternational, & Kieft, M. (2018, October 26). Algorithms Rule Us All - VPRO documentary - 2018. Retrieved December 02, 2020,