“The bigger philosophical question is who decides what things mean? Who is 100% female? Is it Barbie or is it Grace Jones?” —Trevor Paglen in "A Space Oddity," 1843 Magazine

Sight Machine

“I really don’t think art is good at answering questions — it’s much better at posing questions, and even better at simply asking people to open their eyes,” —Trevor Paglen in "A String Quartet Concert, With an A.I. Assist," The New York Times

On January 14, 2017, a crowd gathered in one of San Francisco's pier warehouses to listen to the Kronos Quartet. There were four of them on stage, but one audience member stole the show.

Trevor Paglen's Sight Machine, a collaboration with the musicians and AI programmers, offered people a glimpse of how AI machines see the world. Above the quartet, a large screen displayed what computers trained with computer vision and AI algorithms saw as they watched the performance. Paglen explains in the short video below.

Categories to See With

"Neural networks cannot invent their own classes; they're only able to relate images they ingest to images that they've been trained on." —Trevor Paglen, “Invisible Images (Your Pictures Are Looking at You)" in New Inquiry

AI computer-vision systems learn how to see category by category, name by name. For example, say you wanted to train an AI to recognize flowers. To do so, you would provide your AI with a training set of millions of flower images that would allow it to incrementally build its understanding of what makes flowers flowers (voila, we have the AI version of Goethe's Urpflanze, some sort of archetypal flower/plant). An AI is considered to be successful at learning how to see flowers when it consistently and accurately identifies the flowers within an entirely new set of mixed images.

Categories to Think With

In a way, the language available to us to describe how AI see, understand, or even think is still woefully limited—at best, the words serve as metaphors or in terms as they apply to us. But part of developing our own understanding of AI as it grows more pervasive in our world is the need to make these machines visible, at least in our own imagination. Paglen's work seeks to do just that.

If we return to Paglen's Sight Machine performance with the Kronos Quartet and his team of AI programmers, we are presented with the dilemma of there being an Ur-anything. In one moment of the concert, the cellist Sunny Yang is deemed "59.65% female." Tempting as it may be to applaud the machine for its non-binary determination of gender, this mid-way designation suggests that there is some image of what it means to be 100% female.

Bias, Not Bugs

"If AIs can be trained to recognize and respond to the world in any way its developers choose, we should be concerned about who is doing the training and where its gaze is being turned." —from "A Space Oddity," 1843 Magazine

One of the concerns about developing AI's capabilities regards how we might be introducing our own biases into the training of these machines. When we admire the truly astonishing capabilities of AI it can be easy to see these machines as merely objective automatons. What we may not consider is that the categories "to see with" are derived from the categories humans use "to think with," biases and all.

Let us return to the question of gender designation. According to one article on the efforts of researchers to consider the ethical consequences of AI development, "even when machine-learning systems are programmed to be blind to race or gender, for example, they may use other signals in data such as the location of a person's home as a proxy for it."

A separate article provides examples of these biases. Computer science professor Vicente Ordóñez's and his team found that AI machines tended to associate people cooking in the kitchen as being women, thereby misnaming men in images due to the AI's association with females in the kitchen. In a separate research project by Boston University and Microsoft, researchers asked an AI machine trained to collect text from Google News to complete the following statement: "Man is to computer programmer as woman is to X." It's reply? "Homemaker."

Unfortunately, these examples may only confirm what we already know about representations of gender out and about the virtual world. But as AI machines take on more prevalent roles in the infrastructure of our physical and virtual environment—and as they take on more roles programmed to serve the aims of people and corporations—there is a real need to consider who will and will not benefit from these biases.

AI: Work in Progress

Creativity and artistic bursts of innovation come into full bloom at the intersections of cultures, disciplines, and technologies. When we think about how ideas are brought to life, we can catch a glimpse of how ideas move, branch off, or merge when we peer into these in-between spaces. The burgeoning field of AI gives not only new eyes, but new terrains. Within this space are areas of serious ethical consideration. The art of this area may raise more questions than answers, but this seems to mimic the AI field itself.

Additional Resources

To see behind-the-scenes of preparing for the Sight Machine performance, watch this 5:00 minute video.

Follow up Paglen's Sight Machine with another intriguing project: A Study of Invisible Images, featured in 1843 Magazine. In this project, Paglen trained AI on human inventions of thought ("omens and portents;" objects from Freud's "The Interpretation of Dreams;" "American predators") and then "asked" the AI to create its own images for each of the different categories. The final selection of images are those that a second AI trained with the same categories was able to identify.