Machine Vision: How Algorithms are Changing the Way We See the World.
by Jill Walker Rettberg, (Polity Press, 2023)
In this lively, information-rich, book, Jill Walker Rettberg discusses the ways automated visual technology is changing our understanding of identity, security, and even what constitutes the visible world. This book is not about AI exclusively, but focuses on computationally automated processes, those activities in the technology of vision that are driven by algorithms, some “smarter” than others.
Rettberg’s project is a critical investigation of the cultural changes that current innovations are enacting in our lives. These changes are not merely new capacities to see in the dark, have optical access to the microscale of the world, or find images of guitar-playing frogs in a collection of other cartoon amphibians, but instead, in Rettberg’s view, are transformations of the larger “assemblages” of human-technology relations. Throughout her discussion, Rettberg keeps this concept of assemblages central—the identity and impact of technology are co-dependent with the social and cultural circumstances in which they operate. Technology is not simply a thing, an object or device, but is always embedded in relationships with activities and beliefs. Among the examples on which she demonstrates these principles is a contrast between the use of surveillance camera systems in urban Chicago and a town in Norway. Same technology, completely different effect, perception, and evaluation of the costs and benefits. Rettberg’s personal and professional experience have immersed her in a variety of lived circumstances. The impact on her academic study of digital culture has been a visceral realization that technology is neither neutral nor independent of its circumstances of use. In other words, the agency wielded by optical technology is not deterministic, it is conditional.
The book is organized in five chapters, each focused on different aspects of augmented visual experience—Seeing More, Seeing Differently, Seeing Everything, Being Seen and Seeing Less. The book is filled with ideas and insights about cultural patterns and the ways technology participates in expectations for and shaping of experience. Rettberg is intent on emphasizing that increasingly we understand ourselves and the world through digitally mediated images and the systems and operations to which they are attached, whether we realize this or not. Her examples are provocative. Who doesn’t have an opinion about selfies and the algorithms that change the proportions of our features without our realizing? Or have thoughts about automated shopping in unstaffed stores that use facial recognition? Or the ethics of security cameras weighed against their benefits? Again, rather than lift these innovations out of their circumstances as if they operated independently, Rettberg is careful to situate them in such a way that the technologies are seen to inform human behaviors as well as being themselves formed to serve certain purposes.
Rettberg’s intellectual development is apparent here, and her fluency with critical concepts makes her adept at practicing the “situated data analysis” of which she speaks in the introduction. No perception, action, or innovation is “outside of” the circumstances in which it appears, is perceived, and operates. The final term in that series is crucial, since Rettberg is also keen to engage with the distinction she draws between “representational” images, those that show something, and “operational” images, those that do something. Given the long history of visualization with which she begins her study, this is important since the latter category has not always been familiar. Human beings have long understood that images picture or stand in for something, they are pictures of something. But the concept of pictures as instruments of intellectual thought or cognitive activity may be less familiar. In a world where visual interfaces are ubiquitous, and their graphical features organize our interaction with complex systems, the need to think about the way such graphical forms perform and to a great extent limit and program our relation to many activities is highly important.
Rettberg’s first chapter, “Seeing More,” offers a succinct history of optical technology—beginning with the oldest mirror, a surface of polished black obsidian which may date to 6000 BCE. She then discusses ground glass lenses used in antiquity for magnification, and other extensions of human visual capacity from light-based to infrared cameras. Rettberg has the ability to infuse these inventions with a sense of wonder—imagining what it might have been like to see oneself as an image for the first time reflected in a shiny rock. She covers a long time span by using carefully chosen examples in a necessarily selected and abbreviated history. The invention of perspectival systems for the visual arts, for instance, well-known and charted territory in art historical studies, is presented here with a striking example: In 1425 the painter Filippo Brunelleschi exhibited a canvas depicting buildings in his native Florence. He put a peephole through it, and offered the viewers an opportunity to stare through to a reflection of the image in a mirror. By constraining the field of vision, Brunelleschi reinforced the immersive effect of the illusion. So rather than describing the 15th-century Renaissance perspectival system of Leon Battista Alberti with its rules and principles, Rettberg situates it in an embodied relation of reciprocity between viewer and image. The chapter continues with discussion of the ways perspective created new expectations for viewing that become structured into cameras as photographs produced novel points of view into mainstream culture.
She begins her second chapter, “Seeing Differently,” with a description of images she receives from her robo-vacuum, familiarly known in the household as Alfred, as an introduction to machine vision technologies. Here she draws on the concept of “non-conscious cognition” developed by N. Katherine Hayles to describe a capacity to interpret information within contexts and apply it. This is a useful alternative to the positing of “intelligence” or “consciousness” within automated systems and stresses the algorithmic aspect of developing the capacity to process optically obtained information by using various parameters. In other words, the concept works well to describe the efficacy of the formally structured systems that are fundamental to computation. Alfred does not need to “know” anything or “think” to understand that an encounter with an unfamiliar object of a certain size and shape is outside of the categories its systems can match. So, Alfred sends a picture to Rettberg’s phone. Beyond this, Rettberg’s historical and theoretical discussion ranges widely, from Soviet filmmaking and the notion of the “Kino-eye” to more contemporary media theory in the writings of Wilhelm Flusser’s idea of the human feedback machine that helps camera technology advance and improve, a dramatic demonstration of Rettberg’s idea of assemblage.
“Seeing Everything,” the third chapter, is framed around personal experiences in Oak Park, a Chicago suburb, and an assault suffered by her husband that was recorded on surveillance cameras in a train station. The unfolding of the argument moves through various venues and situations in which such technologies and related facial recognition systems are integrated into social space. She uses the tension between trust and fear, the differences in attitudes in Norway and the United States communities with which she is familiar, to connect ideas of “omnivoyance” with technologies that operate in a way that various scholars have identified as “frameless” or having a “pointless view” because they lack a clear position from which they are produced. Again Rettberg emphasizes the ways technologies are considered has a major effect on how they operate. Systems are not deterministic, but are part of interactions and feedback logics.
“Being Seen” discusses systems of facial and physical identification and analysis that make use of various metrics in ways that extend some of the 19th century work (e.g. Adolphe Quetelet) using statistical methods to determine standard norms within human populations. The chapter shifts to discussion of staffless libraries and grocery stores and the technological infrastructure on which they work. The final section brings in a popular culture reference, the fictional figure of Thunderhead from the Arc of the Scythe series of young adult novels by Neal Shusterman. In an image of supposedly benevolent AI, that story describes “a shift in society where AI and machine vision have become integral parts of our world.” (137) Thunderhead understands its role as caring, not surveillance, in providing benevolent guidance to the human populations. Rettberg registers her ambivalence about the characterization, fully aware of the oppressive aspects of omniscience. The final full chapter on “Seeing Less” deals with the “Blindspots of Machine Vision” where biases and ethical concerns are addressed, but she ends on a hopeful note in her concluding remarks, perhaps because she lives in society where a degree of civility and respect prevail.
Many more details and citations could be pulled from Rettberg’s book that show her ability to combine theoretical frameworks from cyborg theory and visual studies, historical knowledge, technical expertise, and popular as well as literary references. The compelling issues of algorithmic agency, emergent properties of smart systems, are woven throughout, but other speculative and dystopic topics hover on the edges of her arguments, held at bay by a pragmatic and balanced sensibility that privileges engagement with what is rather than what might be. Rettberg frequently reiterates that we are situated in complex networks of information processing, communication exchange, social and material systems whose intersections are manifold and inflect our perception. As she states clearly in referencing the “technological imaginary,” it is not just the way these devices work that matters, it is the way we think they work that becomes instrumentalized in cultural discourse.
My final enthusiastic response to this book was generated by the recent texts Rettberg references with which I was not familiar. These citations almost made me wish I were still teaching so I could assemble a course on “machine vision.” This would connect her work with that of many she cites, but also others from art history, visual studies, and digital media whose analyses of vision, representation, reproduction, and transmission create a rich field for research and knowledge of our ongoing experience of the world.