As an academic, I find that I do not get enough exercise. This is common, though research by Whipple, Kinney, and Kattenbraker (2008) suggests that being well-educated often correlates with higher physical activity, despite the obvious and common barriers to working out many in higher education have. My knowledge of this deficit has led me to repeatedly ask one of my colleagues, a dear friend, to go on walks with me on campus. On these walks, we take our steps, but also engage in conversation that is of a stimulating nature, usually related to our vocation, Technical Communication.
Today, while on one such walk, we happened upon a discussion of a new topic alarming to us and the field of Tech Comm, or at least we think it should be. That topic is Artificial Intelligence User Interfaces (or AIUIs) and the fate of the graphical user interface (or GUI).
If you are unfamiliar with AIUI, do not worry, you are not alone! AIUI is defined as an interface design that enables 2-way verbal communication with an Artificial Intelligence (AI). It aims to close the gap between human users and AI agents. It is believed that AIUI will make it easier for people to interact with AI in a natural and conversational manner. This, on its face, is not what troubled us during our jaunt—in fact, we tend to agree with and appreciate the usability and accessibility affordances that the addition of such a UI can add to technology. What we found to be problematic is that during a recent interview Sam Altman, CEO of OpenAI, spoke about using AI as the main interface between users and their technology foretelling the potential fate of the GUI to the world (Matthew Berman, 2024).
The replacement of the GUI with an AIUI of this order, even just the thought of it, immediately put my mind to work on a response to this monumental shift (you happen to be reading something of the result right now). The use of GUI to interact with computer technology has become ubiquitous since Xerox developed Alto in 1973 (KASS, n.d.). So, the suggestion that an AIUI is set to replace the long-standing GUI fills me with great trepidation and causes concern on behalf of human users. And, before you call me a Luddite or decry this work as the ponderance of an entrenched academic pushing back against a paradigmatic shift as Khun (2012) proffered at such precipices, please allow me to explain.
When Altman suggested that GUIs be replaced by AIUIs, something primordial stirred within me, a deep-seated aversion to acting in the world without the visual information I have become accustomed to as my guide.
Though I am early in my academic career, I have begun to be recognized as a scholar with an investment in visual communication, specifically information design for today’s media-rich environments. According to the International Visual Literacy Association (IVLA, n.d.), we are living in a visually rich digital world, and some contemporary scholarship on communication in human Culture has uncovered that we are trending back toward a more simplified communication culture—away from a literate (or written) one (Ong, 2013; Drucker & McVarish, 2021). Scholars working in this area, myself included, are attributing the shift to the highly visual nature of online communication, products, and services (Brumberger, 2007; Gallagher, 2019). In addition to this research, I currently teach courses in visual communication, information design, usability, and accessibility in UX. In fact, a current book project for Routledge, one my friend and I are both on, is the development of a new edition of Designing Visual Language for professional and technical communication courses. So, I am intimately interested in the future of visual communication, especially of the digital and interactive sort like GUIs.
In the moment of fervor, I experienced when learning of the GUI’s potential fate, I exclaimed, “We cannot so easily discard the GUI. It goes against the nature of human information processing to strip away visual communication of information!” According to Danish physicist Tor Nørretranders, who conducted research to convert the bandwidth of human senses to computer data amounts, he found that our sense of sight has the highest bandwidth for data input at about 1250mbs (megabytes per second), equivalent to a computer network (McCandless, 2010). It is our most powerful sense for the uptake of information. So, I ask, does Altman genuinely believe that AI will do away with our highest-bandwidth method of information transmission between humans and their technology in favor of AIUI—whose interactions with us are auditory? I say, let us not be too hasty, Sam.
The speed of data transfer via aurality or our sense of hearing, per Nørretranders, only clocks in at about 12.5mbs, or the same random write speed as an old-fashioned WD hard disk drive (McCandless, 2010).
I do not know about you, but I am not willing to trade my network speed sight for any old sound platter just yet! I crave the speed of visual information transfer between me and my devices while I interact with them, still more, I relish the control that click-and-touch interaction places in my hands. If we get rid of the GUI, and talk to our devices, 1) that is not very accessible on its own, and 2) how will we know what the AI is doing in response to our prompts to interact with our device? Will it show us? Can we stop it or correct it? What if the AIUI stops working and we need to interact with the device? What then? There is a veritable Pandora’s box of problems that we can encounter when, and if, we face the death of the GUI. However, not everyone is as pessimistic as me on this point. Both my colleague and Google’s BARD believe that there is less need for a GUI in the future of technology.
To argue the shrinking need for GUI, I excerpt BARD’s response to my prompting to take a side on the future of the GUI in the face of AIUI. BARD had this to say:
Under the hypothetical scenario that I must choose, I would cautiously argue for a gradual shift away from GUIs towards more hybrid interfaces that incorporate voice, natural language, and other modalities alongside carefully designed visual elements.
Here is my reasoning:
1. Accessibility and inclusivity: Voice and natural language interfaces can be more accessible to a wider range of users, including those with visual impairments, motor disabilities, or limited literacy. This aligns with a vision of technology that is inclusive and empowering for all.
2. Naturalness and intuitiveness: Interacting with voice assistants and AI agents through natural language feels more akin to human conversation, potentially reducing the learning curve and cognitive load for users, especially for everyday tasks.
3. Context-awareness and personalization: Future interfaces that combine modalities can leverage environmental and user context to provide more intelligent and personalized experiences. For example, imagine a voice assistant offering relevant information or completing tasks based on your location, activity, and previous interactions.
4. Efficiency and multitasking: Voice and natural language interfaces can free up our hands and eyes for other tasks, enabling efficient multitasking and hands-free interaction in various scenarios.
Dear user, as you can observe, like my friend and I, BARD recognizes the readily accessible and potentially inclusive nature of adding an AIUI to contemporary technology. Furthermore, BARD offers that an AIUI can make it easier for us to use our devices more efficiently, which I want to take issue with briefly. BARD’s idea of making our devices easier to use is a bit of a misnomer or at least an oversimplification. I admit, using AIUI to have a device dim your lights or provide you with a template for your business letter is easy. But what is not as easy is knowing how the AI interacts with your internet of things (IoT) or how to find (and use) that letter template well by yourself. In short, as we become more dependent on technology and it becomes more intuitive, we know less about how it works and how to use it in more than a superficial way (UOPeople.edu, n.d.). We accept this already. But what happens when we give over the little interaction, we still do have with our devices to AIUI?
Well, we may know even less and be even more dependent on technology, I am afraid. But even BARD pushes back on Altman, providing us with a spark of hope. BARD states,
However, this shift [to AIUI] would not be an absolute abandonment of GUIs:
Visual elements will remain crucial: For tasks requiring precision, spatial awareness, or complex data visualization, well-designed visual interfaces will still be essential.
Hybrid approaches hold the key: The future lies in seamless integration of various modalities like voice, touch, and gesture, depending on the task and user context.
Accessibility must be paramount: The transition should prioritize accessibility, ensuring that everyone, regardless of ability, can benefit from these advancements.
In conclusion, while GUIs have served us well, a future with more natural, accessible, and context-aware interfaces, enabled by a thoughtful blend of modalities, holds immense potential to enhance human-computer interaction in meaningful ways. Remember, this is just one perspective, and the path forward will require careful consideration and collaboration among various stakeholders.
BARD’s view of our future with GUIs is limited, but hopeful. However, its grasp on the essential situations where GUIs will be necessary is tenuous. And it does not seem to consider the necessary means by which users transact most information with their devices, via the sense of sight. Far be it from me to forget about accessibility and inclusivity, I regularly publish on the power of understanding diversity (Arp, Tye-Williams, and Gallagher, 2023) and how to make technical communications more accessible for every body through making visuals accessible for low-vision and blind users, as well as making sound accessible for users who have hearing loss or who are deaf (Gallagher & Gallagher 2022; Gallagher & Gallagher 2024; Gallagher & Gallagher, forthcoming). I agree with BARD, that an AIUI can improve the equity of interaction between some users with disabilities and their devices, but I do not think we should stray too far from keeping the GUI at the center of human-computer interaction, while providing AIUI a secondary, but still significant role to play in the exchange.
What do you think of The Death of the GUI?
References
Arp, A., Tye-Williams, S., & Gallagher, P. B. (2023). “Um, Diversity Definition…that’s Hard:” Student communication about diversity in the classroom and beyond. Communication Studies. https://doi.org/10.1080/10510974.2023.2274133
Brumberger, E. R. (2007). Making the strange familiar: A pedagogical exploration of visual thinking. Journal of Business and Technical Communication, 21(4), 376-401. https://doi.org/10.1177/1050651907304021
Drucker, J., & McVarish, E. (2021). Graphic Design History: A Critical Guide. (2nd edition). Boston, MA: Pearson. https://www.pearson.com/en-us/subject-catalog/p/graphic-design-history/P200000002800/9780137559176
Gallagher, P. B. (2019). Design: Changing zeitgeists, changing communication. In D. M. Baylen (Ed.), IVLA Book of Selected Readings 2018/19. (pp. 151-162). IVLA Press. https://ivla.org/welcome-to-the-ivla-book-of-selected-readings-2019/
Gallagher, P. B. & Gallagher, M. J. (2022). New era, same problem: Making visual content accessible for visually impaired Users. 2022 STC Summit Conference & Expo Conference Proceedings. (pp. 66-71). https://www.researchgate.net/publication/360630250_New_Era_Same_Problem_Making_Visual_Content_Accessible_for_Visually_Impaired_Users
—. (2024). Preparing professionals to make equitable experiences of visual information for users with vision impairment. In A. Lancaster & C. S. T. King (Eds.), Amplifying Voices in UX: Balancing Design and User Needs in Technical Communication. (pp. Routledge. https://sunypress.edu/Books/A/Amplifying-Voices-in-UX
—. (Forthcoming). Accessible sound: Aural information literacy for technical communication design. Communication Design Quarterly.
International Visual Literacy Association. (n.d.). What is visual literacy? Retrieved from https://visualliteracytoday.org/what-is-visualliteracy/
KASS. (n.d.). The graphical user interface history. KASS: KOCHI ARTS & SCIENCE SPACE. https://kartsci.org/kocomu/computer-history/graphical-user-interface-history/
Khun, T. S. (2012). The Structure of Scientific Revolutions. (4th edition). Chicago Press. https://press.uchicago.edu/ucp/books/book/chicago/S/bo13179781.html
Matthew Berman. (2024, January 23). Sam Altman Just Revealed NEW DETAILS About GPT-5 In Spicy Interview [Video]. YouTube. https://www.youtube.com/watch?v=RYg5Mz4_tf8
McCandless, D. (2010, July). The beauty of data visualization [Video]. TED. https://www.ted.com/talks/david_mccandless_the_beauty_of_data_visualization?language=en
Ong, W. J. (2013). Orality and literacy. (3rd edition). London, UK: Routledge. https://www.routledge.com/Orality-and-Literacy-30th-Anniversary-Edition/Ong/p/book/9780415538381
UOPeople.edu. (n.d.). Is society too dependent on technology? 10 Shocking facts. UOPeople.edu. Retrieved from https://www.uopeople.edu/blog/society-too-dependent-on-technology/
Whipple, K., Kinney, J., & Kattenbraker, M. (2008). Maintenance of physical activity among faculty and staff in university settings. The Health Educator, 40(1), 21-28. https://files.eric.ed.gov/fulltext/EJ863506.pdf