The world is full of visual information that is not easily accessible. For blind people, frustrating accessibility problems because of vision are commonplace and pervasive. For space owners, important visual information that could be used to help them better monitor, manage, and optimize the environment is often left uncaptured. Two trends are converging that make solving these problems tractable: artificial intelligence (AI) and human computation. Although AI shows promise in understanding the visual world, they struggle in many real-world, uncontrolled situations, and do not easily generalize across diverse human environments. On the other hand, humans can be more robust and flexible in cases where AI systems fail. However, using human intelligence is slow and expensive, thus not scalable.
In my work, I investigate hybrid human- and AI-powered methods to provide robust and interactive access to visual information in the real world. They tradeoff between the advantages of humans and AI to create systems that are nearly as robust and flexible as human, and nearly as quick and low-cost as automated AI. To make physical interfaces accessible for blind people, I develop (i) VizLens, a screen reader to help blind people access static physical interfaces; (ii) Facade, a crowdsourced fabrication pipeline to automatically generate tactile overlays to appliances; and (iii) StateLens, a solution that makes existing dynamic touchscreens accessible. Furthermore for environmental sensing, I develop and deploy (iv) Zensors++, a camera sensing system that collects human labels to bootstrap automatic processes to answer real-world visual questions, allowing end users to actionalize AI in their everyday lives.
Anhong Guo is a Ph.D. candidate in the Human-Computer Interaction Institute in the School of Computer Science at Carnegie Mellon University, advised by Dr. Jeffrey Bigham. He is also a Snap Inc. Research Fellow, and a Swartz Innovation Fellow for Entrepreneurship. He has published in many top academic conferences in interface technologies, wearable computing, accessibility and computer vision, including two best paper nominees. Before CMU, he received his Master’s in HCI from Georgia Tech. He has also worked in the Ability and Intelligent User Experiences groups in Microsoft Research, the HCI group of Snap Research, the Accessibility Engineering team at Google, and the Mobile Innovation Center of SAP America. See more at: https://guoanhong.com