Unlocking Data, Unlocking Interaction

Date

Oct 12, 2016

Time

12:00 PM - 1:30 PM

Location

Kane 225

Title

Unlocking Data, Unlocking Interaction

Speaker

Abstract

The applications we create are framed by the tools we use to create them. On one hand, tools codify effective practice and empower design. On the other, that same codification eventually constrains design. My research examines new approaches to interactive systems in light of this tradeoff, often with an emphasis on unlocking existing codifications to enable new designs. This talk will focus on three examples:

• I will first present our work on unlocking data with interactive machine learning. Dominant models of interaction fail to support expressiveness and control in many emerging forms of everyday data. Exploring such domains as web image search and gesture recognition, our work shows how interactive machine learning can support people in extending the underlying language of an interaction.
• I will then present our work on using pixel-based reverse engineering to unlock existing graphical interfaces, allowing runtime modification of those interfaces without their source. Pixel-based methods allow prototyping new possibilities atop the existing ecosystem of applications and tools, accelerating innovation and informing the next-generation ecosystem.
• Finally, I will consider how these challenges combine in the emergence of self-tracking and personal informatics. Data is no longer a distant concept, but an everyday barrier to interaction, self-knowledge, and personal empowerment. The tools we create to support these applications will define the future of everyday interaction with personal data.

Given these examples, I argue research must consider not only specific applications, but also the assumptions codified by underlying tools and how those tools frame our understanding of what application designs are even possible.

Bio

James Fogarty is an Associate Professor of Computer Science & Engineering at the University of Washington (https://homes.cs.washington.edu/~jfogarty). His broad research interests are in engineering interactive systems, often with a focus on the role of tools in developing, deploying, and evaluating new approaches to the human obstacles surrounding everyday adoption of ubiquitous computing and intelligent interaction.

Video