research > Proximity Toolkit

Proximity Toolkit

Prototyping Proxemic Interactions in Ubiquitous Computing Ecologies

Proximity Toolkit

Top: three entities – person, tablet and vertical surface; Bottom: visualizing relationships between entities (e.g., orientation, distance, pointing rays) in the Proximity Toolkit's visual monitoring tool.

Ubicomp ecologies are now common, where people's access to digital information increasingly involves near-simultaneous interaction with multiple nearby digital devices of varying size, e.g., personal mobile phones, tablet and desktop computers, information appliances, and large interactive surfaces. This is why a major theme in ubiquitous computing is to explore novel forms of interaction not just between a person and a device, but also between a person and their set of devices. Proxemic interaction is one strategy to mediate people's interaction in room-sized ubicomp ecologies. It is inspired by Hall's Proxemic theory [11] about people's understanding and use of inter-personal distances to mediate their interactions with others. In proxemic interaction, the belief is that we can design systems that will let people exploit a similar understanding of their proxemic relations with their nearby digital devices, thus facilitating more seamless and natural interactions.

A handful of researchers have already explored proxemic-aware interactive systems. These range from spatially aware mobile devices, office whiteboards, public art installations, home media players, to large public ambient displays. All developed novel interaction techniques as a function of proxemic relationships between people and devices.

Building proxemic-aware systems, however, is difficult. Even if sensing hardware is available, translating low-level sensing information into proxemic information is hard (e.g., calibration, managing noise, calculations such as 3D math). This introduces a high threshold for those wishing to develop proxemic interaction systems. As a result, most do not bother. Of the few that do, they spend most of their time with low-level implementation details to actually access and process proxemic information vs. refining the interaction concepts and techniques of interest.

To alleviate this problem, we built the Proximity Toolkit. Our goal was to facilitate rapid exploration of proxemic interaction techniques. To meet this goal, the Proximity Toolkit transforms raw tracking data gathered from various hardware sensors (e.g., infra-red motion capturing systems, depth sensing cameras) into rich high-level proxemic information accessible via an event-driven object-oriented API. The toolkit includes a visual monitoring tool that displays the physical environment as a live 3D scene and shows the proxemic relationships between entities within that scene. It also provides other tools: one to record events generated by entities for later playback during testing; another to quickly calibrate hardware and software.

Publications

The Proximity Toolkit: Prototyping Proxemic Interactions in Ubiquitous Computing Ecologies

The Proximity Toolkit: Prototyping Proxemic Interactions in Ubiquitous Computing Ecologies

Marquardt, N., Diaz-Marino, R., Boring, S., and Greenberg, S.

In ACM Symposium on User Interface Software and Technology - UIST'2011. Santa Barbara, CA, USA, ACM Press, 11 pages, October 16-19.