Interaction Using Situated Spatial Gestures

From The Theme


What if we could improve interfaces for context, comprehension and human-machine performance, in order to develop machines that could sense and respond to people’s movements and behaviors?

Spacial Gestures Image

We set out to improve human-machine interaction by enabling machines to detect and respond to people’s actions, gestures and motions. Our approach applied state of the art spatial sensing and goal inference technologies to responsive interaction design. We experimented with Microsoft Kinect and video cameras for gesture capture and to create models of where people are in space, how they are moving, and what their intent and activities are. We also designed a robotic sofa (Sofabot) for testing interactions. Our design process used personality stereotypes to help drive the design of robotic behavior and interactions.

The process of designing an interactive robotic sofa (Sofabot) demonstrated how the use of metaphors and stereotypes of personality facilitate and enrich the design process, allowing designers to base their design choices on a strong inner logic. Our research found that people interacting with Sofabot found the robot’s behavior to be consistent with its personality. That is, users were able to recognize different behaviors designed according to different stereotypes of personality. We believe that our method can contribute to the design of more engaging and richer experiences.

Marco Spadafora, Victor Chahuneau, Nikolas Martelaro, David Sirkin, and Wendy Ju. 2016. Designing the Behavior of Interactive Objects. In Proceedings of the TEI ’16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction (TEI ’16). ACM, New York, NY, USA, 70-77. DOI:

Larry LeiferLarry Leifer is Professor of Mechanical Engineering at Stanford University and Director of the Center for Design Research. Dr. Leifer’s engineering design thinking research is focused on instrumenting design teams to understand, support, and improve design practice and theory. Specific issues include: design-team research methodology, global team dynamics, innovation leadership, interaction design, design-for-wellbeing, and adaptive mechatronic systems.

Wendy JuWendy Ju is an Assistant Professor, Information Science, at Cornell Tech. At the time of this project and during her time at Stanford, she was the Executive Director of Interaction Design Research at the Center for Design Research (CDR). Dr. Ju’s research in the areas of physical interaction design and ubiquitous computing investigates how implicit interactions can enable novel and natural interfaces through the intentional management of attention and initiative.