CollAid: User Differentiation of Speech and Touch at the Tabletop

Tabletops have the potential to provide new ways to support collaborative learning generally and, more specifically, to aid people in learning to collaborate more effectively. To achieve this potential, we need to gain understanding of how to design tabletop environments so that they capture relevant information about collaboration processes so that we can make it available in a form that is useful for learners, their teachers and facilitators.
Arquitecture

We design our Collaid (Collaborative Learning Aid) environment in order to be able to capture  multimodal data about collaboration from tabletop activity using a microphone array and a depth sensor. This made it possible to integrate these data with other parts of the learning system; transforming the data into visualisations depicting the processes that occurred during the collaboration at the table; and applying sequence mining of the interaction logs.

System

Collaid extends an ordinary interactive tabletop to differentiate which learner is touching what in a non-intrusive manner. It relies on an overhead depth sensor (http://www.xbox.com/kinect) that associates each touch performed on the interactive surface with a specific student. The system captures the overhead depth video stream and then, making use of a greedy search algorithm, matches the touch with the position of each learner (Figure 1, top left). Additionally, we capture verbal participations and turn-taking through an array of microphones situated on one of the edges of the tabletop (Figure 1, right). We used a 7-channel microphone (http://www.dev-audio.com) that distinguishes sounds based on the location of the source, in our case, the learners seating around the tabletop. The audio information is recorded into audio files and the shared database.
modular_structure_without_extensions

Publications:

R. Martinez, A. Collins, J. Kay, and K. Yacef. Who did what? who said that? Collaid: an environment for capturing traces of collaborative learning at the tabletop. In ACM International Conference on Interactive Tabletops and Surfaces, ITS 2011, pages 172-181, 2011.

Clayphan, A., Martinez-Maldonado, R., Ackad, C. and Kay, J. (2013) An approach for designing and evaluating a plug-in vision-based tabletop touch identification system. Australian Computer-Human Interaction Conference, OZCHI 2013, pages 373-382.

 

Leave a Reply

Your email address will not be published. Required fields are marked *