ABSTRACT

As introduced in Chapter 5, multimodal interaction and interfaces enable people to use two or more input modalities alone or in combination, such as speech, pen, touch, manual gestures, and gaze. Interpersonal communication is inherently multimodal, so it may not be surprising that people also prefer to communicate multimodally when using a computer (Oviatt, 2012; Oviatt, Coulston et al., 2004). Chapter 7 presents empirical research from the cognitive, linguistic, computational, and learning sciences on how human communication and cognition are inuenced by multimodal interface design. It also discusses how multimodal interfaces leverage our multisensory processing abilities to facilitate problem solving and learning.