In second language acquisition (SLA) research, the attention of scholars has traditionally focused more on verbal language than on non-verbal language such as gestures. It is only in the last few years that the relationship between gestures and speech in second language (L2) oral communication has interested a number of scholars, as the topic introduces a new way to investigate L2 development and acquisition (see discussion in Gullberg, 2006, 2009). People around the world perceive and interpret the form, meaning, and function of gestures in distinctive ways, whether gestures occur in combination with verbal language or by themselves (Gullberg, 1998, 2006; Gullberg & McCafferty, 2008; Jungheim, 2006; Kendon, 2004; McCafferty & Gullberg, 2008; McCafferty & Stam, 2008; McNeill, 1992, 2000, 2005). In this chapter we will discuss the interpretation of L2 gestures by L2 learners with different linguistic and cultural backgrounds in order to offer insights towards the understanding of how bilingual and multilingual speakers organize concepts and meanings in their minds and in their language.