This study examines the co-occurrence of speech and gestures based on a multimodal corpus to investigate the extent to which speech-accompanying gestures differ between Chinese as L1 and English as L2 speech among speakers of different L2 proficiency levels. Thirty-two Chinese-speaking learners of English, equally distributed between advanced (C1) and low-intermediate (B1) proficiency levels were recruited. The face-to-face casual conversation in L1 and L2 among friends were video-recorded, and speech-accompanying gestures were then coded for different types: deictic gestures, iconic gestures, metaphoric gestures, beats, and others. The analysis of overall frequencies of L2 gestures, English proficiency level was found to have a significant effect; in particular, speakers at higher levels of L2 proficiency are more likely to produce more beats and iconic gestures in conversational interaction for additional emphasis, while less proficient L2 speakers tended to produce more deictic gestures, accompanied with other communication strategies. Furthermore, the comparison of the types of gestures that accompanied the speakers’ L1 and L2 conversation show that except metaphoric gestures, speakers tended to produce more beats, deictic and iconic gestures in L2 than in L1, with similar results found in both advanced and low-intermediate learners.