Jump to ContentJump to Main Navigation
Show Summary Details
More options …

Proceedings on Privacy Enhancing Technologies

4 Issues per year

Open Access
See all formats and pricing
More options …

Don’t Interrupt Me While I Type: Inferring Text Entered Through Gesture Typing on Android Keyboards

Laurent Simon / Wenduan Xu / Ross Anderson
Published Online: 2016-05-06 | DOI: https://doi.org/10.1515/popets-2016-0020


We present a new side-channel attack against soft keyboards that support gesture typing on Android smartphones. An application without any special permissions can observe the number and timing of the screen hardware interrupts and system-wide software interrupts generated during user input, and analyze this information to make inferences about the text being entered by the user. System-wide information is usually considered less sensitive than app-specific information, but we provide concrete evidence that this may be mistaken. Our attack applies to all Android versions, including Android M where the SELinux policy is tightened.

We present a novel application of a recurrent neural network as our classifier to infer text. We evaluate our attack against the “Google Keyboard” on Nexus 5 phones and use a real-world chat corpus in all our experiments. Our evaluation considers two scenarios. First, we demonstrate that we can correctly detect a set of pre-defined “sentences of interest” (with at least 6 words) with 70% recall and 60% precision. Second, we identify the authors of a set of anonymous messages posted on a messaging board. We find that even if the messages contain the same number of words, we correctly re-identify the author more than 97% of the time for a set of up to 35 sentences.

Our study demonstrates a new way in which system-wide resources can be a threat to user privacy. We investigate the effect of rate limiting as a countermeasure but find that determining a proper rate is error-prone and fails in subtle cases. We conclude that real-time interrupt information should be made inaccessible, perhaps via a tighter SELinux policy in the next Android version.

Keywords: mobile; smartphone; android; side channel; interrupt; typing; gesture; gesture typing; SwiftKey; Google keyboard; keyboard; procfs; virtual file system; virtual file; artifical neural network; neural network; reccurent neural network; RNN; machine learning; ML


  • [1] A. T. Ozcan, C. Gemicioglu, K. Onarlioglu, M. Weissbacher, C. Mulliner, W. Robertson, and E. Kirda, “BabelCrypt: The Universal Encryption Layer for Mobile Messaging Applications,” in Financial Cryptography and Data Security (FC), 01 2015.Google Scholar

  • [2] K. Zhang and X. Wang, “Peeping tom in the neighborhood: Keystroke eavesdropping on multi-user systems,” in Proceedings of the 18th Conference on USENIX Security Symposium, SSYM’09, (Berkeley, CA, USA), pp. 17-32, USENIX Association, 2009.Google Scholar

  • [3] Q. A. Chen, Z. Qian, and Z. M. Mao, “Peeking into your app without actually seeing it: UI state inference and novel android attacks,” in Proceedings of the 23rd USENIX Security Symposium, San Diego, CA, USA, August 20-22, 2014., pp. 1037-1052, 2014.Google Scholar

  • [4] X. Zhou, S. Demetriou, D. He, M. Naveed, X. Pan, X. Wang, C. A. Gunter, and K. Nahrstedt, “Identity, lo cation, disease and more: Inferring your secrets from android public resources,” in Proceedings of the 2013 ACM SIGSAC Conference on Computer & Communications Security, CCS ’13, (New York, NY, USA), pp. 1017-1028, ACM, 2013.Google Scholar

  • [5] A. Savitzky and M. J. E. Golay, “Smoothing and Differentiation of Data by Simplified Least Squares Procedures.,” Anal. Chem., vol. 36, pp. 1627-1639, July 1964.Google Scholar

  • [6] T. Mikolov, M. Karafiát, L. Burget, J. Cernock`y, and S. Khudanpur, “Recurrent neural network based language model.,” in INTERSPEECH 2010, 11th Annual Conference of the International Speech Communication Association, Makuhari, Chiba, Japan, September 26-30, 2010, pp. 1045-1048, 2010.Google Scholar

  • [7] T. Mikolov, S. Kombrink, L. Burget, J. H. Cernock`y, and S. Khudanpur, “Extensions of recurrent neural network language model,” in Acoustics, Speech and Signal Processing (ICASSP), 2011 IEEE International Conference on, pp. 5528-5531, IEEE, 2011.Google Scholar

  • [8] C. D. Manning and H. Schütze, Foundations of Statistical Natural Language Processing. Cambridge, MA, USA: MIT Press, 1999.Google Scholar

  • [9] J. L. Elman, “Finding structure in time,” Cognitive science, vol. 14, no. 2, pp. 179-211, 1990.Google Scholar

  • [10] D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning representations by back-propagating errors,” Cognitive modeling, vol. 5, no. 3, p. 1, 1988.Google Scholar

  • [11] P. J. Werbos, “Backpropagation through time: what it does and how to do it,” Proceedings of the IEEE, vol. 78, no. 10, pp. 1550-1560, 1990.Google Scholar

  • [12] “Android apps in sheep’s clothing.” http://www.modzero.ch/modlog/archives/2015/04/01/android_apps_in_sheeps_clothing/index.html.Google Scholar

  • [13] “Currentapp.java.” https://gist.github.com/jaredrummler/07a3f723e96ec06fb761.Google Scholar

  • [14] “Activitymanager.” https://developer.android.com/reference/android/app/ActivityManager.html#getRunningTasks%28int%29.Google Scholar

  • [15] E. N. Forsythand and C. H. Martell, “Lexical and discourse analysis of online chat dialog,” in Proceedings of the International Conference on Semantic Computing, ICSC ’07, (Washington, DC, USA), pp. 19-26, IEEE Computer Society, 2007.Google Scholar

  • [16] S. Bird, E. Klein, and E. Loper, Natural Language Processing with Python. O’Reilly Media, Inc., 1st ed., 2009.Google Scholar

  • [17] J. Munkres, “Algorithms for the assignment and transportation problems,” 1957.Google Scholar

  • [18] N. Zhang, K. Yuan, M. Naveed, X. Zhou, and X. Wang, “Leave me alone: App-level protection against runtime information gathering on android,” 2015.Google Scholar

  • [19] Y. Michalevsky, G. Nakibly, A. Schulman, and D. Boneh, “Powerspy: Location tracking using mobile device power analysis,” arXiv preprint arXiv:1502.03182, 2015.Google Scholar

  • [20] L. Simon and R. Anderson, “Pin skimmer: Inferring pins through the camera and microphone,” in Proceedings of the Third ACM Workshop on Security and Privacy in Smartphones & Mobile Devices, SPSM ’13, (New York, NY, USA), pp. 67-78, ACM, 2013.Google Scholar

  • [21] P. C. Kocher, J. Jaffe, and B. Jun, “Differential power analysis,” in Proceedings of the 19th Annual International Cryptology Conference on Advances in Cryptology, CRYPTO ’99, (London, UK, UK), pp. 388-397, Springer-Verlag, 1999.Google Scholar

  • [22] P. C. Kocher, “Timing attacks on implementations of diffiehellman, rsa, dss, and other systems,” in Proceedings of the 16th Annual International Cryptology Conference on Advances in Cryptology, CRYPTO ’96, (London, UK, UK), pp. 104-113, Springer-Verlag, 1996.Google Scholar

  • [23] D. A. Osvik, A. Shamir, and E. Tromer, “Cache attacks and countermeasures: The case of aes,” in Proceedings of the 2006 The Cryptographers’ Track at the RSA Conference on Topics in Cryptology, CT-RSA’06, (Berlin, Heidelberg), pp. 1-20, Springer-Verlag, 2006.Google Scholar

  • [24] D. J. Bernstein, “Cache-timing attacks on aes,” tech. rep., 2005.Google Scholar

  • [25] M. Vuagnoux and S. Pasini, “Compromising electromagnetic emanations of wired and wireless keyboards.,” in USENIX security symposium, pp. 1-16, 2009.Google Scholar

  • [26] L. Zhuang, F. Zhou, and J. D. Tygar, “Keyboard acoustic emanations revisited,” ACM Transactions on Information and System Security (TISSEC), vol. 13, no. 1, p. 3, 2009.Web of ScienceGoogle Scholar

  • [27] M. Backes, M. Dürmuth, S. Gerling, M. Pinkal, and C. Sporleder, “Acoustic side-channel attacks on printers.,” in USENIX Security Symposium, pp. 307-322, 2010.Google Scholar

  • [28] J. Mäntyjärvi, M. Lindholm, E. Vildjiounaite, S. marja Mäkelä, and H. Ailisto, “Identifying users of portable devices from gait pattern with accelerometers,” in in IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005.Google Scholar

  • [29] Y. Michalevsky, D. Boneh, and G. Nakibly, “Gyrophone: Recognizing speech from gyroscope signals,” in Proceedings of the 23rd USENIX Conference on Security Symposium, SEC’14, (Berkeley, CA, USA), pp. 1053-1067, USENIX Association, 2014.Google Scholar

  • [30] S. Nawaz and C. Mascolo, “Mining users’ significant driving routes with low-power sensors,” in Proceedings of the 12th ACM Conference on Embedded Network Sensor Systems, SenSys ’14, (New York, NY, USA), pp. 236-250, ACM, 2014.Google Scholar

  • [31] Z. Xu, K. Bai, and S. Zhu, “Taplogger: Inferring user inputs on smartphone touchscreens using on-board motion sensors,” in Proceedings of the fifth ACM conference on Security and Privacy in Wireless and Mobile Networks, pp. 113-124, ACM, 2012.Google Scholar

  • [32] L. Cai and H. Chen, “Touchlogger: Inferring keystrokes on touch screen from smartphone motion.,” in HotSec, 2011.Google Scholar

  • [33] E. Miluzzo, A. Varshavsky, S. Balakrishnan, and R. R. Choudhury, “Tapprints: your finger taps have fingerprints,” in Proceedings of the 10th international conference on Mobile systems, applications, and services, pp. 323-336, ACM, 2012.Google Scholar

  • [34] A. J. Aviv, B. Sapp, M. Blaze, and J. M. Smith, “Practicality of accelerometer side channels on smartphones,” in Proceedings of the 28th Annual Computer Security Applications Conference, pp. 41-50, ACM, 2012.Google Scholar

  • [35] S. Dey, N. Roy, W. Xu, R. R. Choudhury, and S. Nelakuditi, “Accelprint: Imperfections of accelerometers make smartphones trackable,” in Proceedings of the Network and Distributed System Security Symposium (NDSS), 2014.Google Scholar

  • [36] P. Marquardt, A. Verma, H. Carter, and P. Traynor, “(sp) iphone: decoding vibrations from nearby keyboards using mobile phone accelerometers,” in Proceedings of the 18th ACM conference on Computer and communications security, pp. 551-562, ACM, 2011.Google Scholar

  • [37] J. Cache, “Fingerprinting 802.11 Implementations via Statistical Analysis of the Duration Field,” tech. rep., 2006.Google Scholar

  • [38] “Nmap security scanner.” https://nmap.org/. Accessed: 2015-07-31.Google Scholar

  • [39] V. C. Perta, M. V. Barbera, and A. Mei, “Exploiting delay patterns for user ips identification in cellular networks,” in Privacy Enhancing Technologies, pp. 224-243, Springer, 2014.Google Scholar

  • [40] V. Brik, S. Banerjee, M. Gruteser, and S. Oh, “Wireless device identification with radiometric signatures,” in Proceedings of the 14th ACM international conference on Mobile computing and networking, pp. 116-127, ACM, 2008.Google Scholar

  • [41] T. Stöber, M. Frank, J. Schmitt, and I. Martinovic, “Who do you sync you are?: smartphone fingerprinting via application behaviour,” in Proceedings of the sixth ACM conference on Security and privacy in wireless and mobile networks, pp. 7-12, ACM, 2013.Google Scholar

  • [42] M. Conti, L. V. Mancini, R. Spolaor, and N. V. Verde, “Can’t you hear me knocking: Identification of user actions on android apps via traffic analysis,” in Proceedings of the 5th ACM Conference on Data and Application Security and Privacy, pp. 297-304, ACM, 2015.Google Scholar

  • [43] S. Chen, R. Wang, X. Wang, and K. Zhang, “Side-channel leaks in web applications: A reality today, a challenge tomorrow,” in Security and Privacy (SP), 2010 IEEE Symposium on, pp. 191-206, IEEE, 2010.Google Scholar

  • [44] S. Khattak, L. Simon, and S. J. Murdoch, “Systemization of pluggable transports for censorship resistance,” arXiv preprint arXiv:1412.7448, 2014.Google Scholar

  • [45] C. V. Wright, L. Ballard, S. E. Coull, F. Monrose, and G. M. Masson, “Spot me if you can: Uncovering spoken phrases in encrypted voip conversations,” in Security and Privacy, 2008. SP 2008. IEEE Symposium on, pp. 35-49, IEEE, 2008.Google Scholar

  • [46] A. M. White, A. R. Matthews, K. Z. Snow, and F. Monrose, “Phonotactic reconstruction of encrypted voip conversations: Hookt on fon-iks,” in Security and Privacy (SP), 2011 IEEE Symposium on, pp. 3-18, IEEE, 2011.Google Scholar

  • [47] S. Jana and V. Shmatikov, “Memento: Learning secrets from process footprints,” in Security and Privacy (SP), 2012 IEEE Symposium on, pp. 143-157, IEEE, 2012.Google Scholar

About the article

Published Online: 2016-05-06

Published in Print: 2016-07-01

Citation Information: Proceedings on Privacy Enhancing Technologies, Volume 2016, Issue 3, Pages 136–154, ISSN (Online) 2299-0984, DOI: https://doi.org/10.1515/popets-2016-0020.

Export Citation

© 2016. This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. BY-NC-ND 4.0

Comments (0)

Please log in or register to comment.
Log in