Jump to ContentJump to Main Navigation
Show Summary Details
More options …

Proceedings on Privacy Enhancing Technologies

4 Issues per year

Open Access
See all formats and pricing
More options …

Portrait of a Privacy Invasion

Detecting Relationships Through Large-scale Photo Analysis

Yan Shoshitaishvili / Christopher Kruegel / Giovanni Vigna
Published Online: 2015-04-18 | DOI: https://doi.org/10.1515/popets-2015-0004


The popularity of online social networks has changed the way in which we share personal thoughts, political views, and pictures. Pictures have a particularly important role in the privacy of users, as they can convey substantial information (e.g., a person was attending an event, or has met with another person). Moreover, because of the nature of social networks, it has become increasingly difficult to control who has access to which content. Therefore, when a substantial amount of pictures are accessible to one party, there is a very serious potential for violations of the privacy of users. In this paper, we demonstrate a novel technique that, given a large corpus of pictures shared on a social network, automatically determines who is dating whom, with reasonable precision. More specifically, our approach combines facial recognition, spatial analysis, and machine learning techniques to determine pairs that are dating. To the best of our knowledge, this is the first privacy attack of this kind performed on social networks. We implemented our approach in a tool, called Creepic, and evaluated it on two real-world datasets. The results show that it is possible to automatically extract non-obvious, and nondisclosed, relationships between people represented in a group of pictures, even when the people involved are not directly part of a connected social clique.


  • [1] A. Acquisti, R. Gross, and F. Stutzman. Faces of facebook: Privacy in the age of augmented reality. 2011.Web of ScienceGoogle Scholar

  • [2] APeWatch. App Permission Watcher. http://apewatch.de/.Google Scholar

  • [3] M. Balduzzi, C. Platzer, T. Holz, E. Kirda, D. Balzarotti, and C. Kruegel. Abusing social networks for automated user profiling. In Recent Advances in Intrusion Detection, pages 422-441. Springer, 2010.Google Scholar

  • [4] M. Bar-Hillel. The base-rate fallacy in probability judgments. Acta Psychologica, 44(3):211-233, 1980.Google Scholar

  • [5] P. N. Belhumeur, J. P. Hespanha, and D. J. Kriegman. Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 19(7):711-720, 1997.Google Scholar

  • [6] C.-C. Chang and C.-J. Lin. LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2:27:1-27:27, 2011. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm.Google Scholar

  • [7] D. J. Crandall, L. Backstrom, D. Cosley, S. Suri, D. Huttenlocher, and J. Kleinberg. Inferring social ties from geographic coincidences. Proceedings of the National Academy of Sciences, 107(52):22436-22441, 2010.Google Scholar

  • [8] C. A. Davis Jr, G. L. Pappa, D. R. R. de Oliveira, and F. de L Arcanjo. Inferring the location of twitter messages based on user relationships. Transactions in GIS, 15(6):735-751, 2011.Google Scholar

  • [9] Facebook. The Formulation of Love. https: //www.facebook.com/notes/facebook-data-science/theformation- of-love/10152064609253859.Google Scholar

  • [10] Facebook, Ericcson, and Qualcomm. A Focus on Efficiency. https://fbcdn-dragon-a.akamaihd.net/hphotos-akash3/ 851590_229753833859617_1129962605_n.pdf.Google Scholar

  • [11] GigaOM. Instagram reports 200M users. http://gigaom.com/2014/03/25/instagramreports-200m-users-20b-photos-shared/.Google Scholar

  • [12] P. Gundecha, G. Barbier, and H. Liu. Exploiting vulnerability to secure user privacy on a social networking site. In Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 511-519. ACM, 2011.Google Scholar

  • [13] T. Independent. How average Briton is caught on camera 300 times a day. http://www.independent.co.uk/news/uk/thisbritain/ how-average-briton-is\discretionary{-}{}{}caught-oncamera-300-times-a-day-572781.html.Google Scholar

  • [14] C. Jernigan and B. F. Mistree. Gaydar: Facebook friendships expose sexual orientation. First Monday, 14(10), 2009.Google Scholar

  • [15] J. K. Lars Backstrom. Romantic partnerships and the dispesion of social ties: A network analysis of relationship status on facebook. In Proceedings of the 2014 ACM conference on Computer supported cooperative work. ACM, 2014.Google Scholar

  • [16] B. W. Matthews. Comparison of the predicted and observed secondary structure of t4 phage lysozyme. Biochimica et Biophysica Acta (BBA)-Protein Structure, 405(2):442-451, 1975.Google Scholar

  • [17] S. Nagaraja, P. Schaffer, and D. Aouada. Who clicks there!: anonymising the photographer in a camera saturated society. In Proceedings of the 10th annual ACM workshop on Privacy in the electronic society, pages 13-22. ACM, 2011.Google Scholar

  • [18] OpenCV. OpenCV. http://opencv.org.Google Scholar

  • [19] E. Osuna, R. Freund, and F. Girosit. Training support vector machines: an application to face detection. In Computer Vision and Pattern Recognition, 1997. Proceedings., 1997 IEEE Computer Society Conference on, pages 130-136. IEEE, 1997.Google Scholar

  • [20] B. Rouberol. Comparison of face detection tools. http://bit.ly/1olWs4n.Google Scholar

  • [21] H. A. Rowley, S. Baluja, and T. Kanade. Neural networkbased face detection. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 20(1):23-38, 1998.Google Scholar

  • [22] M. Turk and A. Pentland. Eigenfaces for recognition. Journal of cognitive neuroscience, 3(1):71-86, 1991.PubMedGoogle Scholar

  • [23] T. Verge. The new Snapchat brilliantly mixes video and texting. http://www.theverge.com/2014/5/1/ 5670260/real-talk-the-new-snapchatmakes- texting-fun-again-video-calls.Google Scholar

  • [24] P. Wagner. Facerec. https://github.com/bytefish/facerec.PubMedGoogle Scholar

  • [25] P. I. Wilson and J. Fernandez. Facial feature detection using haar classifiers. Journal of Computing Sciences in Colleges, 21(4):127-133, 2006.Google Scholar

About the article

Received: 2014-11-22

Revised: 2015-02-16

Accepted: 2015-02-16

Published Online: 2015-04-18

Published in Print: 2015-04-01

Citation Information: Proceedings on Privacy Enhancing Technologies, Volume 2015, Issue 1, Pages 41–60, ISSN (Online) 2299-0984, DOI: https://doi.org/10.1515/popets-2015-0004.

Export Citation

© 2015. This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License. BY-NC-ND 3.0

Comments (0)

Please log in or register to comment.
Log in