Jump to ContentJump to Main Navigation
Show Summary Details

Proceedings on Privacy Enhancing Technologies

4 Issues per year

Open Access
Online
ISSN
2299-0984
See all formats and pricing

Portrait of a Privacy Invasion

Detecting Relationships Through Large-scale Photo Analysis

Yan Shoshitaishvili
  • Corresponding author
  • UC Santa Barbara
  • Email:
/ Christopher Kruegel
  • UC Santa Barbara
  • Email:
/ Giovanni Vigna
  • UC Santa Barbara
  • Email:
Published Online: 2015-04-18 | DOI: https://doi.org/10.1515/popets-2015-0004

Abstract

The popularity of online social networks has changed the way in which we share personal thoughts, political views, and pictures. Pictures have a particularly important role in the privacy of users, as they can convey substantial information (e.g., a person was attending an event, or has met with another person). Moreover, because of the nature of social networks, it has become increasingly difficult to control who has access to which content. Therefore, when a substantial amount of pictures are accessible to one party, there is a very serious potential for violations of the privacy of users. In this paper, we demonstrate a novel technique that, given a large corpus of pictures shared on a social network, automatically determines who is dating whom, with reasonable precision. More specifically, our approach combines facial recognition, spatial analysis, and machine learning techniques to determine pairs that are dating. To the best of our knowledge, this is the first privacy attack of this kind performed on social networks. We implemented our approach in a tool, called Creepic, and evaluated it on two real-world datasets. The results show that it is possible to automatically extract non-obvious, and nondisclosed, relationships between people represented in a group of pictures, even when the people involved are not directly part of a connected social clique.

References

  • [1] A. Acquisti, R. Gross, and F. Stutzman. Faces of facebook: Privacy in the age of augmented reality. 2011. [Web of Science]

  • [2] APeWatch. App Permission Watcher. http://apewatch.de/.

  • [3] M. Balduzzi, C. Platzer, T. Holz, E. Kirda, D. Balzarotti, and C. Kruegel. Abusing social networks for automated user profiling. In Recent Advances in Intrusion Detection, pages 422-441. Springer, 2010.

  • [4] M. Bar-Hillel. The base-rate fallacy in probability judgments. Acta Psychologica, 44(3):211-233, 1980.

  • [5] P. N. Belhumeur, J. P. Hespanha, and D. J. Kriegman. Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 19(7):711-720, 1997.

  • [6] C.-C. Chang and C.-J. Lin. LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2:27:1-27:27, 2011. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm.

  • [7] D. J. Crandall, L. Backstrom, D. Cosley, S. Suri, D. Huttenlocher, and J. Kleinberg. Inferring social ties from geographic coincidences. Proceedings of the National Academy of Sciences, 107(52):22436-22441, 2010.

  • [8] C. A. Davis Jr, G. L. Pappa, D. R. R. de Oliveira, and F. de L Arcanjo. Inferring the location of twitter messages based on user relationships. Transactions in GIS, 15(6):735-751, 2011.

  • [9] Facebook. The Formulation of Love. https: //www.facebook.com/notes/facebook-data-science/theformation- of-love/10152064609253859.

  • [10] Facebook, Ericcson, and Qualcomm. A Focus on Efficiency. https://fbcdn-dragon-a.akamaihd.net/hphotos-akash3/ 851590_229753833859617_1129962605_n.pdf.

  • [11] GigaOM. Instagram reports 200M users. http://gigaom.com/2014/03/25/instagramreports-200m-users-20b-photos-shared/.

  • [12] P. Gundecha, G. Barbier, and H. Liu. Exploiting vulnerability to secure user privacy on a social networking site. In Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 511-519. ACM, 2011.

  • [13] T. Independent. How average Briton is caught on camera 300 times a day. http://www.independent.co.uk/news/uk/thisbritain/ how-average-briton-is\discretionary{-}{}{}caught-oncamera-300-times-a-day-572781.html.

  • [14] C. Jernigan and B. F. Mistree. Gaydar: Facebook friendships expose sexual orientation. First Monday, 14(10), 2009.

  • [15] J. K. Lars Backstrom. Romantic partnerships and the dispesion of social ties: A network analysis of relationship status on facebook. In Proceedings of the 2014 ACM conference on Computer supported cooperative work. ACM, 2014.

  • [16] B. W. Matthews. Comparison of the predicted and observed secondary structure of t4 phage lysozyme. Biochimica et Biophysica Acta (BBA)-Protein Structure, 405(2):442-451, 1975.

  • [17] S. Nagaraja, P. Schaffer, and D. Aouada. Who clicks there!: anonymising the photographer in a camera saturated society. In Proceedings of the 10th annual ACM workshop on Privacy in the electronic society, pages 13-22. ACM, 2011.

  • [18] OpenCV. OpenCV. http://opencv.org.

  • [19] E. Osuna, R. Freund, and F. Girosit. Training support vector machines: an application to face detection. In Computer Vision and Pattern Recognition, 1997. Proceedings., 1997 IEEE Computer Society Conference on, pages 130-136. IEEE, 1997.

  • [20] B. Rouberol. Comparison of face detection tools. http://bit.ly/1olWs4n.

  • [21] H. A. Rowley, S. Baluja, and T. Kanade. Neural networkbased face detection. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 20(1):23-38, 1998.

  • [22] M. Turk and A. Pentland. Eigenfaces for recognition. Journal of cognitive neuroscience, 3(1):71-86, 1991. [PubMed]

  • [23] T. Verge. The new Snapchat brilliantly mixes video and texting. http://www.theverge.com/2014/5/1/ 5670260/real-talk-the-new-snapchatmakes- texting-fun-again-video-calls.

  • [24] P. Wagner. Facerec. https://github.com/bytefish/facerec. [PubMed]

  • [25] P. I. Wilson and J. Fernandez. Facial feature detection using haar classifiers. Journal of Computing Sciences in Colleges, 21(4):127-133, 2006.

About the article

Received: 2014-11-22

Revised: 2015-02-16

Accepted: 2015-02-16

Published Online: 2015-04-18

Published in Print: 2015-04-01


Citation Information: Proceedings on Privacy Enhancing Technologies, ISSN (Online) 2299-0984, DOI: https://doi.org/10.1515/popets-2015-0004. Export Citation

© 2015. This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License. (CC BY-NC-ND 3.0)

Comments (0)

Please log in or register to comment.
Log in