Jump to ContentJump to Main Navigation
Show Summary Details
More options …

Proceedings on Privacy Enhancing Technologies

4 Issues per year

Open Access
See all formats and pricing
More options …

The Right to be Forgotten in the Media: A Data-Driven Study

Minhui Xue / Gabriel Magno / Evandro Cunha / Virgilio Almeida / Keith W. Ross
Published Online: 2016-07-14 | DOI: https://doi.org/10.1515/popets-2016-0046


Due to the recent “Right to be Forgotten” (RTBF) ruling, for queries about an individual, Google and other search engines now delist links to web pages that contain “inadequate, irrelevant or no longer relevant, or excessive” information about that individual. In this paper we take a data-driven approach to study the RTBF in the traditional media outlets, its consequences, and its susceptibility to inference attacks. First, we do a content analysis on 283 known delisted UK media pages, using both manual investigation and Latent Dirichlet Allocation (LDA). We find that the strongest topic themes are violent crime, road accidents, drugs, murder, prostitution, financial misconduct, and sexual assault. Informed by this content analysis, we then show how a third party can discover delisted URLs along with the requesters’ names, thereby putting the efficacy of the RTBF for delisted media links in question. As a proof of concept, we perform an experiment that discovers two previously-unknown delisted URLs and their corresponding requesters. We also determine 80 requesters for the 283 known delisted media pages, and examine whether they suffer from the “Streisand effect,” a phenomenon whereby an attempt to hide a piece of information has the unintended consequence of publicizing the information more widely. To measure the presence (or lack of presence) of a Streisand effect, we develop novel metrics and methodology based on Google Trends and Twitter data. Finally, we carry out a demographic analysis of the 80 known requesters. We hope the results and observations in this paper can inform lawmakers as they refine RTBF laws in the future.

Keywords: Privacy; Right to be Forgotten; Streisand effect; Latent Dirichlet Allocation


  • [1] M. L. Ambrose. Speaking of forgetting: Analysis of possible non-EU responses to the right to be forgotten and speech exception. Telecommunications Policy, 38(8-9):800-811, Sept. 2014.Google Scholar

  • [2] M. L. Ambrose and J. Ausloos. The right to be forgotten across the pond. Journal of Information Policy, 3, 2013.Google Scholar

  • [3] J. Ausloos. The “Right to be Forgotten” - Worth remembering? Computer Law & Security Review, 28(2):143-152, 2012.Google Scholar

  • [4] BBC. List of BBC web pages which have been removed from Google’s search results. http://www.bbc.co.uk/blogs/internet/entries/1d765aa8-600b-4f32-b110-d02fbf7fd379, 25 June, 2015.Google Scholar

  • [5] BBC. BBC forgotten list “sets precedent”. http://www.bbc.com/news/technology-33287758, 26 June, 2015.Google Scholar

  • [6] Bert-Jaap Koops. Forgetting footprints, shunning shadows: A critical analysis of the “Right to be Forgotten” in big data practice. SCRIPTed, 8(3):229-256, Dec. 2011.Google Scholar

  • [7] D. M. Blei, A. Y. Ng, and M. I. Jordan. Latent Dirichlet Allocation. the Journal of machine Learning research, 3:993-1022, 2003.Google Scholar

  • [8] R. L. Bolton. The right to be forgotten: Forced amnesia in a technological age. John Marshall Journal of Information Technology & Privacy Law, 31(2):133-144, 2015.Google Scholar

  • [9] M. Cacciottolo. The Streisand effect: When censorship backfires. http://www.bbc.com/news/uk-18458567, 2012.Google Scholar

  • [10] CNIL. CNIL orders Google to apply delisting on all domain names of the search engine. http://www.cnil.fr/english/newsand-events/news/article/cnil-orders-google-to-applydelisting-on-all-domain-names-of-the-search-engine/, 2015.Google Scholar

  • [11] R. Dey, Y. Ding, and K. W. Ross. Profiling high-school students with facebook: how online privacy laws can actually increase minors’ risk. In Proceedings of the 2013 conference on Internet measurement conference, pages 405-416. ACM, 2013.Google Scholar

  • [12] E. P. Goodman. Open letter to Google from 80 Internet scholars: Release RTBF compliance data. https://medium.com/@ellgood/open-letter-to-google-from-80-internetscholars-release-rtbf-compliance-data-cbfc6d59f1bd#.at81h9i60, 2015.Google Scholar

  • [13] Google FAQ. Google frequently asked questions: European privacy in search. https://www.google.com/transparencyreport/removals/europeprivacy/faq, 2016.Google Scholar

  • [14] Google Transparency Report. Google transparency report: European privacy requests for search removals. https://www.google.com/transparencyreport/removals/europeprivacy, 2016.Google Scholar

  • [15] Hidden From Google. http://hiddenfromgoogle.com/, 2014.Google Scholar

  • [16] B.-C. Kim and J. Y. Kim. The Economics of the right to be forgotten. NET Institute Working Paper, 2015.Google Scholar

  • [17] E. Lee. The right to be forgotten v. free speech. Free Speech (August 26, 2015). Chicago-Kent College of Law Research Paper Forthcoming, 2015.Google Scholar

  • [18] H. J. Lee, J. H. Yun, H. S. Yoon, and K. H. Lee. The right to be forgotten: Standard on deleting the exposed personal information on the Internet. In J. J. Park, I. Stojmenovic, H. Y. Jeong, and G. Yi, editors, Computer science and its applications, pages 883-889. Springer, 2015.Google Scholar

  • [19] V. Mayer-Schönberger. Delete: the virtue of forgetting in the digital age. Princeton University Press, 2011.Google Scholar

  • [20] M. Mondal, B. Viswanath, A. Clement, P. Druschel, K. P. Gummadi, A. Mislove, and A. Post. Defending against largescale crawls in online social networks. In Proceedings of the 8th international conference on Emerging networking experiments and technologies, pages 325-336. ACM, 2012.Google Scholar

  • [21] A. L. Newman. What the “Right to be Forgotten” means for privacy in a digital age. Science, 347(6221):507-508, 2015.Google Scholar

  • [22] M. L. Rustad and S. Kulevska. Reconceptualizing the right to be forgotten to enable transatlantic data flow. Harvard Journal of Law and Technology, 28:349, 2015.Google Scholar

  • [23] C. Tang, K. Ross, N. Saxena, and R. Chen. What’s in a name: a study of names, gender inference, and gender behavior in facebook. In Database Systems for Adanced Applications, pages 344-356. Springer, 2011.Google Scholar

  • [24] The Guardian. Google to extend “Right to be Forgotten” to all its domains accessed in EU. http://www.theguardian.com/technology/2016/feb/11/google-extend-right-to-be-forgottengooglecom?CMP=twt_a-technology_b-gdntech, 2016.Google Scholar

  • [25] The New York Times. Google will further block some European search results. http://www.nytimes.com/2016/02/12/technology/google-will-further-block-some-europeansearch-results.html, 2016.Google Scholar

  • [26] The New Yorker. The solace of oblivion. http://www.newyorker.com/magazine/2014/09/29/solace-oblivion, 2014.Google Scholar

  • [27] S. Tippmann and S. Pamiés. Google’s data on the right to be forgotten. http://sytpp.github.io/rtbf/index.html, 2015.Google Scholar

  • [28] A. Tsesis. The right to erasure: Privacy, data brokers, and the indefinite retention of data. Wake Forest Law Review, 433(49), 2014.Google Scholar

  • [29] R. H. Weber. The right to be forgotten: More than a Pandora’s box? Journal of Intellectual Property, Information Technology and E-commerce Law, 2:120-130, 2011.Google Scholar

About the article

Received: 2016-02-29

Revised: 2016-06-02

Accepted: 2016-06-02

Published Online: 2016-07-14

Published in Print: 2016-10-01

Citation Information: Proceedings on Privacy Enhancing Technologies, Volume 2016, Issue 4, Pages 389–402, ISSN (Online) 2299-0984, DOI: https://doi.org/10.1515/popets-2016-0046.

Export Citation

© 2016. This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. BY-NC-ND 4.0

Comments (0)

Please log in or register to comment.
Log in