J. R. Mayer and J. C. Mitchell, “Third-party web tracking: Policy and technology,” in IEEE Symposium on Security and Privacy, 2012, pp. 413-427.
 B. Ur, P. G. Leon, L. F. Cranor, R. Shay, and Y. Wang, “Smart, useful, scary, creepy: Perceptions of online behavioral advertising,” in Proceedings of the Eighth Symposium on Usable Privacy and Security. ACM, 2012, pp. 4:1-4:15.
 Google, “About ads settings,” https://support.google.com/ ads/answer/2662856, accessed Nov. 21, 2014.
 Yahoo!, “Ad interest manager,” https://info.yahoo.com/ privacy/us/yahoo/opt_out/targeting/details.html, accessed Nov. 21, 2014.
 Microsoft, “Microsoft personalized ad preferences,” http: //choice.microsoft.com/en-us/opt-out, accessed Nov. 21, 2014.
 Executive Office of the President, “Big data: Seizing opportunities, preserving values,” Posted at http://www. whitehouse.gov/sites/default/files/docs/big_data_privacy_ report_may_1_2014.pdf, 2014, accessed Jan. 26, 2014.
 R. Zemel, Y. Wu, K. Swersky, T. Pitassi, and C. Dwork, “Learning fair representations,” in Proceedings of the 30th International Conference on Machine Learning (ICML-13), S. Dasgupta and D. Mcallester, Eds., vol. 28. JMLR Workshop and Conference Proceedings, May 2013, pp. 325-333. [Online]. Available: http: //jmlr.org/proceedings/papers/v28/zemel13.pdf
 F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay, “Scikit-learn: Machine learning in Python,” Journal of Machine Learning Research, vol. 12, pp. 2825-2830, 2011.
 E. Jones, T. Oliphant, P. Peterson et al., “SciPy: Open source scientific tools for Python,” 2001, http://www.scipy.org/.
 M. C. Tschantz, A. Datta, A. Datta, and J. M. Wing, “A methodology for information flow experiments,” ArXiv, Tech. Rep. arXiv:1405.2376v1, 2014.
 P. Good, Permutation, Parametric and Bootstrap Tests of Hypotheses. Springer, 2005.
 C. E. Wills and C. Tatar, “Understanding what they do with what they know,” in Proceedings of the 2012 ACM Workshop on Privacy in the Electronic Society, 2012, pp. 13-18.
 P. Barford, I. Canadi, D. Krushevskaja, Q. Ma, and S. Muthukrishnan, “Adscape: Harvesting and analyzing online display ads,” in Proceedings of the 23rd International Conference on World Wide Web. International World Wide Web Conferences Steering Committee, 2014, pp. 597-608.
 B. Liu, A. Sheth, U. Weinsberg, J. Chandrashekar, and R. Govindan, “AdReveal: Improving transparency into online targeted advertising,” in Proceedings of the Twelfth ACM Workshop on Hot Topics in Networks. ACM, 2013, pp. 12:1-12:7.
 M. Lécuyer, G. Ducoffe, F. Lan, A. Papancea, T. Petsios, R. Spahn, A. Chaintreau, and R. Geambasu, “XRay: Increasing the web’s transparency with differential correlation,” in Proceedings of the USENIX Security Symposium, 2014.
 S. Englehardt, C. Eubank, P. Zimmerman, D. Reisman, and A. Narayanan, “Web privacy measurement: Scientific principles, engineering platform, and new results,” Manuscript posted at http://randomwalker.info/ publications/WebPrivacyMeasurement.pdf, 2014, accessed Nov. 22, 2014.
 S. Guha, B. Cheng, and P. Francis, “Challenges in measuring online advertising systems,” in Proceedings of the 10th ACM SIGCOMM Conference on Internet Measurement, 2010, pp. 81-87.
 R. Balebako, P. Leon, R. Shay, B. Ur, Y. Wang, and L. Cranor, “Measuring the effectiveness of privacy tools for limiting behavioral advertising,” in Web 2.0 Security and Privacy Workshop, 2012.
 L. Sweeney, “Discrimination in online ad delivery,” Commun. ACM, vol. 56, no. 5, pp. 44-54, 2013.
 R. A. Fisher, The Design of Experiments. Oliver & Boyd, 1935.
 S. Greenland and J. M. Robins, “Identifiability, exchangeability, and epidemiological confounding,” International Journal of Epidemiology, vol. 15, no. 3, pp. 413-419, 1986.
 T. M. Mitchell, Machine Learning. McGraw-Hill, 1997.
 D. D. Jensen, “Induction with randomization testing: Decision-oriented analysis of large data sets,” Ph.D. dissertation, Sever Institute of Washington University, 1992.
 Alexa, “Is popularity in the top sites by category directory based on traffic rank?” https://support.alexa.com/hc/enus/ articles/200461970, accessed Nov. 21, 2014.
 C. M. Bishop, Pattern Recognition and Machine Learning. Springer, 2006.
 S. Holm, “A simple sequentially rejective multiple test procedure,” Scandinavian Journal of Statistics, vol. 6, no. 2, pp. 65-70, 1979.
 Google, “Google privacy and terms,” http://www.google. com/policies/technologies/ads/, accessed Nov. 22, 2014.
 H. Abdi, “Bonferroni and Šidák corrections for multiple comparisons,” in Encyclopedia of Measurement and Statistics, N. J. Salkind, Ed. Sage, 2007.
 D. Hume, A Treatise of Human Nature: Being an Attempt to Introduce the Experimental Method of Reasoning into Moral Subjects, 1738, book III, part I, section I.
 Pew Research Center’s Social and Demographic Trends Project, “On pay gap, millennial women near parity - for now: Despite gains, many see roadblocks ahead,” 2013.
 T. Z. Zarsky, “Understanding discrimination in the scored society,” Washington Law Review, vol. 89, pp. 1375-1412, 2014.
 R. S. Zemel, Y. Wu, K. Swersky, T. Pitassi, and C. Dwork, “Learning fair representations,” in Proceedings of the 30th International Conference on Machine Learning, ser. JMLR: W&CP, vol. 28. JMLR.org, 2013, pp. 325-333.
 Adgooroo, “Adwords cost per click rises 26% between 2012 and 2014,” http://www.adgooroo.com/resources/blog/ adwords-cost-per-click-rises-26-between-2012-and-2014/, accessed Nov. 21, 2014.
 L. Olejnik, T. Minh-Dung, and C. Castelluccia, “Selling off privacy at auction,” in Network and Distributed System Security Symposium (NDSS). The Internet Society, 2013.
 C. J. Clopper and E. S. Pearson, “The use of confidence or fiducial limits illustrated in the case of the binomial,” Biometrika, vol. 26, no. 4, pp. 404-413, 1934.
Published Online: 2015-04-18
Published in Print: 2015-04-01
Citation Information: Proceedings on Privacy Enhancing Technologies. Volume 2015, Issue 1, Pages 92–112, ISSN (Online) 2299-0984, DOI: https://doi.org/10.1515/popets-2015-0007, April 2015
© 2015. This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License. (CC BY-NC-ND 3.0)