Do Not Track Annotated Bibliography

« Back to DoNotTrack.Us

This page summarizes works related to online tracking, behavioral advertising, and regulatory approaches. Papers that are particularly relevant to ongoing Do Not Track policy discussions are marked recommended. We briefly discuss several studies, marked discussion.

Privacy · Regulation · Economics · Technology


When to Research Is to Reveal: The Growing Threat to Attorney and Client Confidentiality From Online Tracking
Virginia Journal of Law and Technology (2011)

Anne Klinefelter argues that online tracking threatens the attorney-client and work product privileges, as well as the legal profession's ethical duty of confidentiality.

Balancing Expectations of Online Privacy: Why Internet Protocol (IP) Addresses Should Be Protected as Personally Identifiable Information
Depaul Law Review (2011) (forthcoming)

Joshua J. McIntyre argues that Internet Protocol addresses should receive the same privacy law protections as physical addresses and Social Security numbers.

Why Privacy Discussions About Pervasive Online Customer Profiling Should Focus on the Expanding Roles of Third-Parties
International Journal of Private Law (2011)

Nancy J. King notes the privacy concerns unique to third-party behavioral advertising and reviews recent regulatory developments.

Cookie Confusion: Do Browser Interfaces Undermine Understanding?
Proceedings of the 28th ACM Conference on Human Factors in Computer Systems (2010)

Aleecia M. McDonald conducts in-depth interviews on online advertising and discovers significant confusion about how online advertising works, what privacy choices are available, and how current technical options in the browser function.


Americans' Attitudes About Internet Behavioral Advertising Practices
Workshop on Privacy in the Electronic Society (2010)

Aleecia M. McDonald and Lorrie Faith Cranor survey user preferences about online advertising and reveal significant user confusion about existing advertising opt-out tools.

Broken Promises of Privacy: Responding to the Surprising Failure of Anonymization
UCLA Law Review (2010)

Advances in re-identifying seemingly anonymous data lead Paul Ohm to advocate abandoning the legal concept of personally identifiable information (PII).

Timing Is Everything? The Effects of Timing and Placement of Online Privacy Indicators
Proceedings of the 27th ACM Conference on Human Factors in Computer Systems (2009)

Serge Egelman et al. show that the timing and location of privacy indicators can have a significant impact on outcomes.


Know Privacy
(unpublished) (2009)

Joshua Gomez et al. conduct a detailed survey of the state of third-party web tracking, circa 2009.


Americans Reject Tailored Advertising and Three Activities That Enable It
(unpublished) (2009)

A national survey conducted by Joseph Turow et al. shows the vast majority of Americans oppose behavioral advertising.


Targeting Bad Behavior: Why Federal Regulators Must Treat Online Behavioral Marketing as Spyware
Hastings Communications and Entertainment Law Journal (2009)

Heather Osborn Ng compares behavioral advertising to spyware, notes the possible harms, and proposes new federal legislation.

The Cost of Reading Privacy Policies
I/S: A Journal of Law and Policy for the Information Society (2008)

Aleecia M. McDonald and Lorrie Faith Cranor estimate reading privacy policies would take the average American 201 hours per year.



AdChoices? Compliance With Online Behavioral Advertising Notice and Choice Requirements
(unpublished) (2011)

Saranga Komanduri et al. document non-compliance and inconsistent opt-out commitments in online advertising self-regulation.

Privacy and Regulatory Innovation: Moving Beyond Voluntary Codes
I/S: A Journal of Law and Policy for the Information Society (2011) (forthcoming)

Ira S. Rubinstein reviews the history of self-regulation in online privacy and notes the advantages of statutory safe harbors over voluntary codes of conduct.

The Law and Policy of Online Privacy: Regulation, Self-Regulation, or Co-Regulation?
Seattle University Law Review (2011)

Dennis D. Hirsch reviews regulatory, self-regulatory, and co-regulatory approaches to online privacy.

Profiling the Mobile Customer - Is Industry Self-Regulation Adequate to Protect Consumer Privacy When Behavioural Advertisers Target Mobile Phones?
Computer Law and Security Review (2010)

Nancy J. King and Pernille Wegener Jessen detail how self-regulation of mobile behavioral advertising has failed to protect consumers.

The Network Advertising Initiative: Failing at Consumer Protection and Self-Regulation
(unpublished) (2007)

Pam Dixon documents the failed history of the Network Advertising Initiative, including its ineffective opt-out mechanism, its inability to police member violations, and how advertising networks departed once the FTC's scrutiny lifted.

A Coasean Analysis of Marketing
Wisconsin Law Review (2006)

Eric Goldman argues for technology that allows users to easily impose granular and changing advertising preferences.



Do Not Track: Revenue Impact on Online Advertising
(unpublished) (2011)

An analyst at Bloomberg Government conducts a back-of-the-envelope calculation of Do Not Track's economic impact on the online advertising industry.

Discussion The paper naively multiplies the proportion of users who claim they would opt out of behavioral targeting by the behavioral advertising market size. This approach glosses over many of the complexities of online advertising economics. A couple significant omissions: First, the paper assumes that spending on behavioral advertising is independent of spending on other forms of online advertising. Second, the paper does not situate behavioral advertising in context from an advertising-supported business's perspective—behavioral advertising accounts for a very small share of revenue.


Privacy Regulation and Online Advertising
Management Science (2011)

Avi Goldfarb and Catherine Tucker analyze the impact of the EU e-Privacy Directive on online advertising. They conclude that it greatly reduced online advertising's effectiveness.

Discussion This study rests on several questionable design choices, and we encourage readers to draw inferences from it with caution. First, it relies on a non-standard advertising performance metric; the study does not demonstrate its metric is consistent or correlated with an objective performance metric. The metric exhibits a number of oddities that undermine its validity. For example, after the EU e-Privacy Directive non-EU advertising is twice as effective on EU viewers as on non-EU viewers.

Second, the study assumes the EU e-Privacy Directive significantly limited ad targeting. To be sure, the e-Privacy Directive is replete with language criticizing the use of third-party cookies and other tracking technologies. In practice, however, it had little effect on web tracking; the EU is only now considering significant limits. The study does not explore the wide array of possible alternate causes for a change in EU advertising effectiveness, such as page layout or content.

Last, the study's empirical hypothesis does not reflect changes in the online advertising market. Behavioral advertising was relatively non-existent in 2001 and a very small share of online advertising in 2008. In the same time period significant advances were made in contextual and demographic ad targeting. Assuming EU law did negatively affect behavioral advertising, we should expect to see across-the-board performance for both EU and non-EU ads rise, with a slightly greater rise in non-EU performance. Instead, the authors predict and demonstrate a decrease in EU performance and constant non-EU performance.


Economic Impact of Privacy on Online Behavioral Advertising
(unpublished) (2010)

The Ponemon Institute, a for-profit privacy research and consulting group, conduct a survey of online advertisers. They find that advertisers are not investing in behavioral advertising owing to privacy concerns.

Discussion This survey appears to have been conducted with an informal methodology; participants were not representative of the advertising market, and interviewers did not follow a set script. A number of the questions are leading. Several questions ask for a quantitative measure; the paper does not indicate participants were given an opportunity to consult internal data. Many questions rely on overloaded, undefined terms, such as "performance," "cost change," and "improvement" relative to other forms of advertising. In the one analytical section of the paper, the authors assert that advertisers' behavioral advertising budget is independent of other online ad budgets. As justification the authors only note that "[t]his assumption was validated with a subsample of companies."


The Value of Behavioral Targeting
(unpublished) (2009)

Howard Beales compares the pricing and effectiveness of behavioral advertising to untargeted advertising.

Discussion The reader is advised to take note of this study's background: it was industry sponsored and not peer reviewed. There are three particularly suspect design features. First, the study relies on a very small, unrepresentative sample of advertising networks. For some statistics it relies on data from fewer than five companies.

Second, the study compares behavioral advertising to "run of network" advertising—bargain bin untargeted ads placed across an entire ad network. The relevant comparison for Do Not Track policy purposes is between behavioral targeting and the next-best non-tracking alternatives (contextual, demographic, and geographic targeting).

To clarify the point, here's an analogy: Suppose you're considering a regulation that could impact Super Bowl advertising. This study's approach would compare Super Bowl ads to public access cable ads, and discover the former are more expensive but more effective. The more relevant comparison is between Super Bowl ads and other primetime ads.

Third, the study suggests that behavioral advertising's value should be assessed by whether it is both more effective and costs more. The interplay between performance, cost, and advertising revenue is more subtle. Advertisers face a cost-benefit tradeoff in choosing where to allocate their budgets; they'll allocate ad dollars to the best advertising deals—forms of advertising that cost less and do more. If behavioral advertising is more effective but more expensive than alternatives, it might not be much of a draw.

Taking up the TV advertising analogy: Super Bowl spots are significantly more expensive and more effective than other TV ads. In assessing whether to invest in a Super Bowl ad, an advertiser weighs cost and effectiveness against each other.


How Much Can Behavioral Targeting Help Online Advertising?
Proceedings of the 18th International World Wide Web Conference (2009)

Jun Yan et al. validate the assumptions behind behavioral advertising that users who click an ad are similar, and similar users can be clustered to improve advertisement targeting. The study also provides support for the conclusion that short-term data is generally better for targeting than long-term data, and user query data is generally better for targeting than URLs visited.

Discussion This study provides persuasive evidence that behavioral targeting could improve search engine advertisement performance. The reader should note that this study is intended to provide theoretical limits on how idealized behavioral targeting could perform. It is not intended to assess the business of behavioral advertising, which involves a number of practical considerations and limitations beyond the scope of the paper. The study is also limited to search engine advertising, not the display advertising that is relevant to Do Not Track policy discussions.


The Online Advertising Industry: Economics, Evolution, and Privacy
Journal of Economic Perspectives (2009)

David S. Evans sketches the economics of the online advertising industry circa 2009. He notes that behavioral advertising accounts for "a small portion of the advertising revenue earned by publishers."

A Coasean Analysis of Marketing
Wisconsin Law Review (2006)

Eric Goldman argues for technology that allows users to easily impose granular and changing advertising preferences.


ObliviAd: Provably Secure and Practical Online Behavioral Advertising
IEEE Security and Privacy (2012)

ObliviAd takes a somewhat unusual approach to targeting without tracking. The user profile is maintained in the browser, which is sent in encrypted form to the ad server when an ad needs to be served. The server may use an arbitrary algorithm to select an ad, but this is executed in in a piece of trusted hardware which is certified via remote attestation. The selected ad is downloaded by the client via Private Information Retrieval (PIR). Thus the server never sees the plaintext profile. A system of electronic tokens similar to e-cash is used for anonymously billing the right advertiser.

The barriers to adoption of ObliviAd are the need for ad companies to shift to a new infrastructure involving trusted computing, the cost and performance of PIR, and the security challenges of isolated code execution.

Auctions in Do-Not-Track Compliant Internet Advertising
ACM Conference on Computer and Communications Security (2011)

In an extension to Privad, Reznichenko et al. evaluate designs for privacy-preserving advertising auctions. The work emphasizes the trade-off between an advertising company’s ability to conceal its ranking algorithm and bids and a user’s ability to prevent pseudonymous tracking.

Targeted, Not Tracked: Client-side Solutions for Privacy-Friendly Behavioral Advertising
4th Workshop on Hot Topics in Privacy Enhancing Technologies — HotPETs (2011)

Bilenko and Richardson propose an approach for keyword-based search advertising that provides privacy against a weaker threat model. The search advertising company is trusted to temporarily compute on user profile data, but then store the data in the browser and delete its copy. The authors ran their algorithm against 60 days of Bing search advertising logs and achieved almost all the benefit of current server-side behavioral targeting. Specifically, they report capturing over 95% of the increase in click-through rates, generating approximately 4% greater revenue than search advertising without behavioral targeting. We are skeptical that the temporary data-use model is likely to be adopted; web services in general, and online advertising companies in particular, have historically been loath to voluntarily discard logs. The model also introduces the risk of inadvertent or surreptitious collection of third-party tracking data.

Towards Street-Level Client-Independent IP Geolocation
Proceedings of the 8th USENIX Symposium on Networked Systems Design and Implementation (2011)

Yong Wang et al. develop an IP address geolocation technique that has a median accuracy of less than a half mile.

Privad: Practical Privacy in Online Advertising
Proceedings of the 8th USENIX Symposium on Networked Systems Design and Implementation (2011)

Saikat Guha et al. propose a model for privacy-preserving interest-targeted advertising. Privad is designed to conceal a user’s activities from an advertising network by interposing an anonymizing proxy between the browser and the ad network. In this approach, trusted client software subscribes to streams of possibly relevant ads, selects relevant ads locally, submits candidates for auction, and then reports results. While the Privad model is designed to offer comprehensive privacy guarantees, it requires broad adoption of high-performance anonymizing proxies. This seems unlikely in the near future.

RePriv: Re-Imagining Content Personalization and In-Browser Privacy
Proceedings of the IEEE Symposium on Security and Privacy (2011) (forthcoming)

RePriv by Matthew Fredrikson and Ben Livshits is a verifiable policy architecture that enables users to selectively grant permission for generating and sharing client-side data stores that enable website personalization. The RePriv model holds promise as a general-purpose platform for building privacypreserving advertising like Privad and Adnostic. But, like Adnostic, RePriv would have to be translated from its current implementation as a single-platform browser extension into existing web technologies for near-term deployment

An Empirical Study of Privacy-Violating Information Flows in JavaScript Web Applications
Proceedings of the 17th ACM Conference on Computer and Communications Security (2010)

Dongseok Jang et al. provide an architecture for detecting privacy violations in JavaScript and apply their approach to popular websites.

Challenges in Measuring Online Advertising Systems
Proceedings of the 10th Annual Conference on Internet Measurement (2010)

Saikat Guha et al. conduct a preliminary study of ad targeting methodologies by comparing the ads displayed to different users.

AdNostic: Privacy Preserving Targeted Advertising
Proceedings of the 17th Annual Network and Distributed System Security Symposium (2010)

Vincent Toubiana et al. present an architecture for interest-targeted advertising that does not require tracking. Like Privad, Adnostic uses client-based functionality to perform ad selection, but it eliminates anonymizing proxies at the cost of less precise ad targeting. Adnostic also simplifies cost-per-click billing by allowing the advertising network to learn of a user’s ad clicks. Cost-per-impression billing would still require a low-performance trusted intermediary so as to not reveal the user’s ad impressions. As implemented, Adnostic requires a browser extension, which is a practical barrier to more widespread adoption

Robust Defenses for Cross-Site Request Forgery
15th ACM Conference on Computer and Communications Security (2008)

Collin Jackson et al. find that custom headers reach a server in approximately 99.9% of HTTP requests.