Dynamic Creative Optimization – Where Online Data Meets Advertising Creative

Dynamic Creative Optimization Basics

Dynamic creative optimization is a relatively new technology to the online advertising space, but it is sure to see tremendous growth this year.  Why?  Because dynamic creative optimization allows marketers to put the right message in front of the right consumer.  It allows them to differentiate and custom-tailor ad creative to people using data, and it is extremely effective at increasing a campaign’s performance, almost to the point of suspicion.  Many case studies show an increase to CTR well above 100% vs. non-personalized creative, and decreases in cost metrics of 60 – 70%.  Combine these two metrics and the impact to ROI is usually eye-popping.

As we’ve written before, thanks to data marketplaces and real-time bidding through ad exchanges, marketers now have very powerful tools to reach a very precise audience on the web thanks to an explosion in consumer data.  Literally hundreds of companies are in the business of qualifying people into anonymous buying segments and reselling that intent data to marketers, who then use it to target those users with ads across the web.  So technology has allowed marketers to reach the right person at the right time with fairly impressive scale and pricing efficiency.  Powerful stuff to be sure, but what about the right message?  For example, if you knew about a single, 25 year old male in California who was in the market to buy a car, the last thing you want to show him is an ad for a minivan that performs well in the snow.  He’s probably much more interested in a hybrid car that can store a surfboard on its roof.  That’s a comically extreme example, but the point is that regardless of the targeting, marketers still need compelling creative. And dynamic creative optimization is where data meets the creative message.  It’s difficult to understate how exciting this technology is – placing the right ad in front of the right person and the right time is practically a holy grail for marketers and it looks as though the online advertising industry has figured out how to do it, or at least invented the tools to do it. (more…)

Tracking Billable Impressions and 3rd Party Discrepancies with Ad-Juster

The Problem with 3rd Party Discrepancies

It’s a sad fact that after more than a decade of innovation and growth in the digital display business, virtually nothing has been done to address the cost of 3rd party discrepancies on the industry.  As I detailed in my post on how 3rd party ad serving works, because Publisher ad servers and Marketer ad servers count an impression at a different point in the technical process, there is always a variance in the numbers, and reconciling those figures to cut an invoice is a manual, time-consuming process, and a huge administrative cost on the industry.  Discrepancies are typically around 10%, but can often exceed this, especially if there is a technical problem with the ad.  In virtually all cases however, publishers simply have to accept losses due to discrepancies as the cost of doing business.

Third party ad servers have never made it easy to address this issue.  Their publisher reporting tools are woefully inadequate and in some cases comically inefficient.  For example, the leading ad server, DoubleClick’s DART product, does not provide site level reports for publishers that allows them to see everything running on their site from that ad server, but only allows publishers to get reports advertiser-by-advertiser.  That means billing departments on every major online publisher spend days pulling hundreds of reports every month out of Dart alone. That means for most operations folks, a centralized reporting database that maps 3rd party delivery to local ad server delivery at the creative or flight level and updates automatically is practically a holy grail.

The Industry’s Response: An Impression Exchange

The IAB has recommended their own solution to address the matter via the Impression Exchange project, but I find the project fundamentally flawed.  For one, the technical process the IAB uses to centralize impression reporting between systems adds another call in the ad serving process and so creates a discrepancy on the discrepancy it reports.  Additionally, it has been very slow to win adoption by the ad servers – a year and a half in and DART is the only ad server currently on board.

Ad-Juster, The Superior Solution

A far better solution is for publishers to look at a company called Ad-Juster, which has created a way to centralize third party reports and map third party delivery against their internal flights down to the creative level.  Ad-Juster has essentially mapped the schema for every ad server reporting system, figured out how to pull large data dumps from every major third party ad server on a regular basis and map it with a unique identifier back to the third party tag running on a publisher’s local ad server.  In other words, it allows them to create a unified database across lots of systems. While the system is just a read-only version of the reporting you can get yourself, the speed and automation it brings to the table is very compelling for any large publisher.

Ad-Juster offers some canned reports that actually calculate the discrepancy between systems as well as some helpful filters that automatically email you a discrepancy report on flights that launched in the last three or five days for example, which allows operations staff to quickly catch implementation or technical issues.  Since you can monitor the entire network on a regular basis, it is easy to adjust the padding that most publishers add to client goals to makeup for an expected discrepancy.  You may very well find that some third parties track closer than others, so you can reduce the padding for those campaigns.  The reports are a boon to operations folks, but also extremely useful for billing departments.  Now the billing staff doesn’t have to spend all their time waiting in an ancient publisher-facing ad server UIs, they can push bills to clients faster.

The system isn’t perfect, especially if you run the same third party tags in multiple flights (the system can have a hard time attributing the right amount of third party impressions at the flight level in that case), but in most cases it offers tremendous benefits.  Recently Ad-Juster has partnered with Solbright and Fattail to push their data into those workflow systems, but they also offer an API for clients that want to push the data to proprietary systems.

Highly recommended for large publishers seeking reporting relief.

How RTB Ad Serving Works

Diagram of the Ad Exchange Redirect Process

I wrote about the basics of how DSPs, SSPs, and Ad Exchanges work from a high level and the value they add to the marketplace in my last post – for this piece we’re going to dig into the nitty gritty of how RTB ad serving works from a technical perspective.

In my explanation of third-party ad serving, I outlined a 12-step process to get from publisher ad call to marketer ad creative.  In an Exchange environment the process is basically the same, but with three new parties involved: the SSP, the DSP, and the Ad Exchange itself, which all act as the ad selector, instead of the publisher’s ad server.

You can view a diagram of the RTB ad serving process at the top of this post – the numbers in the text refer to the steps labeled in the diagram.

When a browser navigates to a publisher website (1), the publisher’s web server sends back a bunch of HTML code (2) that tells the browser where to get the content (3) and how to format it.  Part of the HTML code returned to the browser (4) will include a coded link known as an ad tag.  That part is the same as in regular third-party ad serving.  But instead of returning a DFA or Atlas tag, the Publisher Ad server will return a tag that points to a RTB-enabled SSP, typically through a dynamic Javascript tag that passes information like the publisher’s ID, the site ID, and ad slot dimensions.

From there, the user calls the SSP server (5) where the SSP reads that user’s SSP cookie ID, likely already on their machine.  Assuming the user already has that SSP’s cookie on their machine (and most users will, given that the largest SSPs boast 80 – 90% reach rates to the US internet population), the SSP starts the auction by requesting bids from a host of demand sources, the DSPs (6).  If the user does not have an SSP cookie on their machine, their ad inventory can technically still be auctioned, but since nothing is know about that user, the price will be very low and more related to the site context than the user’s attributes.  For the DSPs to truly value the impression though, they need to know something about the user that is going to see it.  This is where the SSP cookie ID comes in – packaged with the bid request is the SSP’s cookie ID, along with the url the impression will deliver on, and what the current user’s frequency is on the site.

All these factors help the DSP value the impression.  First, through a rather complex cookie-syncing process, DSPs are able to match the SSP’s cookie ID to their own cookie on that user, which is tied to a huge cache of marketer data and 3rd party data.  What kind of 3rd party data?  Using the cookie ID, the DSP will be able to know if that user recently priced out a car, is flying to Paris in the next 90 days, was recently shopping for shoes, and even more general demographic information about the user such as their age, gender, income range, credit score, and much, much more.  I’ll cover how that all works in a later post.  But suffice to say, rich data is far and away the driver of higher bids, and the cookie ID is the mechanism through which data is associated to a user.

In addition to the cookie though, where the ad will appear, the URL, is also important.  Many brands don’t want their ads to appear on just any site, even if they want that user.  If the user is on a site with PG-13 content, for example, the advertiser might bid a lower amount or not at all.  Similarly, the frequency of that user to the site they are on is also important to valuation.  Advertisers are willing to pay a premium to reach users on their first or second pageview on a site vs. their 50th pageview for the simple fact that users are less engaged with site content and more likely to respond to an ad during their first few pageviews.

Using those pieces of data, the DSPs all value that impression and submit a bid back to the SSP (7) as well an ad redirect to send the user should their bid win the auction.  The SSP picks the winning bid and passes the DSP’s redirect back to the user (8).  From here the process is basically the same as third party ad serving – The user calls the DSP (9), the DSP sends the user the marketer’s ad server redirect (10), and user calls the marketer’s ad server (11) and the marketer serves the user the final ad (12).  The RTB ad serving process is complete – whew!

How Real Time Bidding, DSPs, SSPs, and Ad Exchanges Work

Let’s say you’re online one day and decide to do a little shoe shopping. You navigate to your favorite store, check out a pair of boots, add something to your cart and just as you’re about to checkout, the phone rings and you get distracted.  By the time you’re done talking to your friend, it’s late and you decide to buy the shoes later.

Then, the next day, you check to see who won the big game last night and you notice an ad from the shoe store you were on last night.  Not only is it an ad for the store, but the exact pair of boots you were looking at are in the ad!  Weird.  You decide to check your email and see that your mom sent you a link to a news article. You go to read the article and staring you in the face on the page is another ad for the same pair of boots, this time tempting you with a 10% discount!

How did the ads know you were shopping for shoes last night, and how did they wind up on all those different sites?  The answer is, probably through an ad exchange.

Ad Exchanges have been around for a few years, but have exploded in importance in the last year.  Along with Demand Side Platforms (DSPs) and Supply Side Platforms (SSPs), Ad Exchanges are dramatically changing the way digital media is bought and sold. If you are a digital marketer or publisher, it is a very exciting time to be working in the industry.

What makes these companies so innovative is how they allow buyers and sellers to value inventory on an impression by impression basis and in real-time.  That’s right, real time.  That means that when you clicked on your mom’s link to the news article and your browser requested an ad from the news site, the publisher put that ad up for auction on an exchange, marketers bid on that impression, and it was served to your browser in about 250 milliseconds, so fast it was indistinguishable from the time it took any other image on the page to render.  Welcome to the world of real-time bidding, or RTB, where marketers value each impression as it is created and the Ad Exchange is where it all happens.

From a technical perspective though, how does the ad exchange process differ from regular ol’ third-party adserving?  I’ll answer that question and diagram the process in my next post, similar to what I showed you in my diagram for how third-party ad serving works.

Get Pixel Tracking Transparency with Ghostery

Thanks to a series of articles in the WSJ, publishers around the country are taking a hard look at their privacy practices and trying to get a handle on who collects data on their site.  You would think this would be a simple task, after all, the publisher owns the site and controls everything on it, right?

Well, not exactly.  In fact, thanks to the off-site redirects inherent to 3rd party adserving, publishers often have no idea when an advertiser or marketer attempts to redirect the user within a 3rd party ad tag.  Due to the number of players involved, it’s actually quite difficult to assess which tags are attempting to cookie the user for audience aggregation.  If publishers can’t audit their site, how can they enforce their privacy policy and contractual agreements with marketers?

Thankfully, the people at Better Advertising have developed a rather brilliant browser extension called Ghostery to make pixel tracking more transparent.  Ghostery runs on your browser and sifts through all the code and ad calls to quickly identify which 3rd parties are tracking data on your site. This particular example is from Dictionary.com – as you can see, the tool quickly pulls up a list of the various companies with pixels running on the site or somehow spawning to the browser.


From there, you can take a deeper dive on any particular tracker you want, view a brief summary of what the company does, how to access its privacy policy, and even other sites where that company was seen.  I have to say, Ghostery is a quantum leap ahead of other tools for identifying which ads are spawning pixels or running piggyback cookie requests.

Ghostery was actually developed more for Consumers to give them a way to see who is tracking their behavior online and actually block it, but I see huge potential for industry folks as well to audit their site.  Do you know what is running on your site?

P.S. – the Ghostery Blog isn’t half bad, either…