Author: Ben Kneen

How RTB Ad Serving Works

Diagram of the Ad Exchange Redirect Process

I wrote about the basics of how DSPs, SSPs, and Ad Exchanges work from a high level and the value they add to the marketplace in my last post – for this piece we’re going to dig into the nitty gritty of how RTB ad serving works from a technical perspective.

In my explanation of third-party ad serving, I outlined a 12-step process to get from publisher ad call to marketer ad creative.  In an Exchange environment the process is basically the same, but with three new parties involved: the SSP, the DSP, and the Ad Exchange itself, which all act as the ad selector, instead of the publisher’s ad server.

You can view a diagram of the RTB ad serving process at the top of this post – the numbers in the text refer to the steps labeled in the diagram.

When a browser navigates to a publisher website (1), the publisher’s web server sends back a bunch of HTML code (2) that tells the browser where to get the content (3) and how to format it.  Part of the HTML code returned to the browser (4) will include a coded link known as an ad tag.  That part is the same as in regular third-party ad serving.  But instead of returning a DFA or Atlas tag, the Publisher Ad server will return a tag that points to a RTB-enabled SSP, typically through a dynamic Javascript tag that passes information like the publisher’s ID, the site ID, and ad slot dimensions.

From there, the user calls the SSP server (5) where the SSP reads that user’s SSP cookie ID, likely already on their machine.  Assuming the user already has that SSP’s cookie on their machine (and most users will, given that the largest SSPs boast 80 – 90% reach rates to the US internet population), the SSP starts the auction by requesting bids from a host of demand sources, the DSPs (6).  If the user does not have an SSP cookie on their machine, their ad inventory can technically still be auctioned, but since nothing is know about that user, the price will be very low and more related to the site context than the user’s attributes.  For the DSPs to truly value the impression though, they need to know something about the user that is going to see it.  This is where the SSP cookie ID comes in – packaged with the bid request is the SSP’s cookie ID, along with the url the impression will deliver on, and what the current user’s frequency is on the site.

All these factors help the DSP value the impression.  First, through a rather complex cookie-syncing process, DSPs are able to match the SSP’s cookie ID to their own cookie on that user, which is tied to a huge cache of marketer data and 3rd party data.  What kind of 3rd party data?  Using the cookie ID, the DSP will be able to know if that user recently priced out a car, is flying to Paris in the next 90 days, was recently shopping for shoes, and even more general demographic information about the user such as their age, gender, income range, credit score, and much, much more.  I’ll cover how that all works in a later post.  But suffice to say, rich data is far and away the driver of higher bids, and the cookie ID is the mechanism through which data is associated to a user.

In addition to the cookie though, where the ad will appear, the URL, is also important.  Many brands don’t want their ads to appear on just any site, even if they want that user.  If the user is on a site with PG-13 content, for example, the advertiser might bid a lower amount or not at all.  Similarly, the frequency of that user to the site they are on is also important to valuation.  Advertisers are willing to pay a premium to reach users on their first or second pageview on a site vs. their 50th pageview for the simple fact that users are less engaged with site content and more likely to respond to an ad during their first few pageviews.

Using those pieces of data, the DSPs all value that impression and submit a bid back to the SSP (7) as well an ad redirect to send the user should their bid win the auction.  The SSP picks the winning bid and passes the DSP’s redirect back to the user (8).  From here the process is basically the same as third party ad serving – The user calls the DSP (9), the DSP sends the user the marketer’s ad server redirect (10), and user calls the marketer’s ad server (11) and the marketer serves the user the final ad (12).  The RTB ad serving process is complete – whew!

How Real Time Bidding, DSPs, SSPs, and Ad Exchanges Work

Let’s say you’re online one day and decide to do a little shoe shopping. You navigate to your favorite store, check out a pair of boots, add something to your cart and just as you’re about to checkout, the phone rings and you get distracted.  By the time you’re done talking to your friend, it’s late and you decide to buy the shoes later.

Then, the next day, you check to see who won the big game last night and you notice an ad from the shoe store you were on last night.  Not only is it an ad for the store, but the exact pair of boots you were looking at are in the ad!  Weird.  You decide to check your email and see that your mom sent you a link to a news article. You go to read the article and staring you in the face on the page is another ad for the same pair of boots, this time tempting you with a 10% discount!

How did the ads know you were shopping for shoes last night, and how did they wind up on all those different sites?  The answer is, probably through an ad exchange.

Ad Exchanges have been around for a few years, but have exploded in importance in the last year.  Along with Demand Side Platforms (DSPs) and Supply Side Platforms (SSPs), Ad Exchanges are dramatically changing the way digital media is bought and sold. If you are a digital marketer or publisher, it is a very exciting time to be working in the industry.

What makes these companies so innovative is how they allow buyers and sellers to value inventory on an impression by impression basis and in real-time.  That’s right, real time.  That means that when you clicked on your mom’s link to the news article and your browser requested an ad from the news site, the publisher put that ad up for auction on an exchange, marketers bid on that impression, and it was served to your browser in about 250 milliseconds, so fast it was indistinguishable from the time it took any other image on the page to render.  Welcome to the world of real-time bidding, or RTB, where marketers value each impression as it is created and the Ad Exchange is where it all happens.

From a technical perspective though, how does the ad exchange process differ from regular ol’ third-party adserving?  I’ll answer that question and diagram the process in my next post, similar to what I showed you in my diagram for how third-party ad serving works.

Get Pixel Tracking Transparency with Ghostery

Thanks to a series of articles in the WSJ, publishers around the country are taking a hard look at their privacy practices and trying to get a handle on who collects data on their site.  You would think this would be a simple task, after all, the publisher owns the site and controls everything on it, right?

Well, not exactly.  In fact, thanks to the off-site redirects inherent to 3rd party adserving, publishers often have no idea when an advertiser or marketer attempts to redirect the user within a 3rd party ad tag.  Due to the number of players involved, it’s actually quite difficult to assess which tags are attempting to cookie the user for audience aggregation.  If publishers can’t audit their site, how can they enforce their privacy policy and contractual agreements with marketers?

Thankfully, the people at Better Advertising have developed a rather brilliant browser extension called Ghostery to make pixel tracking more transparent.  Ghostery runs on your browser and sifts through all the code and ad calls to quickly identify which 3rd parties are tracking data on your site. This particular example is from Dictionary.com – as you can see, the tool quickly pulls up a list of the various companies with pixels running on the site or somehow spawning to the browser.


From there, you can take a deeper dive on any particular tracker you want, view a brief summary of what the company does, how to access its privacy policy, and even other sites where that company was seen.  I have to say, Ghostery is a quantum leap ahead of other tools for identifying which ads are spawning pixels or running piggyback cookie requests.

Ghostery was actually developed more for Consumers to give them a way to see who is tracking their behavior online and actually block it, but I see huge potential for industry folks as well to audit their site.  Do you know what is running on your site?

P.S. – the Ghostery Blog isn’t half bad, either…

How Does Ad Serving Work?

Interactive ads are everywhere these days, but when it comes to the technical process of getting an ad on the page and how publishers and marketers verify it delivered, not many people outside ad operations can explain what actually happens in detail.  Read this article though and you’ll be one of them!  Below I’ve detailed step-by-step how a browser gets from the initial call to a publisher’s website to the final ad creative, and when and how each party counts an impression.  You can view a diagram of the ad serving process at the bottom of this post – the numbers in the text refer to the steps labeled in the diagram.

So, without further ado – (more…)

The Four Stages of Managing Digital Ad Inventory

Digital inventory management is usually handled in one of a couple ways at most major companies, depending how granular they sell, and how much traffic they get.  These are the main strategies, listed in order of sophistication as I see it.

1. Complete and utter reliance on the ad server’s availability forecasting tool
2. An excel-based inventory model based on historical ad server reports and pivot tables
3. A technology solution as part of a workflow suite such as Solbright, or Dart Sales Manager, often called DSM in the industry
4. A stand-alone technology solution for inventory management such as Yieldex

There aren’t too many major publishers left wallowing at stage 1, and those that are likely have more inventory and traffic than they can possibly sell.  There is a glut of impressions across the net, but unlimited is a lot, and there aren’t more than a handful of sites that are truly in this situation, except perhaps some of the social media and user-generated content sites like Youtube, MySpace, Facebook, etc.  For full disclosure, I have no idea if these companies are indeed flying by the seat of their pants or have a more sophisticated solution in place to manage their digital ad inventory.  On the other side of the coin, there are certainly a few small publishers out there that need an inventory solution, but either can’t afford the investment in the people or technology to push them forward.  These are the small publishers that hold a moderately valuable audience, or perhaps a big piece of a niche demographic, but drive the agency teams crazy with their constant underdelivery.

Stage 2 is usually a stepping stone to Stage 3 or 4, where a company has recognized it needs someone in an analyst role to help them deliver campaigns and support their sales team to build realistic proposals.  This is the start of a true inventory management branch of an Ad Ops department, and the best people in this role have a consultative approach and help sales find creative solutions to adjust plans before they go out the door, and what can be done to help struggling campaigns.  Unfortunately, as great as ad server reports are, this isn’t a manageable solution for any company with a significant sales team or significant growth.  At a certain point, excel doesn’t have the firepower, and human fingers aren’t fast enough to keep up with the sales machine.  When the department becomes overwhelmed, campaigns start underdelivering again, or a company loses the talent holding things together, they typically look to a technology solution.

At this point, things go one of two ways, and honestly, the decision tends to come down to what kind of billing system the company is already using, and if they are already paying for some kind of sales pipeline / workflow management tool.  The choice is between a stand-alone technology for the Ad Ops department, or a full-fledged, workflow management system that organizes everyone from Sales Management to ad traffickers.  Getting a whole company to buy into a solution and rethink their current system is not an easy undertaking however, especially when the ones clamoring for help are usually the back office folks.  Making the transition and fully integrating something like a Solbright is also daunting, and usually a year-long, full-time project for a handful of people who represent each end-user.  Sales has to be involved, management has to be involved, Ops has to be involved, billing has to be invovled, and usually, IT does, too.  These tools do a laundry list of tasks as well, so it’s no surprise they’re also more expensive.

For companies that are already on something like a Salesforce.com or proprietary / legacy billing system, a stand-alone solution probably makes more sense, and is usually faster to implement, barring a complex custom integration.  My experience with Yieldex has been somewhat limited, but I have had an opportunity to work with the system for a few months at this point, and when it works, it works impressively.  The main challenge I see with Yieldex is from a usability standpoint.  While the system provides an incredible level of data, it’s often very difficult to get to that data quickly, and en masse.  Still, Yieldex is certainly one of the more exciting options to come to market for companies looking to increase accuracy and visibility across a set of complex, overlapping products.