Interactive ads are everywhere these days, but when it comes to the technical process of getting an ad on the page and how publishers and marketers verify it delivered, not many people can explain what actually happens in detail. Read this article though and you’ll be one of them! Below I’ve detailed step-by-step how a browser gets from the initial call to a publisher’s website to the final ad creative, and when and how each party counts an impression. You can view a diagram of the ad serving process at the bottom of this post – the numbers in the text refer to the steps labeled in the diagram.
So, without further ado -
When a browser navigates to a publisher website (1), the publisher’s web server sends back a bunch of HTML code (2) that tells the browser where to get the content (3) and how to format it. Part of the HTML code returned to the browser (4) will include a coded link known as an ad tag.
Here’s an example of what an ad tag from Doubleclick, one of the major ad serving companies, looks like:
http://ad.doubleclick.net/ABC/publisher/zone;topic=abc;sbtpc=def;cat=ghi;kw=xyz;tile=1;slot=728x90.1;sz=728x90;ord=7268140825331981?
Click here to read more about ad tags and how they are constructed.
The ad tag points the browser to the Publisher’s Ad Server (5), a system designed exclusively for delivering and tracking advertising. In most cases, the Publisher’s Ad Server is actually a network of cloud servers owned and maintained by a separate company. In this case, the content server tells the browser to fetch the ad from Doubleclick, a company owned by Google, that then makes the very complex decision on which ad to serve using a program called an Ad Selector.
In many cases the ad server is deciding among thousands upon thousands of potential options in mere milliseconds. The computational power behind the Ad Selector is mind blowing – Atlas, the major rival to Doubleclick calls the supercomputer running its Ad Selector “WARP” and it is among the most powerful in the world, making billions of decisions a day and trillions in its lifetime. The Ad Server makes a decision, and in most cases sends back another ad tag (6), or redirects the browser by pointing it to the Marketer’s Ad Server. These redirects are technically speaking 302 redirects, which tells the browser the page has been “temporarily moved”. This allows Ad Servers to count the 302 call as an impression and host the actual ad content on a different server. Once the publisher’s adserver sends the browser a redirect to the marketer, it counts a delivered impression in its own database (star). The only exception here is if the publisher decides to deliver a house ad or the marketer has asked the publisher to “site-serve” the ads, both of which requires the publisher load the actual creative files into their ad server, meaning the publisher is the final destination, and the browser can skip the loop through the marketer side (steps 7,8,11,12).
Click here to read more about why Publishers and Marketers have their own Ad Servers.
The browser now calls the Marketer’s Ad Server (7) and is redirected yet again to a Content Delivery Network, or CDN, (8) a global network of cloud servers that actually house the raw creative graphics to fetch the actual ad. Why, you ask? Well, as powerful as ad servers are, they just aren’t equipped to handle the volume and bandwidth required to deliver content as heavy as image files. Redirects are often nothing more than a 1×1 pixel requiring just a few bytes of memory. Image files on the other hand are kilobytes or even megabytes in size, could be called millions of times a day, and require a much faster and robust infrastructure. Ad Servers might maintain three to six data centers across the world, but a CDN can process the heavy bandwidth and deliver the content faster because they operate hundreds of data centers and can route requests to the one nearest to the user, no matter where they are on earth. You can think of the ad server as the brain and the CDN as the brawn. Ad Servers aren’t the only companies that use CDNs, in fact many websites host their bandwidth intensive files in these cloud networks. A CDN is almost always another independent company, such as Akamai, that hosts the heavy creative assets so the Ad Server doesn’t have to. There used to be a handful of these companies out there, but Akamai has acquired almost all of them and is the largest player by far in the space.
Here’s what a CDN redirect to an Akamai server hosting a flash file looks like:
http://spe.atdmt.com/ds/ABCDEF12334/filename123_300x250.swf
In addition to sending back the redirect to the CDN, the Marketer’s Ad Server also appends a second redirect (10) back to itself with a query string to fetch a 1×1 pixel (11) after the ad content has been called. When the browser fires this last redirect calling a 1×1 pixel from the Marketer’s Ad Server (11), the Ad Server knows the ad was successfully downloaded and it finally counts an impression in its own database (star).
In many cases, your browser has to make at least four calls for site served ads and six in the case of third-party served ads for this whole process to work, if not even more, but shouldn’t take more than a second regardless of the number of parties involved. To visualize the process explained above, please see the diagram below – 302 redirects are highlighted in blue, and the ad creative is highlighted in red.
In the next post I’ll write about how this process changes when an impression is sold in real-time (RTB) on a Supply Side Platform or Ad Exchange.
{ 20 comments… read them below or add one }
Brillant description. Concise and effective. Thank you !
Hello,
Good explanations indeed
Can you explain clickcount in detail as well?
Is it ; publisher adserver -> marketer adserver -> site ?
Or ; marketer adserver -> publisher adserver -> site ?
How clicktags in flash are handled ?
Thanks !
Hi Matthieu,
Clicks are tracked in much the same way as impressions, with redirects. Typically the publisher’s ad server populates multiple click tags into the href of the ad, concatenated with a query string so that when a user clicks the ad, the browser calls all three links more or less simultaneously. But yes, the specific order with any 3rd party system is always publisher ad server, marketer ad server, final destination. Highly simplified for sake of readability, the href to track clicks on a standard .gif ad might look something like this: href=”http://ad.doubleclick.net/activity;id%123456789%redir%http://marketeradserver.net/click;id%123456789%redir%http://www.destinationurl.com/”. Flash and javascript ads approach how the code functions slightly differently, but the method of using redirects to count the same action in multiple databases is the same.
Hope that helps –
Ben
Brilliant website indeed! It’s especially useful for people like me who are very new to this industry and working as ad-ops engineer. I’m still trying to learn the ropes of this industry and this website has been a great help to me, a life-savior almost
Thank you. And continue with the great job!
Awesome, fantastic i really enjoyed while reading this and clear all my doubt and comcepts how an ad is delivered while an ad call is made….thanks Ben
I’ve been wondering how ad servers worked, and this is by far the clearest explanation I can find. Very well written, thank you, Ben!
Job well done, Ben. Thanks for posting. This is one the clearest descriptions of the overall process I’ve seen.
Great article! Please I have 2 questions:
1. If a marketer has asked a publisher to “site-serve” the marketer’s ad, can the marketer still use a marketer ad server to verify the impressions and clicks delivered by the publisher?
2. If a marketer is using a marketer ad server to serve their ad, can they still use a second marketer ad server to verify the impressions and clicks delivered by the first marketer ad server?
Hi Gbolahan,
Thanks –
1. I suppose so – they could ask the publisher to host the creative, but still embed some type of tracking pixel in the ad. I’m not sure why anyone would do this though – usually creative is site served because the advertiser is too small to have a need for their own ad server. Perhaps if the publisher was responsible for developing the creative on the advertiser’s behalf, they would host the creative, but still embed some kind of 3rd party tracking mechanism for the advertiser. Both the publisher and advertiser ad servers will end up hosting the raw creative asset on a CDN like Akamai, so there isn’t a benefit from a latency standpoint.
2. They can, this is called 4th party tracking and tends to happen when the marketer will have a handful of rich media ads that are served through a different system than their primary ad server. In this scenario, the rich media vendor would be the 3rd party from a publisher perspective, and the primary ad server, perhaps DART, would be the 4th party. Generally, publishers do not allow advertisers to bill on a 4th party tracker because they incur a dual discrepancy, but the practice of using them is quite common to centralize reporting.
Hope that helps -
Ben
Thanks a lot for your reply Ben. I’m clear now.
Loving your write ups, quick question.
With regards to how ad.serving works between the marketer’s ad.server and content delivery network, does the marketer send a 302 redirect to the content network via the browser or is it a different type of redirect?
As in you say once the 302 redirect has occurred the impression is counted for the ad.server. But in the case of the marketer, the impression isn’t counted till the actual ad is served (having bounced the query pixel off the browser and back) which is after the initial redirect of browser to content network.
Cheers,
Allan
Hi Allan,
Yes, the ad server tells the browser to fetch the creative file from the CDN through a 302 redirect. If you were to view a page through a web development tool like Firebug, you would be able to look at both the HTTP request and response headers for each piece of the ad call, and you would see that the initial call starts with a 200 request, or a standard GET request to the publisher’s ad server, that call responds with a ’302 ‘Object moved’ header, and the location of that file, redirecting the browser to the marketer’s ad server, which responds in kind with a 302 ‘Object moved’ header, and the location of the file on the CDN, which it can finally fetch.
To be clear though, it’s usually only that simple with simple .gif or .jpeg ads. It can get a little confusing in practical application because the process may not look so straightforward if, for example, the tag loads a flash creative. In that case, the tag may redirect to a script, and the confirmation pixel via a 302 response, and the script may have the subsequent calls to the CDN already embedded in the file. In that case, it isn’t technically a 302 redirect, but likely just looks like another 200 GET request. You can start to wander down the technical rabbit hole pretty quickly here, so my goal was to explain the process as simply and in a way that was as easy as possible to understand. To really dig in, I would recommend you speak to a sales engineer at your ad server, who will know the specifics for your setup with their technology.
Hope that helps –
Ben
Ben,
I built a .net site for a client, and created a custom ad rotator that logs click throughs to a database and then redirects the browser to the target url. It works great for all the jpg’s in the rotator, EXCEPT for a Doubleclick.net ad. I can log the image click, but can’t response.redirect to the target url. I just get a “page can’t be displayed” error. I’ve read that Doubleclick ads need to open in a new window, and tried several variations of window.location… and window.open(), but they go nowhere. Any help would be greatly appreciated. The ad url is below, and the TimeStamp var correctly inserts the proper format of the date, as required in the url.
http://ad.doubleclick.net/jump/N2886.291385..COM/B5246111;sz=190×190;ord=?”
Thanks very much!
Bob
Hi Ben,
I find your explanation extremely interesting, I was really looking for something like that.
I would like to ask you other informations, in particular I’m focusing my attention on CPA: CostPerAction. I know we can divide it in different subcategories: CostPerLead, CostPerSale, CostPerEngagement. For example, I don’t want to pay just because someone arrives on my landing page by clicking on a banner, but I want to pay if someone arrive on my landing page and complete a registration to my web site (CPL) or, even better, if he complete a purchase (CPS). How can the ad server “understand” that the customer has completed a purchase? How can it know that he performed a registration?
Another new type is CostPerCall, especially in the mobile area, where an advertiser pays when someone calls, for example, a call center. I have seen different apps that, after a click on a banner, automatically redirect you to your phone app, with the number already composed, you just have to press the “call” button. But my question is: how can the ad server be sure that I complete the call process? How can it know that I have not pushed the “cancel” button?
Thank you very much
Cheers
Alex
Hi Bob,
Thanks for your comment – I’m not sure I’m technical enough to understand what you’re trying to do, but it sounds like you’re trying to append your own click tracker on the fly to an ad tag by standing in the middle of the redirect path? That particular doubleclick tag won’t open for me, so there’s not much advice I can give you at this point without seeing more details, and even then I’m not sure I have enough development skills to really help. You might try looking through the doubleclick documentation online, or connecting with a broader forum of Ops people at AdMonsters, which has a free forum where you can ask questions.
Sorry I couldn’t be of more help – good luck!
Ben
Hi Alex,
Thanks – ad servers track conversions, however you want to define it, through pixels. Effectively, you would get a 1×1 image file from the ad server that you would place on the confirmation page for your conversion – this might be a thank you page or an order summary page, whatever destination would confirm the user actually took the desired action. Now, when a user converts, they land on that page and calls that pixel, which tells the ad server they converted, and allows the ad server to associate the right media to that user by looking at the cookie it has on that user’s machine. It’s effectively the same process used to track an impression, just without the ad itself involved in the process. You would have to do a little extra work in trafficking the ad to link everything up correctly (I would refer you to your ad server’s documentation for the specific details that apply to you and your ad server configuration), but it’s fairly straightforward to do, and a common strategy, so it should be well documented.
In terms of cost per call, I believe that is exclusively a search concept vs. the display ad concepts I tend to focus on here, but as I understand it, there are a few ways to do that, depending on how you plan to execute. For example, Google’s bid per call in the desktop environment requires the advertiser to use Google Voice generated tracking numbers (usually positioned to the user as an extension), which allows it to track the conversion. For mobile devices, I’m less familiar with the approach, but I imagine it works either through a click tracker that fires when a user clicks on a phone number in a search result, or also depends on a Google generated call tracking number. Google is the only company I’m aware of that actually offers this type of service.
Hope that helps –
Ben
This is great, thank you. Do you know if there are best practices as a publisher re: working with multiple ad exchanges? Should i be calling each exchange concurrently or sequentially based on their response?
Thanks.
Hi Ben,
First off, excellent job on this article. I’ve searched high and low for simple explanations of ad serving and this is by far the easiest and most intuitive read yet. Second, I had a quick question about using fourth-party calls in Google swf banners being served up via the Google Ad Network.
In short, my banner swf file is dynamically calling on three separate jpg files using absolute paths (i.e. http://www.mydomain.com/frame1.jpg). These banner frame jpg’s are hosted on our own internal server so that we can update the banner creative on demand “without” having to edit, change, or resubmit new swf files to 3rd party vendors.
The problem I’m running into is that when I upload my banner swf into the Google Ad Network I get denied as they don’t permit 4th-party server calls. Is there any sort of workaround for this by chance? We still want to use the Google Ad Network for banner tracking ect (clickTag) but on the other hand we’d also like to host our own banner creative. Any advice is much appreciated and thanks again for such an awesome article.
Best regards,
NM
Hi Chris,
Ideally, you would call each exchange concurrently, and take the highest paying bid from either. Without an SSP to sit as a layer between the exchanges and help you manage this process, I’m not sure exactly how you would do this. If you don’t have the scale to make it working with an SSP make sense, my advice would be to try and test a few exchanges at the same time and see who can pay the most. The best test setup would be to run rotate a cookie value on your users for however many exchanges you want to test, and silo the exchange calls per cookie value so that each exchange gets the entire user session. If you mix impressions between exchanges from the same user session, you’ll confuse the frequency tracking of buyers, and potentially throw a wrench in your final results.
That said, my guess is that if you execute the test successfully, you’ll see similar results from every exchange, since most bidders are on all the exchanges, so it shouldn’t really matter all that much which one you pick.
Good luck!
Ben
Thanks NM,
Probably not – Google Engineers are pretty smart – they’re not likely to allow a loophole, at least not for very long. That said, Google does allow some advertisers to 3rd party serve, and that would pretty much solve your problem http://support.google.com/adwordspolicy/bin/answer.py?hl=en&answer=94230. With a 3rd party tag, you could swap out the display creative whenever you want, without having to change the actual ad tag in the Network. Google says this functionality is available only by invitation, but you could ask support if you really wanted, and probably find a way to get there as long as you can work with an approved 3rd party. Of course, this probably doesn’t make sense unless you are a big advertiser. If you’re a small shop, I doubt there’s a workaround. This is really a question for Google Support – I’m just not this intimately familiar with the terms and policies enough to give you a solid answer here.
Best of luck!
Ben
{ 1 trackback }