SSP to DSP Cookie-Synching Explained

The matching process of the SSP cookie ID to the DSP cookie ID happens through a parallel process to serving ads called cookie-syncing. Cookie-syncing is necessary because as a standard security process, web servers of any kind can only request cookies that are set to their own domain. Since the SSP sits between the end-user and all the DSP bidders in a real-time auction however, the DSP needs a way to identify the users it is looking for.

So let’s take a simple example on a store trying to retarget their users. Let’s say that you run storeABC, and user123 drops in and adds a pair of $150 shoes to their shopping cart, but never makes it to the checkout. You want to retarget that user and serve them an ad directing them back to your store to try and close the sale. Since you work with DSP456, you have a 1×1 pixel sitting on your shopping cart page, which forces the user to call out to DSP456′s web server as they load the shopping cart page, giving DSP456 a chance to drop a cookie on user123. That cookie ID is DSPcookie789. Now, user123 is surfing around the web, and lands on awesomesite.com, which is using SSP123 to monetize their ad inventory. awesomesite.com serves a 3rd party redirect to SSP123, which drops a cookie on user123. That cookie ID is SSPcookieXYZ. SSP123 now requests a bid from DSP456 among other bidders for the impression that SSPcookieXYZ is about to view. But wait, how does DSP456 know that SSPcookieXYZ = DSPcookie789? On this first ad, it doesn’t, so your DSP doesn’t bid on the impression. Bummer. Remember, the SSP can only read and pass its own cookie ID to bidders.

After SSP123 selects a winning bidder though, it runs one last piece of javascript that forces user123 to call out to a handful of regular bidders, including DSP456. In that redirect, using a query string, the SSP passes its cookie ID on user123 (SSPcookieXYZ). Now the user IS calling DSP456′s web server and the DSP can request its own cookie from user123 in a process the industry refers to as “piggybacking”. Eureka! SSPcookieXYZ also has DSPcookie789 – it’s user123! The DSP knows the SSP’s cookie ID because of the query string in the piggyback call, and it can read its own cookie ID because that user called its web server as the end destination with the piggyback call. DSP456 now writes into its database that DSPcookie789 = SSPcookieXYZ for bid requests from SSP123. The next time user123 hits a page that SSP123 is helping to monetize, DSP456 will know it is the same user that abandoned their shopping cart and can bid appropriately. The process repeats for sites using other SSPs.

It sounds complex, but all it means is that for every ad served through RTB, about 10x as many technology companies are involved in the cookie-synch process. The reason the synching process occurs after the auction happens and not before it is because of latency and user experience. If user123 had to call 10 DSPs, wait for those DSPs to cookie their machine, write the matching SSP id into their database, and then bid, it would dramatically slow down the entire auction process for everyone. If the cookie synch fails on the other hand, well there will be many more opportunities for that.

If you are interested, here are the URL’s that run the piggyback script for each major SSP – these pages look blank, so you’ll have to ‘view source’ to see the code.
Pubmatic: http://ads.pubmatic.com/AdServer/js/syncuppixels.html It looks as though vcode is the item passing the cookie ID.
Rubicon Project: http://tap2-cdn.rubiconproject.com/partner/scripts/rubicon/emily.html It looks as though nid is the item passing the cookie ID.
Admeld appears to take a slightly different solution, passing a browser partner by partner with callbacks to Admeld on each run rather than calling out to a handful at once like the SSPs above. I’m not able to identify the logic here as easily, but they must accomplish the same end result.

25 comments

  1. Could you expand on this statement:

    “After SSP123 selects a winning bidder though, it runs one last piece of javascript that forces user123 to call out to a handful of regular bidders, including DSP456”

    So how exactly does that happen?

    Thx

  2. Hi Jason,

    Basically speaking, the SSP’s tag is a chunk of JavaScript that in turn references other scripts. That’s pretty common with JS ad tags in general, actually – they usually have to do a few things to get the ad to work as designed. Sometimes they need to reference jQuery libraries, access browser plugins, compile information from other servers, reference cookies values, and a bunch of other things. Specific to the SSPs, the first sub-script usually redirects the user to the SSP’s server which starts the auction process. The last sub-script is a call which tells the browser to go call a bunch of other companies for the sake of cookie syncing users. For most browsers, this last script in practice may run more or less simultaneous to the auction / ad load, because by then, the browser has been redirected downstream to other parties and has moved on to the next part of the tag. Most modern browsers can keep 6 – 8 HTTP connections active at any given time, so once it executes the first sub-script, it starts following that path with some of the available connections and starts executing the next sub-script with others, depending on what else it has to load on the page.

    Hope that makes sense –

    Ben

  3. Hi Ben,

    Can one company be both SSP and DSP, then no syncing is necessary? Is it because of privacy concern?

  4. Hi Wes,

    No, no company plays both sides of the fence and is at once a DSP and an SSP. Google does own both types of companies – Invite Media is a DSP and Admeld is an SSP, but those companies still maintain separate cookie spaces on separate domains, so still need to cookie sync with one another. Their is no real privacy concern, the reason that there isn’t a DSP / SSP hybrid is because having both advertisers and publishers as a client on the same platform is a large conflict of interest – how would the platform represent both sides fairly – that if such a platform existed, I doubt they would have any customers. You wouldn’t hire a real estate agent to sell your house if they also represented the buyer, would you? Why would the agent have any incentive to get a fair market value for your house? For the same reason, DSPs and SSPs must stay as separate systems – many expect clients to abandon Admeld because of the conflict of interest for Google to represent both a buying and selling technology, but I can’t say I know that it’s actually happened in practice. As far as I know Google keeps those two companies very separate, and there’s no reason to think that Invite and Admeld don’t effectively operate as two separate, independent entities.

    Ben

  5. Thanks for the great explanation, just curious to know
    1. Will DSP always have their code on advertiser website or there will be cases when the DSP cookie needs to be synched with advertiser’s cookie ?

    2. As explained in your another post, DMP will have cookies synched with various 3rd party systems, now how will DMP integrate with DSP, will it require cookie synching ?

  6. Hi Akhil,

    To answer your questions –

    1. There’s no reason I can see why an advertiser wouldn’t fire their DSP’s cookie at the same time they drop their own cookie for any users they want to target in the exchange. In some cases however, an advertiser may have offline data and work through a match partner and data management platform (DMP) to bring that data online to a cookie, in which case the DMP would need to sync with the DSP before those users would be active in the exchange. That’s a much less common use case however, and in that instance, the advertiser doesn’t really have a cookie in the classic sense, dropped from their own domain, but really reusing someone else’s cookie space for their own purposes.

    2. Yes, DMPs integrate with DSPs through cookie syncing. A system to system sync only needs to happen once however, not once per system per client. Once the users are synced, the bridge benefits all the clients on both systems, who can move securely move proprietary data from one place to the other through a server to server integration.

    Hope that helps –

    Ben

  7. Hi,
    I am not sure I understand this:

    “The DSP knows the SSP’s cookie ID because of the query string in the piggyback call, and it can read its own cookie ID because that user called its web server as the end destination with the piggyback call. DSP456 now writes into its database that DSPcookie789 = SSPcookieXYZ for bid requests from SSP123. The next time user123 hits a page that SSP123 is helping to monetize, DSP456 will know it is the same user”

    – wouldn’t it be in the DSP’s favor to ‘fake’ the cookie match? Say for e.g. the DSP456 says that the DSPcookie999 = SSPcookieXYZ, then SSP has no way of knowing whether DSP has genuine match or not and it will have to favor DSP456 for the subsequent bids.
    [here I am assuming that winning the bid matters the most to the DSP rather than winning the right bid, for the purpose of getting visibility or some other xyz reason…]

  8. Hi Rohan,

    It’s important to realize that the DSP and the SSP have a mutual benefit in syncing cookies – browsers only allow a cookie to be read by the domain from which it was set, meaning the SSP can only read its own cookie on a user, and can’t know if a DSP also has a cookie on that user. In the bid request then, the SSP can only show bidders its own cookie ID, it can’t provide the DSP’s cookie ID. So it’s critical then that the DSP have a way to know if it has a cookie on that user, and what it knows about that cookie, from the SSP’s cookie ID only. The way it does this is through the cookie syncing process.

    To your question about faking the cookie match, though, the DSP doesn’t really have anything to gain in that scenario. It’s not as if the SSP knows anything meaningful about that user through the match, they just know if the user exists in the DSPs system, and vice versa. The cookie match simply facilitates the transaction. And the fact of the matter is that DSPs don’t bid more for un-synced cookies, they bid less, because they don’t know if they know anything about that user. Even if they were lying some of the time, in general, the DSP will likely bid less for unsynced users overall, so the SSP won’t favor them at all in that case. But again, there’s not much point – it’s far more important for the DSP to win the right bids than to simply win any bid, that’s the entire point of using a DSP. Otherwise, you could just use an ad network to buy as many impressions for the cheapest price possible, regardless of any knowledge about the user set.

    Hope that clears things up –

    Ben

  9. Hi Ben,

    What other information can be passed via an SSP ID to the DSP to add value in the bidding process. From your description, the cookie syncing only supports a binary decision from the DSP as to whether or not there is a DSP cookie on the users machine – yes or no. It is commonly referenced within the industry that publishers have a wealth of data points on their users, and the onus should be on the publishers to expose this data with a view to better monetizing their sites. With the above definition, all of the information used as the basis for the bidding decision is on the advertiser side; through the retargeting pixel. Is there any other information that can be passed via the SSP cookie which would give the bidders greater insight into the user and their behaviour, so for example; the segment IDs (and segment definitions) to which the user belongs as classified by the publisher or the SSP.

  10. “you have a 1×1 pixel sitting on your shopping cart page”

    Hello,

    you mean embedded javascript right? not a real pixel ?

    Ty

    Franck

  11. Hi Ty,

    What I mean by that 1×1 pixel is basically, “you have a mechanism to track a conversion event” – that might manifest as a simple image tag for a transparent 1×1, or a piece of javascript may configure that conversion beacon call in some kind of dynamic fashion, or you may simply drop a cookie on that user through an image call. But whatever the specific technical execution, the point is the same, to track a conversion, and associate it to a specific user and that user’s other actions and touchpoints.

    Ben

  12. Hi Ben

    I’ve been reading all your blog trying to figure out the scope of every player or system in the ecosystem of RTB.

    I will try to explain my understanding and my doubts.

    I understood that in some cases the SSP and the Ad Exchange are the same system / company, is that right?

    Also the SSP is who offers the publisher inventory to the DSP’s.

    The thing that I don’t understand is how the SSP and DSP are connected? How is a deal between those players in commercial terms? Let’s say that I want to make an SSP, how do I connect with the DSPs?.

    I’ve readed that the DSP and SSP are not the same company in order to give some transparency and to avoid conflict of intesrest but Right Media is offering a DSP and SSP as well.

    I hope that you can help me to clarify these points.

    Thanks!

  13. Hi Hernan,

    To your first question, the SSP and Ad Exchange can fill similar roles in the ecosystem in terms of providing a marketplace to aggregate supply and demand, and create liquidity in the ad market. The main difference between the two is that the SSP is a technology that work to optimize revenue for publishers only, whereas the Ad Exchange just creates a market and is agnostic to getting the best price for the advertiser or publisher. You might think of SSPs as a sub-type of Ad Exchanges.

    To your second question, yes, the SSP offers inventory to the DSP. The SSP and DSP connect through an API, which in most cases is compliant with the OpenRTB specification, detailed here: http://www.iab.net/media/file/OpenRTB_API_Specification_Version2.0_FINAL.PDF The SSP and DSP do sign commercial terms if they want to transact with each other, and while it’s hard to boil a lengthy legal contract down, it effectively says the DSP will bid on requests from the SSP, and if the SSP accepts the bid, the DSP must pay the SSP that price within a certain period of time. If you wanted to start an SSP, you’d need to aggregate some supply by working with publishers or networks, and then you’d have to negotiate contracts to sell that supply of inventory to DSPs (or ad networks, trading desks, or advertisers directly).

    To your last question, Right Media is an exchange, but as you can see, the lines between exchange, DSP, and SSP are easily blurred. Right Media might offer services to buyers that enable them to bid on inventory in real-time, but it wouldn’t connect to many different exchanges like a classic DSP. Similarly, Right Media might help publishers sell their inventory, and even offer them tools to set rules and controls around what bidders are allowed, and at what price, but it won’t take an algorithmic approach to optimizing for revenue. It would be a conflict of interest to run the auction and optimize publisher revenue, and a paradox to optimize publisher revenue and advertiser spend from the same platform.

    Many other players facilitate RTB connections to both the buy and sell side, for example, Appnexus is another company that can serve both sides of the house, but isn’t a SSP or DSP in the classic sense. As you navigate this space, sometimes it’s helpful to ask these companies how they make money – do they sell services to buyers, or sellers, or both? Do they make a transaction fee regardless of clearing price, or a revenue share on the clearing price? Based on the answers to those questions, you can often figure out how the company has positioned themselves in the RTB ecosystem.

    Hope that helps –

    Ben

  14. I’m why curious SSP123 “runs one last piece of javascript that forces user123 to call out to a handful of regular bidders” as the last task rather than the first task? It would seem that registering SSPcookie123 with DSP456 would be the first thing to do so that the relationship would be established before the bidding began. Am I missing something?

  15. Hi Ken,

    The reason this last script runs is for the benefit of the overall marketplace. While you are correct that SSPcookie123 has to be synced with DSPcookie456 before the DSP would bid for that user, that syncing does not take place before an impression the DSP would actually bid on. Meaning, the marketplace effectively burns one request per unsynced user to establish the relationship between systems. There’s not enough time to sync the user and then make a bid, it can only be one or the other.

    Because of that, SSPs and DSPs have a major financial incentive to sync users outside of auctions, and keep them in sync. That’s why they run this last script, which has the sole purpose of syncing cookie identities between systems. That way, the user is synced between 5, 10, 15, or more systems simultaneously so that for the next impression, the relationship is already established between systems for that user. This is important, because there are many types of bidders out there, some that bid on lots of impressions, and some that bid on just a few. The difference is that the DSPs that buy few impressions may be willing to pay the most, because they are looking for very specific users. If they only had the opportunity to sync users with supply sources for the impressions that they had the opportunity to bid on, the would have reduced reach, and therefore every party in the market would suffer, except for maybe the big DSPs that bid often but pay lower prices.

    Hope that clarifies –

    Ben

  16. Hi Ben, great article. However, it looks like publishers (in this case the shopping cart page) have to add a pixel for every single DSP behind the exchange? Isn’t that inefficient for the publishers since they now need to learn about the tags for these different DSP’s? Moreover, is the publisher supposed to know about all the DSP’s sitting behind the exchange? Thanks.

  17. Hi Posharma,

    Actually no; through cookie syncing scripts, a single tag can actually fire calls to many different systems. Thus, through a single tag, a user may receive 10 or more cookies from different DSPs. That’s only if the publisher wants this to happen though, for example if they wanted to target a user on the exchange, or sell that data to other bidders. Otherwise, publishers are likely to see this as a form of data leakage.

    Hope that helps –

    Ben

  18. Hi

    In the process described above, does the SSP also know that its user 123, has the DSP cookie 789..and hence is able to make a note for future transactions? Since impressions always happen at the SSP side, isn’t it important for SSP’s to know and detect the match before sending bid requests to DSPs in order to minimize wastage?

  19. Hi Shuba,

    The first statement you made is correct – the SSP can store the match table of what ID on its side matches which ID on the DSP side. But the second part isn’t as accurate; typically the SSP is simply sending its own cookie ID in the bid request to each DSP, which it expects has synced its ID (user 123) to their own (DSP cookie 789), rather than try to lookup the DSP’s ID and send the DSP their own ID in a bid request.

    So 123 gets sent to every bidder, not 789. This simplifies things for the SSP; they just know if a user is synced (and therefore worth more), they don’t have to do the dynamic operation of finding and injecting a unique ID per bid request per DSP integration.

    Hope that helps –

    Ben

  20. Great article. Does this work effectively on mobile? Specifically, iOS is blocking 3rd party cookies by default which it seems would prevent the synching and tracking from happening correctly on those devices at least.

  21. Hi Kurt,

    While the process runs as described in mobile, I would say that no, it does not work effectively. You’ve hit the nail on the head with iOS’s policy of blocking 3rd party cookies by default, which throws a wrench into this process. In fact, in mobile there is a whole set of companies that are working on what we call the ‘persistent ID problem’, usually by relying on device IDs created by the OS (IDFA for iOS, Advertiser ID for Android) or creating a synthetic or server side cookie. Device IDs are great, but they are only readable by mobile applications, and not a solution for mobile web traffic.

    But what exactly is a synthetic or server side cookie, you ask? Well, this gets into another set of questions on how those things work, and I’d refer you there to Gavin Dunaway’s truly excellent piece on AdMonsters on deterministic (I put a cookie that has a unique serial number on your machine that I can read again as long as you don’t delete it) vs. probabalistic (I have a supercomputer that compiles lots of disparate attributes about your device, like language, font size, screen resolution, carrier, location, IP, server clock variance, etc. that creates enough entropy between you and other users to effectively create a unique ID I can read) IDs. You might also think to look up AdTruth, who I would say was a pioneer in the synthetic ID space before being acquired by Experian last year.

    It’s an interesting topic in the industry, and one that’s still a moving target I would say.

    Hope that helps –

    Ben

  22. Ben,

    Great article! However, I am a bit puzzled by this
    “If user123 had to call 10 DSPs, wait for those DSPs to cookie their machine, write the matching SSP id into their database, and then bid, it would dramatically slow down the entire auction process for everyone.”

    How would DSPs know what to bid on the user impression if they don’t know whether the user’s attributes?

    Do they just bid on the user the first time just to get the SSP cookie ID to have their match tables updated? You also mentioned that the user is forced to DSP’s ad server only after a winning bidder is determined. So, the first time DSPs have just bid blindly. Am I right?

  23. Hi Amit,

    The DSPs likely would not bid on users for which they do not have cookied, unless they are buying on a contextual basis. The cookie sync scripts are what allow all the DSPs to write those cookies independent of the auction process. So as users navigate the web, they’re naturally exposed to the sync scripts, which identify them to DSPs, and enable the DSP to bid on them shortly afterwards.

    DSPs don’t bid blindly or just see users they’ve bid on because the cookie sync script which runs after every auction exposes new (and existing) users to their servers outside the auction.

    Hope that makes sense!
    Ben

Leave a Reply

Your email address will not be published. Required fields are marked *