The Extreme Art of Placing Ads in Articles
Jason Goldstein and Juliet Sabol
It’s a subtle problem, but a critical one. As you read through an article, you pass over ads, newsletter signups, links to related stories, book modules, magazine widgets, and all sorts of other contextual tools that connect our readers to the rest of the publication.
The stakes are high: ads seen per pageview are a major dial of the business, and all those recommended reading and newsletter signup CTAs have an outsized impact on our ability to grow our audience, connect our readers to offerings like live events, and keep doing great journalism.
At the same time, too many sites overdo it to the point that these elements crowd out the very content they’re meant to support.
We think of this less as a balance than an art form: if space is finite, we need to be smarter about how we use it, so we can optimize both the reader experience and business needs at the same time.
The endgame is a grand theory of what we call Unified Yield: the ability to consider the reader, where they are in their relationship to The Atlantic, the context of the article they are reading, and dynamically complete the page with the right elements that make the most sense and perform best for that moment.
To do this, we need a flexible, powerful, and high precision system for inserting components into the new article page.
The Limitations of Server-side Injection
We first tackled this problem in 2014. We were about to launch a responsive redesign, and viewability had changed the nature of the ad ecosystem: we could now spread ads throughout a story. When you’re a publication known for long feature journalism, that’s a really big deal.
We built an injector in Python that would attempt to place them intelligently.
Running on the server, the code couldn’t know the height of the screen, and while it could read the content as HTML it couldn’t know anything about the layout. Instead, if we wanted to aim for one ad per screen, we’d have to guess.
This is tricky. You can count elements, but some paragraphs are long, some are short, you need to account for images of all sizes, and a blockquote is a single element that can run on for quite a while.
Instead, we used character counts as a proxy for length. On desktops, we’d attempt to place an ad every 2800 characters, or about every 560 words. Over time we learned it worked best if we only placed them between two paragraphs, avoiding images and special widgets entirely.
But we knew it wasn’t perfect. It is, after all, a blunt instrument.
Estimating one ad per screen isn’t actually one ad per screen. It struggled with floating elements that were side by side with other content, got stuck on edge cases, and was difficult to AB test or tune with the level of precision we need.
Plus, we have other things we want to show our readers: podcasts, newsletters, recommended articles, subscriptions, live events, apps, and seemingly more each year.
At the same time, we’re obsessed with the reader’s experience, and we want to avoid cluttering the article with furniture.
We want to spread all these things out intelligently, using strategies and algorithms that take the device’s actual size, the length and nature of the article, and in some cases what we know about the reader into account to form a system that’s performant and elegant.
We set out to tackle this problem in a more sophisticated way in 2021. And to do this, with the true device screen size and layout of the article content, the injector code would need to run in the browser.
Into the browser
This isn’t easy.
For starters, we have real concerns about performance, layout thrashing, and cumulative layout shift. Which meant it would need to be fast: all elements had to be in place before the reader scrolled to them.
At a high level, that meant the layout of the article had to change into a single column. The previous article page had both a main content well and a right rail, as is common on many news sites. The article was broken up into sections, with the wide ads spanning beyond both columns. There was no practical way we could do this client-side (in the browser) without completely re-rendering the page or taking a significant performance hit.
Switching to a single-column layout was a drastic change, but it was one everyone agreed on. The Design and Story Engineering teams wanted it to give themselves more flexibility to introduce new visually interesting modules within the article. Advertising was prepared to give up inventory in the right rail in exchange for the power and flexibility the new injectors offer. But above all, we believed the streamlined layout was a better reading experience.
This would be a running theme in our new article page, and it was less about compromises or happy accidents than carefully designing a machine so each piece would click into place.
At a low level, that meant the code should touch the DOM (document’s HTML) as little as possible. It would take a single pass to measure the height of each component, store that in a lightweight data structure, and all the rest of the process would be comparing numbers. Logical comparisons are much cheaper for the browser than touching the real elements in the layout — this is why popular frameworks like React have a virtual DOM. When you’re running a large number of operations, it can matter.
Finally, we needed the article to be available to our codebase as structured data. This allowed us to tell the difference between all the different kinds of things that appear in an article without running complex heuristics on the DOM: we’d know not to place an ad after a dropcap — which would look terrible — because it’s labeled as a paragraph with a dropcap.
One of our big success stories as a team is taking the friction out of AB testing complex systems. If a machine is easy to tune and test, we can move fast on all sorts of optimizations and experiments.
This was top of mind as we designed the new system. In the old system, we had to update the core business logic to change the injector, update the unit tests to match it, and even small changes could be fraught and challenging.
The new system would separate out the core logic, which was complex, from the configuration, which would be friendly, approachable, and easy to tinker with.
Powered by a set of rules
The goal of this system is to spread injected items out appropriately and avoid putting them in positions that don’t look very good, and the configuration reflects that. It’s made of a list of rules, each of which specifies where an injected item can’t go.
As of August 1, the current rules are:
- Injected items must be at least 150px below and 100px above a figure (image/video)
- Injected items cannot be adjacent to a blockquote or pullquote
- Injected items cannot be immediately after a heading
- Injected items cannot be immediately before a list
- Injected items cannot be immediately after a “salutations” paragraph that ends with a comma or colon
- Injected ads must be about 1 browser height after the last injected ad but not lower than 700px (to avoid too much density on mobile)
- Injected house ads (newsletters, podcasts, live events) must be at least 1/2 a browser height after an injected ad
- Any injected item should be at least 200px from the end of the article
- Injected items must be 250px after an injected recirc module, which floats
- Any injected item must be at least 250px away from non-injected recirc units
- The first injected item cannot be immediately before a dropcap paragraph
- There must be at least 320px between the top of the article and the first injected item on mobile, and 200px on desktop
The rule syntax is small and designed to be scannable, but it can do a lot. Each rule describes a type of item we will insert, a type of top-level component it needs to stay away from, and how far away it needs to be from above or below.
Here’s an example:
{
item: InjectedTypes.Any,
after: "150px",
before: "100px",
from: ArticleElementTypes.Image,
}So, this is saying that any injected item, regardless of what kind, should be at least 150px after and 100px before a figure element.
There’s a lot of power in this.
Many of our requirements are around avoiding direct adjacencies. For example, we don’t want to inject anything between a section heading and its first paragraph.
We can describe this by telling the injector to ensure it’s at least 1 pixel away from the bottom of the heading.
{
item: InjectedTypes.Any,
after: "1px",
from: ArticleElementTypes.Heading,
}With this rule in place, the injector may place an ad directly above the heading — that looks fine — but if it checks the spot directly below the heading, it will see the distance is zero, and skip forward to the next position.
Finally, since the rules can contain any logic that can run in the browser, when you need more complex logic, it’s available. We want the ads to be at least one screen apart from each other, but we also don’t want to overcrowd the content on small phones.
We can set the rule to use the larger of either the screen’s height or 700px.
{
item: InjectedTypes.Ad,
after: `${Math.max(parseToPx(“100vh”), 700)}px`,
from: InjectedTypes.Ad,
}Comparing notes with our editors we discovered quite a few cases where ads were appearing directly after the line “Dear Therapist,” which appears all the time in Lori Gottlieb’s popular series where she responds to letters. That’s fairly disruptive and just looks awful. We added a rule to describe any paragraph that ends with a comma or colon as “Salutations,” and could then instruct the injector that it should never place anything directly between that line and its opening paragraph.
The rule looks just like the one for headlines.
{
item: InjectedTypes.Any,
after: “1px”,
from: ArticleElementTypes.Salutations,
}Because the system is so flexible, we could eliminate that edge case as fast as we could describe it.
The algorithm
To use these, the injector needs to break the article into a format it can understand.
We’ve parsed our article into a list of top level components and their Y top and bottom positions, which we can loop over to decide where it’s safe to place an injected thing.
Our injection algorithm runs through the article once from top to bottom. After each element in the article, each paragraph, each image, it’ll ask, “can I inject something here?”
First, it checks what it should inject — in most cases an ad, but sometimes a recommended reading module or newsletter promotion.
From there, it checks the configuration to see if this injected item is allowed at this position. For each applicable rule, it looks above and below the position to see if it’s far enough from the type of element it’s avoiding, comparing the difference in Y positions to the distance required. If it’s far enough away, the rule passes. If it finds what it’s looking for, it fails, and skips on to the next position.
If all the rules pass, it will place the ad into the array of components, and move to the next step.
One last snag: CSS floats
There’s one more complicating factor. While it’s nice that our article is a simple list of elements that are not nested, they can be overlapping each other. For example, say an image floats alongside the article body.
If we try to place a full width ad in between element 1 and 2, that will make a mess of the page.
At the start of the process, we’re already recording the vertical coordinates (Y positions) of every top level component on the page. We can loop through those elements, comparing their top and bottom Y positions, and determine if any elements are overlapping each other. These form “Overlap Blocks” — sections of the article that nothing can be injected into.
In the server-side injector, there was no way to know with confidence how far down that image would hang. We avoided designs like this because they were a source of headaches. With the new injector, we can measure it and respond to it with precision.
An injector hums to life
Two months out from launch, it’s safe to call this project a wild success.
The new injector is smarter. Being a constraint-solver with pixel-level precision, it aims for one ad per pageview, with a hard lower limit (be nice to readers with small phones), creating a clean experience for 27” monitors and miniature phones alike.
Its higher precision has helped us compensate for the loss of ads in the right rail, all but eliminating revenue downsides of that design change without feeling like we’re overpacking the page.
We’ve been able to add more nuanced rules to prevent awkward placements and unintended side effects that hurt the design.
Meanwhile, our emphasis on making the system approachable and easy to modify has paid off. We’ve made a dozen adjustments to the configuration based on real-world observations. We’ve released our first major AB test to measure the behavior of our newsletter subscription modules. Engineers on our Story teams have been able to tap into the framework to include new book and magazine issue modules in thoughtful and intelligent ways.
Thanks to the flexibility and precision of the new injector, we could make almost all of those changes at the drop of a hat.
The ability to change the system so quickly and see what the implications are is a breath of fresh air to developers, and the key to being able to move fast, act on data, experiment more and improve the results.
Having that level of speed and power opens the door to a whole new range of capabilities and pursuing the grand theory of Unified Yield, where we can intelligently place the right elements in front of the right people at the right time, optimize for business needs and sustain a stunning reader experience while doing it.
