DCN v2

When an online shop suggests a product to you, there’s a prediction behind it. A computational procedure estimates how likely you are to click on that suggestion. To do this, it evaluates many individual pieces of information: your device, the time of day, the country, the product category, your recent clicks. What matters most, however, often lies not in a single piece of information but in their combination. “Evening” and “smartphone” and “sporting goods” together say more than each piece on its own. DCN v2, spelled out as Deep & Cross Network Version 2, is a model architecture developed at Google in 2020 that deliberately forms exactly such combinations instead of hoping the model finds them on its own.

Why Google had to rebuild the predecessor

Recommendations are the business itself for many companies. For an advertising provider, click prediction directly determines revenue. Even one percent better accuracy can mean millions. That’s why research into these models is as intensive as research into language models, just with far less public attention.

The predecessor version, DCN, from 2017, had a well-known problem. Its combination component was built in a mathematically very sparse way and could therefore only capture limited relationships. In practice, simpler competing models regularly beat it. DCN v2 replaces this component with a more expressive variant and performed better in Google’s tests, both on public datasets and in real-world operation.

A second point matters here too: computation time. Such models must deliver millions of predictions per second, often within a few milliseconds. A model that is only more accurate but significantly slower would be useless. DCN v2 was designed from the outset to fit within this tight time frame.

Cross layer and deep network working together

First, all the pieces of information are translated into lists of numbers, so-called embeddings. “Product category: running shoes” thus becomes a series of numbers that a computer can compute with. These lists of numbers are concatenated and form the input for the actual model core.

The core consists of two parts. The first is the cross network: it multiplies the input with itself step by step. After one layer, pairs of features emerge; after two layers, triple combinations; and so on. The degree of combination thus grows in a controlled way with the number of layers. The second part is an ordinary deep neural network that learns freer, harder-to-describe patterns. Both parts can be arranged side by side or one after the other.

The actual trick of version 2 lies in a matrix, that is, a table of numbers, present in each cross layer. The predecessor used only a single row of numbers there and was thus heavily restricted. Because such tables become large and expensive, DCN v2 decomposes them into two smaller tables. This decomposition significantly lowers memory requirements and computation time without noticeably costing accuracy. A common mistake, incidentally, is confusing DCN v2 with the identically named component from image processing: Deformable Convolution v2 is also abbreviated DCNv2, but has nothing to do with it.

Invisible in the feed, visible in expert discussions

As a user, you never see DCN v2 directly. At most, you notice it in that suggestions in a shop, a video app, or a news feed fit surprisingly well. Google reportedly uses this architecture in several products, and it is freely available via the TensorFlow Recommenders library.

In job postings and technical articles, the term appears wherever click prediction or ranking is discussed. It usually appears alongside related names like Wide & Deep, DeepFM, or DLRM. All of them solve the same task with slightly different components. Anyone reading such reports should know: this is not about chatbots, but about the much older and economically very significant discipline of recommendation systems.