When CTR Lies About Recommendation Quality

Click-through rate rewards anything that looks interesting for a second. In recommendation surfaces that often means bright covers, familiar titles, or items the user already intended to open—none of which prove the slate was well ranked for discovery.

We pair CTR with downstream holds: save rate, session continuation into the recommended item, and return within a week. When those diverge, the ranking problem is rarely “more clicks.” It is usually stale slots, repetitive neighbours, or a cold-start path that never escapes popular defaults.

Define the decision the slate is meant to support before you celebrate a CTR bump. A home-shelf meant for discovery should not be judged with the same proxy as a “because you watched” row meant for continuation.

Teams in our Abergarw briefings leave with a short scorecard: which surface, which cohort, which hold metric, and which CTR movement they will deliberately ignore.