We spoke to McKinley Muir Hydén, the FT’s Lead Data Analyst of Audience Engagement, about the newspaper’s engagement metric and the challenges related to measuring online audiences generally.
WAN-IFRA: Can you guide us through your model?
McKinley Muir Hydén: Our company-wide model is RFV, short for recency, frequency and volume. It looks over the last 90 days to see how recently a reader visited us, how many times they visited, and how much they read over the period. We also have a range of metrics that try to balance each other, including pageviews, scroll rate, bounce rate and time on page, and how many people went on to read another article. RFV pretty much powers our commercial side. We actively report on it, and we use it to qualify initiatives and testing. We also have further models based on it, and it works for advertising, B2B marketing, and so on.
Our model works perfectly for our commercial side, but for editorial, much of the decision-making needs to be made in real time on specific pieces of content. A metric that’s at the user level, calculated over 90 days and updated weekly, isn’t conducive to that. In the long run, commercial and editorial are both interested in establishing loyalty – but in editorial we also need supplementary metrics to help us make quick decisions about content.
How did you go about creating the model?
When we were rolling out the analytics tool, we were cautious about the reaction in editorial, so we decided to focus on adoption and make an iterative and flexible tool. One major goal for us was to educate people internally about the limits of the pageview, which still rules in newsrooms. We’ve made quite a bit of progress there.
Now we’re looking at the next iteration of the tool. We have the metric now, but looking at it can also lead to confusion about what you prioritise. You need to have a story. A big challenge is to be able to tell a better story about what engagement looks like with a particular article.
How would you like to develop your model further?
We definitely need to tie RFV to our editorial efforts. The whole company is supposed to move towards maximising user loyalty, which is defined by RFV. So we need to make RFV useful for editorial, to help them move towards this goal and understand the role they play in advancing the cause of getting more engaged (as defined by RFV) readers.
A key aspect of making any metric useful to editorial is making it real-time, so we need to figure out a form of RFV [recency, frequency and volume] that can do that. They need to be able to make decisions quickly, on the spot, and RFV should help with that.
We’re going to look at questions such as these:
- Do you have a bunch of anonymous people coming, or do you have engaged subscribers taking their time with the content?
- Are you missing out on people who usually are loyal?
Also, one big question coming from editorial is if we have sense of what is useful or ‘liked’ content. It would be helpful to have some kind of feedback on what readers found relevant and useful, or what they enjoyed.
We’re also trying to develop a supplementary metric to complement RFV that would focus on operational outcomes, and specify what concrete action should be taken. For instance, do I adjust metadata? Or headline? Placement on the website? And on a more strategic level, we try to be receptive to how people interact with us and draw conclusions about our content based on that. Of course it’s key to make sure that any supplementary metrics have some correlation with RFV.
What about social media? Is that accounted for?
There are a lot of ways that people can engage with us that we can’t quantify, especially concerning social media. Facebook is not particularly brilliant in allowing us to measure what we want to measure. It’s kind of impossible to understand the viral aspect of anything, for example, whether pageviews originate from readers sharing our pieces on social media or from our own promotion there.
We can measure pretty much anything on our site, but with distributed content, the battlefield is not at home anymore.
Everyone in the business is interested in this, wanting to have a sense of the influence and the influencers. It’s so piecemeal at the moment.
How do you see collaboration going with social media companies?
We tried Instant Articles, but decided to stop. It was important for us to tie RFV with Facebook’s platform, but given that we couldn’t, it was unclear why we would want to participate without evidence of benefit. We tend to assess this stuff on a case-by-case basis, keeping our interest in mind.
Any comments about off-the-shelf solutions for measuring engagement?
Some offer a cool package, but a constant theme with the FT is that we like to do things by ourselves. When we have a need for something, we tend to build it.
It’s also individual, a bit like choosing a perfect life partner! Everyone has got their own preferences. So for a lot of companies, an off-the-shelf solution is a great way to kick things off, and it does depend greatly on the nature of the product. We feel that we need more tailoring than what’s otherwise possible, for example in measuring paywalled and non-paywalled content.