“When inaccurate analytics occur, it’s a huge issue as tracking reader engagement is absolutely essential to a media organization,” Morgante Pell, a software engineer at Business Insider, said in an email interview. “Yet, it’s fairly easy to cause inaccurate analytics because most systems rely on a client-side code which can easily break if slight errors are introduced.”
Realizing this issue, Business Insider created Haunted to facilitate analytics validation. A dedicated BI staffer built the tool this month over a period of two weeks, and it’s now available for other news organizations to use.
“We rely on accurate analytics throughout our organization, from the editorial team to ad sales,” Pell said. “Yet, as we work to constantly improve the digital experience for our readers there’s the potential for new issues to be introduced which compromise the validity of those analytics. So we needed a tool to catch any such issues before they’re deployed to production and compromise our numbers.”
Pell pointed out that slight changes to site formatting, such as slideshow display, can lead to miscalculated analytics, like tracking pixels that only fire sometimes. “Most of these flaws occur because different systems are interacting (ex. our ad server and a 3rd-party analytics service) and need to respond to changes in the code,” he said.
And while most news organizations run tests to ensure that their analytics systems remain accurate after changing code, even these trials can miss subtle, sporadic errors that affect tracking data. Haunted aims to catch even these errors before taking new code live.
Basically, Haunted works by loading a local “headless” (invisible) version of BI’s website and going through the same motions a normal user would: reading stories, clicking through slideshows, etc. All the while it’s checking to make sure tracking pixels are being called as they should be and that the number of pageviews for each story read increases accurately. If Haunted finds an error, it’s up to a tech team to comb through new code and figure out which lines may be responsible for the anomaly.
Pell emphasized the importance of news organizations having a plan for catching analytics anomalies and testing changes before they go live as Haunted can do. Media outlets can now make use of the tool, available on GitHub.
“We know that this is an issue which other organizations have to deal with and thought it could definitely prove useful,” Pell said. “At Business Insider, we rely on a wide variety of open source software, from MongoDB to PHP, and this served as a great opportunity to give something back without compromising our core competencies.”
For more information about how analytics are affecting editorial decisions, look out for WAN-IFRA’s upcoming Shaping the Future of News Publishing report on data.