Can you trust your engagement stats?
Many organizations want to track detailed statistics about how their users consume their content, such as:
- who read the bulk email or newsletter we just sent out?
- did webinar attendees actually watch the full webinar video?
- how many people have read a PDF document?
- how much time do visitors spend on our website?
Many web service organizations will provide engagement stats that attempt to satisfy this desire to look over users' shoulders and keep track of what they are doing. But you should treat these numbers with a bit of skepticism.
Generally speaking, you cannot look over your users' s shoulders to determine exactly what they are doing. If your users have decided that they don't want to be spied on, then it would be a privacy violation for you to do so, assuming you even had that capability. (And, realistically, you don't have that capability.)
Most engagement stats use simple tricks to guess what the user is doing. They are not actually monitoring the user. Instead they are looking at your web server's traffic stats to see what content is delivered to the user's computer. To track individual users, the URLs that are used to request that content might be tagged with personalized tracking codes.
So you can look at your web stats to see that a video was accessed, and if you are using tracking codes, you can even infer who it was accessed by (assuming that user did not share the link with someone else). Your web stats can report exactly how much data was sent back, and from that you can infer whether that adds up to the complete video or not. But none of that means the user actually watched the video. Video players will preload (buffer) their content, so the fact that your web server (or video service) delivered the full video doesn't mean the video was actually played, or that it wasn't paused or closed before it got to the end. Because things like pausing happen on the user's personal computer, and not on your web server, they are much harder to track, require more invasive spyware tricks to monitor on the user, and are unreliable even with all that extra effort. (There are many popular browser plugins that block tracking software.)
Monitoring who reads your emails is similarly problematic. As with videos, there is no way to tell that an email was actually read. Because email is not on your website, you cannot even say if it was delivered. But there is a trick you can do to determine if an email was opened. This is done by embedding an image with a tracking code into the email. When that image is downloaded from your web server, you can detect that tracking code, and guess that the email was displayed somewhere. But even in that ideal case, that doesn't tell you if the user actually looked at the email, or were just whipping through their inbox, deleting unwanted messages. Most email programs also provide easy ways to disable automatic image loading, since it can also result in slow performance and unwanted data charges on mobile devices. Without image loading, your tracker simply won't work, and your open stats are not going to be reliable. Either way, many emails that show as opened were not read, and many that show as unopened were read.
Many web analytics packages like to report how much time visitors spent at your website. Again, these are just wild guesses, based on tracking codes, cookies, or click patterns. Even if the user doesn't block cookies, it is very difficult to say how long someone is at your site. How long should an analytics package report that someone is on your site, if your site is left open in a background tab all day long? Who is more engaged with your website, someone who visits the same page three times in one hour, or someone who opens the same page in a tab once but leaves it there all week? There is no clear answer, and analytics tools can only make guesses based on their own assumptions.