Tips for Data Analysts – be a Shepherd (not a sheep)

So you think you’ve suffered a back injury trying to pull off a particularly advanced yoga pose.

You head over to your doctor, and without saying a word she plops a thermometer in your mouth. “98.6. You’re fine,” she says and walks out of the room. “No! data are simply proxy measures for quantifying imprecise outcomes which cannot be counted” you scream as you lie there on the cold paper bedsheet, writhing in pain. Thanks a lot, Big Data.
This fable has a worthwhile moral: we must be cautious about being led astray by data. Data are nothing more than indicators or proxies towards some larger, more nebulous ideals. For instance, how do you measure impact? Or awareness? Or overall health? These outcomes don’t always translate nicely into a digestible metric. When we fail to recognize this, we run the risk of chasing data simply for their own sake and with no larger goal in mind. We must be diligent in not allowing data to become our shepherd. Otherwise, we become data sheep (patent pending).
data sheep data shepherd
Below are some tips for driving insights from your data:

Conceptualize the Data

So your organization is incorporating more data into its strategy. Good. But, as the metrics pile grows higher and higher, the outcome associated with each metric gets lost. Bad.
Sadly, there isn’t a singular metric which tells us everything we need to know. Different metrics are all richly interrelated and all tie into making an approximation towards a certain outcome. Say we are trying to increase awareness of a certain health issue, and as such, make it the topic of a newsletter. So, you say, let’s see how many people open the newsletter and learn about the issue that way. If we can just increase our open rates, we’ll be fine… But wait, there would be more people to potentially open the email, if we built up our email lists. So let’s work on improving signup rates on conversion pages! But wait, there would be more people landing on these conversion pages if we improved our click-thru rates for our advertisements. And on and on. Understanding this connection between different metrics is vital towards understanding your outcome. Relying on one metric or a solitary key performance indicator (KPI) is simply irresponsible. As our buddy Avinash says, give each metric a BFF.
Picture 4

Beware of Aggregate

KPI’s are great for surface level knowledge, but you should always dig deeper to really find any insights. If your analyst is only presenting data in aggregate form, he is slacking, and worse yet, misleading your organization!
Imagine you get a report which says that your website’s monthly total of users is higher than last month’s. Immediately, champagne is popped and you give your workers the day off. But, a more segmented analysis reveals that while your total users may have gone up thanks to a hugely successful Adwords campaign, the number of users from referral and organic traffic have both fallen since the last month. What seemed like one outright success, when broken down, actually proves to be one success coupled with two failures. Now, instead of being able to put on auto-pilot, you get to work on fixing your two lagging traffic sources. You call your workers back to the office…everybody except your lazy analyst, who is promptly fired.

Think of numbers as people

One of the strategies we constantly promote is “putting the people back in the data.” That is, we try to imagine each user or site visitor or subscriber as a living, thinking, breathing human being. Saying “we increased new user signups by 7%” doesn’t sound awfully sexy. But when you rephrase it as “there are 150 more humans benefitting from our service,” the impact becomes much more real and tangible.
When one of our clients was getting lost in data about their donations, we needed to think up a way to more tangibly convey information about their typical donor. Our ever-pragmatic data analyst came up with the idea of sharing a picture of an elderly lady to help bring the data to life. Undeniably, this image resonated with our clients, and got the message across much better than bar graphs or pie charts could have. Data visualizations can be created in a wide range of ways and are powerful tools for driving impact. It sounds kind of blase, but is important to always remember that numbers and metrics represent people and human-driven actions.
put people into data

When is 10,000 bigger than 20,000?

An important thing to remember when analyzing web traffic is that not all users (visitors) are created equal. Engagement rates and conversion rates often differ dramatically between sources of traffic. For instance, organic traffic tends to have great time on site, while referral traffic often has high conversion rates. And, even within a single source of traffic, like Adwords, there exists variation in the quality of these users.
Now, a cautionary tale: One of our clients was dead-set on maximizing their Adwords traffic to the fullest extent of their Google grant (thank you Google for supporting non-profits!) Week after week, their cost-per-click (CPC) continued to fall and the number of Adwords session continued to increase. They were doing well. Or, so they had thought. You see, they were an organization based in California promoting specialized training for art teachers. But, the traffic they were getting was from all over the world, especially click-farm heavy areas such as the Philippines and India. These users simply clicked on the ad, and often submitted their information on the landing page. However, these users were essentially worthless as the service our clients provide do not have much value for users outside of America. As a result, we advised them to apply search filters and limit traffic to users in the USA. This caused CPC to go up, and the number of overall Adwords users to go down. However, the important thing was that they were getting better quality traffic- users who they could actually impact through their product.
Picture 3

Conclusion

Have you heard the story about the group of Japanese tourists who drove their car into the Pacific Ocean because that’s where their GPS told them to go? Don’t be like those tourists! Human knowledge and expertise must always go hand in hand with data. Put data into words. Contextualize your metrics. Understand how the given numbers directly connect to your goals and outcomes.
It is important to always be asking, “Why is this metric important to our organization”? And, if the answer is, “Well, it’s nice to know,” then you might be a data sheep. “It’s directly connected to our mission” and you, my friend, are a data shepherd.
If you need some help download Whole Whale’s Ultimate Nonprofit Dashboard Startup Pack to get ready to use Dashboard, Custom Reports, and Segments to help you delve deeper into your data.