Bright+3

Bright Ideas Change the World

Measuring Impact

Measure what Matters

by

It goes without saying that data helps people make decisions. When deciding how much to pay for a house you want to know the sales price of comparable homes in the neighborhood. When looking for a place to go out for dinner you might look at the average star rating on Yelp.com. This data gathering is tied to a personal goal: paying the right price for your new home and finding a yummy dinner.

But many nonprofits gather and analyze vast amounts of web analytics data that doesn’t help make decisions and isn’t tied to organizational goals. Reports are prepared that talk primarily about pageviews from one month to the next. Detailed reports may display pageviews for specific pages instead of the full site. You may see the number of visits that come via Google and perhaps the search terms used.

Possibly, year to year goals for websites may discuss increasing overall pageviews by 10% or some other amount.

What’s missing? Data that ties to outcomes so that actionable decisions can be made about content, design, search optimization, advertising and so on.

Often, what I don’t see is leaders that know how to ask the right questions about web analytics and related online strategies. It is taken on faith that rising pageviews means success. What’s missing are analytics tied to measurable outcomes driven by organizational goals. These are, as Avinas Kaushik calls them, “faith-based initiatives.” This is an unfortunate way to base resource decisions.

A smarter approach is to focus on Key Performance Indicators (KPIs), preferably those tied to program and organizational goals. This doesn’t need to mean an extra layer of data, analysis and review (aka more work for everyone). But it does mean agreement on asking the right questions and being willing to base tactical decisions on what the data is telling you. In other words, if your goal is to increase readership in California and your blog posts on California issues aren’t getting more pageviews then consider adjusting content.

Put another way, increasing web traffic has nothing to do (in and of itself) with your website meeting organizational goals. If 500,000 people a month view your site but nobody shares content, makes donations, comments on your posts, signs up for your email lists, buys your t-shirts, writes blog posts relating to your content or otherwise acts then are you any better off than you were when 50,000 people a month viewed the site and nobody did anything?

If you are better off then how do you know?

It’s that “how do you know” that counts, right? What are the actions that people are taking as a result of viewing and engaging with your content that matter and how do you measure those?

Perhaps a goal is to increase your reach in California or west coast in general because you are working with key members of congress on an issue that affects trade (or the environment or whatever) in these states. You can hone in on metrics for pageviews (preferably of a certain type of content) from that region. Are they rising during a key timespan? Better yet can you identify why these pageviews are rising and if the rise is tied to visits from AdWords campaigns, content in regional blogs to which you have contacted or another action intended to drive more traffic?

In this case a KPI is still as simple as pageviews but it is tied to reach, specifically your organizational reach in an area key to a program goal.

Another key performance indicator helpful to nonprofit organizations would be items in the category of “conversion.” Are your site visitors doing something after seeing your content that A) you can measure; and B) helps meet at least one of your goals. It is one thing to say that a blog post is getting 500 pageviews when similar posts used to get 300 page views. But it is rare that this tells you much about how (or even if) you are moving the needle towards changing policy (or raising money or building an engaged constituency).

To measure a conversion metric think about what you want people to do after visiting the page. Should they go to a next page with more detailed or related content (a lightweight conversion but an indicator of interest)? Should they subscribe to an email list? Should they share photos or a link on Facebook? Should they comment on the post? Should they make a donation or purchase an item?

If visiting a page is point A then what is point B? Most analytics tools will help you attach values to that step B and/or let you see user navigation paths.

Don’t be disappointed if conversion rates are low. Really low. Two percent can be considered a solid conversion rate.  The reality is that people are looking information and not wanting to be “converted” to what you need. It’s like door to door canvassing. Tough but results count and you can learn from results and try to improve your technique. Test tweaks to design, page layout, headlines, related offers or content.

Measuring Impact

by

Measuring the results of a nonprofit’s campaign can be really important to measuring and maximizing your impact, as Ted wrote a bit about yesterday (“Getting to Stories With Metrics”).  It can also be a tricky business.  Aside from needing to overcome what is often pervasive resistance to the idea, and allocating the resources to do a good job, you also have to figure out what to assess and how to do it.

The easy way to pretend to measure the impact of a training program, for example, would be to report on the number of people who participate (yea, I’m picking on Ted a little for using this as an example in his post yesterday).  Although it might be useful for understanding the scope and reach of your program, alone it doesn’t tell you anything about how good the trainings are or how much impact the trainees themselves will subsequently have on your issues.

One national nonprofit we know goes to the trouble of assessing people before they begin the training and again three months later.  The surveys are subtle and sophisticated – they don’t rely on simple self-reporting but probe in more oblique ways – and the result is a numeric rating for each individual indicating their level of activity and leadership.  Their program evaluation, then, is based on the extent to which their trainees move up the scale as a result of the program.  Perfect?  Nope.  Simple?  Not really, since it takes some real effort to conduct each assessment, and tracking folks down three months after the training can be time-consuming.  Worthwhile?  It sure seems to be, because it allows them to track the real impact of their program and to better finesse both the program itself and their marketing.

Getting to Stories with Metrics

by

Jeff Brooks is the man behind a great blog called Future Fundraising Now. In a recent post he discussed what performance metrics donors are looking for from the nonprofits they support.

Brooks’ thesis is that donors give primarily for emotional reasons and while metrics aren’t irrelevant donors aren’t seeking to connect with a cause or organization on data-driven, analytical levels. Stories about the people involved in and benefiting from the organization’s work (work funded by the donor) fuel the emotion that engages people to give, volunteer, fan organizations on Facebook and spread the word.

Yet Brooks doesn’t dismiss metrics one bit. With respect to stories he writes of good data gathering:

“You’ll get better stories. A system of gathering metrics will put you in contact of what donors really want: stories. And that leads to better fundraising.”

Metrics can tell a story about stories. Donors want stories.

But how do we learn about our storytelling from metrics? There is a ton of data out there. Too much. What can help guide us, inspire us to write better stories for our donors and others in our audience?

We can look at feedback metrics to gauge interest in our work. Some of these measurements also come with commentary that can give insight into the quality of the work. These metrics might be, for example:

  • pageviews of a blog post or other web page,
  • comments on a blog post, and
  • retweets, facebook shares and other social media discussion

If you don’t think that your content is generating the sort of reader numbers or discussion that you expect it could be a sign that you aren’t telling good stories and engaging people in your work through the content. Growing pageviews and comments in a certain type of content or subject area could indicate that those stories are interesting your constituents and may be good topics for further stories and fundraising efforts.

What about programmatic metrics? We can look at data measuring the type, quality or quantity of programs the organization provides to flesh out stories about that work. Here it is going to depend on your work but tying programmatic metrics to the people (and places) you have helped will strengthen stories.

  • How many people have been trained at job training sessions? How many participants are now in the workforce? What is a story of one or more participants about their experience and way life has improved?
  • How many meals were put on a table by your food bank? How many people have access to more local food with better nutritional value through your inner city slow food project? What are some of the stories from participants about how this has made them more independent or better able to feed their family?
  • How many people are receiving calls and emails encouraging them to attend a county hearing on natural gas drilling in the area? How many showed up? How many spoke? How many are now engaged in ongoing efforts to improve energy production in the area?

We work in a time when it is literally possible (almost) to drown in data that measures our performance. Your supporters don’t want to see it all. They will love you because you do great work that changes lives. Focus (for your donors) on telling good stories informed by solid metrics. If you want data for your accountant that may be something altogether different.