Bright+3

Bright Ideas Change the World

Measuring Impact

Testing, testing, testing…is this message on?

by

So you’re sitting there lamenting the somewhat lethargic results of recent email campaigns and wondering if a little tweak to your email or landing page would improve results. Maybe change up the subject line – add or remove the organization. Maybe add a photo or two to the message. Maybe change the placement of a link or form or call to action on a landing page. Would that get more conversions you wonder? There has to be an easy way to bump this up, you think to yourself.

Photo by Sebastian Bergmann, flickr.So you post an email to a handy helpful email list largely made up of folks doing similar work asking if a subject line change would help. The feedback is extensive but largely anecdotal. Hardly anyone offers up actual data and most of the stories are second or third hand… “well, a group I used to work with put the name of the organization in the subject line and it helped a little, I think.”

And you think, “well, that good but it’s not exactly the same situation I’m dealing with here. It’s a good story but doesn’t exactly apply to my list.”

My god, man… then why not test it on your list!

The thing is, testing on one’s own list and pages is pretty darn easy (though we can make it quite complicated and involved, sometimes for good reason and other times not) but rarely done.

Okay, so the ease of testing depends on the tools at hand. If your email system/online CRM is pretty unwieldy or you just don’t know how to use it then little tweaks here and there can be massive potholes in the road, not small bumps. If you don’t know how to move things around on your site – or have the staff to do so – then little changes can be tough.

Yet what I’ve found is more often a lack of interest or curiosity about testing. More often, folks are resigned to the results they get or, if not sure, just don’t know how to proceed. What to test? How to set it up? Is it worth the time?

Valid questions all. But I think the lack of a learning culture is more the culprit. More on that in a future post but first… what to test. [Read more…] about Testing, testing, testing…is this message on?

Social Media Applied: NationalField

by

There is currently a LOT of talk in nonprofit and advocacy campaign circles about social media and how it can be used in campaigns and more generally in support of organizational goals. Approaches are varied but gradually becoming more sophisticated. Some organizations are putting resources towards integrating social media into staffing, metrics, program plans, membership and more.

Campaign and organization leaders are pressured to “do something” with social media. Pressure that filters down. Meanwhile, program leadership – the folks with day to day responsibility for budgets and staff time and “making things happen” – are wondering how to apply social media in a practical sense. How does it affect the numbers? These are, generally speaking, risk averse people and organizations. For them, models, examples, templates and guidelines are an imperative.

So along comes NationalField and it seems a great opportunity to take a close look at this social media software platform that evolved from the 2008 Obama campaign. TechPresident had a great post on NationalField last week including an interview with Aharon Wasserman, one of the founders.

In a nutshell, NationalField brings a social interaction and metrics platform to field work. It seems, primarily at least at this point, to be a closed/private network for campaign staff and volunteers. Field workers interact, sentiment is gauged and relationships are built. Strong stuff for an effective field operation. [Read more…] about Social Media Applied: NationalField

Be More Like a Business!

by

Nonprofit or for-profit, this organization's revenue trend has a problem.
Nonprofits hear the “be more like a business” refrain a lot, despite plenty of evidence that being like a business isn’t all it’s cracked up to be.

I’m persuaded by Jim Collins’ analysis, which more or less argues that within the business community you’ll find the entire spectrum, from extremely effective organizations to highly dysfunctional ones, and the issue is ferreting out the practices that make the good ones good. He extends this to the social sector, and essentially makes the same argument: the challenge isn’t for social sector organizations to be like businesses but to adopt the practices of the really well run businesses, as well as the really well organizations from whatever sector. I don’t think he’s ever evaluated the public sector, but I suspect his argument would be the same . . . learn what we can from the practices of the organizations that really kick ass (and I’ll bet there’s more of this than you might think, but that’s a post for another day).

In other words, conversations that focus on how the nonprofit sector can be more like the business sector are asking the wrong question. A better question: how can nonprofits learn from the best practices of highly impactful organizations within their own sector and across the business and public sectors?

Our frequent insistence on asking the wrong question leads to two kinds of problems. On the other hand, the nonprofit sector is often so allergic to the very idea of the business sector that it runs away from it, and eschews anything that smells like it might have come from the business world (the “we don’t do things that way” reaction). The problem is that there are a bunch of practices you find among the best-run private sector organizations that nonprofits would benefit from using. It’s not “be like a business,” but “adopt the applicable practices of the best businesses.” The most successful of the private sector organizations are often strong at things nonprofits are often weak at: capacity investments, staff cultivation, and staff management among them.

And then there are nonprofits that run in the other direction, thinking that salvation lies in being like a business, and they adopt private sector practices in seemingly wholesale and indiscriminate fashion, which often produces some of the weirdest and worst dysfunctions, since it involves the hybridization of worst practices from both the nonprofit and private sectors.

Nonprofits really are, at root, businesses. They are different in some important ways from traditional profit-oriented businesses, but they are similar in many more ways. Nonprofits live and die on cash flow, for example, and they have to offer a sufficient and appropriate value proposition to their supporters. But the implication isn’t “do whatever private sector businesses do,” since that would be dumb, but rather that the laws of business physics really do apply even if you are serving a social good of some kind. The sense of nonprofit exceptionalism that pervades much of the nonprofit world, the notion that we are special and different because we are a nonprofit, is harmfully misguided. If you apply a discriminating eye toward the private sector, however strong your allergy may be, you’ll find a ton of practices and accumulated wisdom that will help your nonprofit do good, better.

The Nonprofit Dashboard Roadmap

by

Photo by Flickr user istargazer.
Dashboards have been used heavily in parts of the private sector for a long time, but they seem to be making inroads in the nonprofit and government worlds as well. The premise: just as with a car or airplane dashboard, a nonprofit staffer can quickly glance at the gauges and clearly understand the real-time status of key systems and indicators across the organization. The understanding won’t be deep – they may need to probe more thoroughly to fully understand anything they are seeing on the dashboard – but they can very quickly get a sense of how well the organization is functioning and if any problems are emerging.

In our view, dashboards have the following characteristics:

  • They are dynamic. They display information that is changing on a regular basis.
  • They rely heavily on gauges or other data visualization displays to convey information in readily understandable formats.
  • They allow for quick status assessments, although they may enable deeper inquiry

Even with these criteria, however, the dashboard concept is used to describe a diverse range of displays, each distinct in function and design from the next. As the dashboard concept takes great hold among nonprofits we are encountering some confusion among our clients about which is which and what people mean when they use the term.

We took at stab at delineating five distinct types of dashboards:

1) Business Intelligence Dashboards
These display detailed information about a particular area of an organization’s operations. Many customer relationship management systems, for example, like Salesforce and Raiser’s Edge, include dashboards to make it easier for the development staff to track fundraising activities, donations, and other performance measures. Fundraising and advocacy management tools like Convio use dashboards to display campaign status. Google Analytics, with its robust dashboard system displaying key web site metrics, is another example. Technical dashboards help specialized staff keep a close watch on what’s happening within their organizational purview. Technical dashboards are typically inward facing, so that only staff and perhaps board members can view them, but they can be outward facing as well. The Indianapolis Museum of Art’s dashboard is an oft-cited example.

2) Status Dashboards
Organizational status dashboards, like the one the software company Panic described on their blog, are another variant. In contrast to technical dashboards, which tend to focus on a single functional area within an organization, status dashboards display less information from a wider range of functional areas across an organization. A status dashboard is the answer to the question: what is the critical information everyone in the organization should be able to view all the time? Rather than probing deeply into any one area of an organization’s operations, they offer a broader overview of value to everyone.

3) Accountability Dashboards
Increasingly, we are seeing dashboards used in external accountability contexts: a nonprofit or local government that wants to share its real-time performance data with its donors and its community. The Town of Oro Valley in Arizona maintains a financial dashboard displaying the town’s financial performance compared to past trends. It’s not a great example in that it is only updated monthly, and not in real-time, but it’s at least in the ballpark. Over at PlaceMatters (where I spend part of my week), we’ve been doing a lot of work on sustainability dashboards, web-based tools that openly share a community’s performance against its sustainability goals. Incidentally, we described the Indianapolis Museum of Art dashboard as a Technical Dashboard because of its depth, but it really serves as an Accountability Dashboard as well.

4) Tracking Dashboards
These can be inward or outward facing, and typically show visualizations of unfolding data streams in real-time. These aren’t organizational in nature but, rather, are tracking events that are taking place outside the organization. The data stream may have implications for an organization, but it isn’t specific to that organization. Al Jazeera’s “Region in Turmoil” dashboard shows the volume of Twitter traffic by country in the Middle East as a proxy for the level of political activity.

5) Scenario Comparison Dashboards
These are typically designed to compare likely outcomes of a range of future scenarios across a range of key metrics. For instance, MetroQuest uses dashboards to compare multiple regional development scenarios across factors. CommunityViz, a GIS-based data visualization and decision support tool, allows user to analyze the environmental and other community outcomes from a range of land use scenarios, and it uses dashboards to display those outcomes across a range of factors.

We shouldn’t entirely neglect the category of “Displays That Are Called Dashboards But Aren’t.” It probably isn’t useful to use ‘dashboard’ to refer to web pages full of relatively static narrative information, for example. One example is the recycling portion of the Emory University Sustainability Dashboard.

We would welcome your thoughts. Does this seem like the right breakdown? Are we missing anything? What are the terrific examples of each category?

Recipe for Success

by

The Harvard Business Review blog posted a couple of weeks ago on some findings from a massive ten-year study of management practices, and identified three in particular that seem most associated with successful mid-sized companies:  a) ruthless monitoring and continuous improvements across their entire process, b) establishing challenging performance targets for their employees, and c) energetically incentivizing and rewarding high performers and weeding out underperformers.  It’s interesting but not surprising that – as far as we can tell – not one of these practices is pervasive among nonprofits.  It’s easy to come up with excuses for why these practices don’t easily apply to nonprofit organizations.  It’s much harder, and I’ll wager a lot more useful, to figure how they do apply.

Charity Navigator: A New Evaluation System

by

Among the hottest of the hot topics in the philanthropy world these days is nonprofit evaluation, and all eyes are on Charity Navigator in particular as they move from a narrow focus on financial measures to a much broader assessment of organizational effectiveness.  This is good news for the sector, no doubt.  We’ve long believed that a narrow focus on metrics like overhead ratios and program expenditures (core elements of Charity Navigator’s methodology) is not only misguided but really damaging, encouraging nonprofits to focus on how their expenditures look to external eyes rather than on how much they get actually do with those dollars, and creating huge disincentives for nonprofits to make critical capacity investments.  Even worse, those metrics probably don’t tell us anything consistent about an organization’s actual impact or effectiveness.

As Tactical Philanthropy noted in a nice write-up earlier this week, “Charity Navigator is the 800lb in the charity rating space,” so watching them lay out a path for major changes in their rating system over the next couple of years really is a big deal.  Charity Navigator, ironically, probably gets more credit than anyone else for making overhead ratios so important, but under new leadership and increasingly sophisticated thinking in the philanthropic world about evaluating nonprofit performance, they will play a large role in shifting the paradigm.

As they reported at the SOCAP10 conference last month, their new system will consider overhead ratios, working capital (cash on hand), and a liabilities-to-assets ratio but will also focus heavily on third party reviews of organizational effectiveness.  Accountability and transparency will also figure in the rating system.

Charity Navigate presentation on Slideshare.

There’s plenty of opportunity for missteps, but watching an industry giant like Charity Navigator lead this critical cultural change in the philanthropy community is a welcome sight.

The Monitor Institute’s Cool New Data Tool

by

The Monitor Institute has a new data visualization tool designed to help funders see relationships between their funding and grantmaking by other foundations but potentially useful for community planning applications. It’s a very cool tool – it offers clear, clear visualizations of multiple data layers, it’s easy to navigate, and they obviously put a lot of thought into the user interface. This could be a useful tool for mapping all of the organizations that work on a specific issue within a particular community, for instance, or for mapping the relationships between different issues in a community or regional planning process.

On first blush, there are two important elements I’d love to see on the next iteration. One is a Gapminder-type capacity to show change over time. I like that you can change the date range, so you can see the relative longer-term investments across sectors, etc., but you can’t see trends unless you manually change the date from one year to the next – clunky and not easy to do. The trends matter . . . a large investment this year in a subsector or by a particular foundation may mask a downward trend, which would be just as important to notice if you are trying to understand the relationships, seams, and opportunities. The other would be some capacity to show multiple dimensions at once, again like Gapminder does. This basically shows a single set of flat relationships on each screen. In your mind’s eye you can build a less-flat model of how each of the flat pieces relate to one another, but the tool doesn’t really help you do that.

The distribution model is also unclear to me. If their vision is a closed, proprietary system run through the Monitor Institute, well, that might be useful for whichever funders want to play ball, but it’s a lot less useful to the rest of us. If their vision is a stand-alone, self-contained tool, I can picture a lot of very useful ways organizations could use the tool to help them map their landscapes more effectively and more clearly through strategy.

And as the Philanthropy 2173 blog reminds us, with every data visualization tool you have to ask about the data themselves. Garbage-in-garbage-out is still the rule no matter how cool the visualization tool is.

Cross-posted on the PlaceMatters blog.

Why Will This Work?

by

The notion that a campaign or program strategy ought to be based on a coherent idea about why those activities will produce the desired results seems pretty straight-forward.  In fancyfunderspeak, it’s often called a theory of change or a logic model.  But there’s a debate simmering between the theory of change proponents, like Paul Brest of the Hewett Foundation, and the skeptics, like Albert Ruesga at the White Courtesy Telephone blog.

Brest’s basic argument (as he articulates in “The Power of Theories of Change”) is that it’s tough to expect donors to donate if the grantee can’t explain why their work is likely to lead to the desired social change.  The skeptics’ rebuttal seems to be that this constitutes funder overreach and that requiring an explicit theory of change creates a great deal of anxiety and extra work without any real added benefit.

Ruesga, for instance, argues that a clear theory of change is implicit in most grant proposals (“Debunking Theories of Change” Part I and Part II).  His worry is that requiring an explicit theory of change is likely to either confuse grantwriters or send them down rabbit holes of formal logic and dissertation-worthy socio-political models (and yes, in case you were wondering, he includes Carl Hempel citations and linked syllogisms to make his case).  If that’s really what a funder is seeking, I’m likely to going to side with the skeptics.  But plenty of grant proposals, in my experience, really aren’t built on a coherent rationale for why these activities might produce those results.  Waving off a requirement for an explicit theory of change would be like dropping the expectation for a grant proposal to clearly describe the goals of a project or to outline how the grantee will evaluate success.  If you are one of those funders who doesn’t care about evaluating grant outcomes, more power to you, but you ought not drop the requirement for reporting outcomes simply because you think most grant proposals already cover this.  Sure, some grant proposals will include clear goals, a cogent theory of change, and an evaluation mechanism, but many don’t.

I think I get the skeptical instinct here.  Funders can – and sometimes do – go way overboard.  But asking prospective grantees to explain why they believe their work will succeed seems pretty reasonable to me.

Now, there is a much deeper – and I think much more interesting – debate lying barely beneath the surface here about how involved funders themselves should be in crafting the strategic vision around a movement or social change effort.  Who is the steward of the strategy, and what does that mean for the relationship between funders and nonprofits, is a landscape-shaping discussion.  The question has a sibling, or at least a cousin, that gets at the extent to which funders should focus on organization- and capacity-building versus actual social change campaigns.

And these two questions both beg other, even more difficult and more important questions, like why do so many nonprofits doing such important work allow themselves to become so dependent on such challenging funding sources.  A foundation can ask for whatever the heck it wants, after all; if you don’t like their requirements then you can pitch elsewhere.  If you find the expectations of the philanthropic community to be unreasonable, then you might be better off developing a different business model altogether.

As a donor, a funder, or as a potential campaign partner, for that matter, I care a lot that you’ve got a coherent rationale for why this particular suite of strategies is likely to produce the change we are targeting, or, if we are feeling more risk-tolerant and experimental, why we think these untested approaches are worth the effort. If we really are getting stuck on the idea that grant proposals should clearly explain why the organization believes its proposed efforts will produce the desired outcomes, well, that’s not encouraging.

And a hat tip to Tactical Philanthropy for the links.