Measuring Impact

The Monitor Institute has a new data visualization tool designed to help funders see relationships between their funding and grantmaking by other foundations but potentially useful for community planning applications. It’s a very cool tool – it offers clear, clear visualizations of multiple data layers, it’s easy to navigate, and they obviously put a lot of thought into the user interface. This could be a useful tool for mapping all of the organizations that work on a specific issue within a particular community, for instance, or for mapping the relationships between different issues in a community or regional planning process.

On first blush, there are two important elements I’d love to see on the next iteration. One is a Gapminder-type capacity to show change over time. I like that you can change the date range, so you can see the relative longer-term investments across sectors, etc., but you can’t see trends unless you manually change the date from one year to the next – clunky and not easy to do. The trends matter . . . a large investment this year in a subsector or by a particular foundation may mask a downward trend, which would be just as important to notice if you are trying to understand the relationships, seams, and opportunities. The other would be some capacity to show multiple dimensions at once, again like Gapminder does. This basically shows a single set of flat relationships on each screen. In your mind’s eye you can build a less-flat model of how each of the flat pieces relate to one another, but the tool doesn’t really help you do that.

The distribution model is also unclear to me. If their vision is a closed, proprietary system run through the Monitor Institute, well, that might be useful for whichever funders want to play ball, but it’s a lot less useful to the rest of us. If their vision is a stand-alone, self-contained tool, I can picture a lot of very useful ways organizations could use the tool to help them map their landscapes more effectively and more clearly through strategy.

And as the Philanthropy 2173 blog reminds us, with every data visualization tool you have to ask about the data themselves. Garbage-in-garbage-out is still the rule no matter how cool the visualization tool is.

Cross-posted on the PlaceMatters blog.

The notion that a campaign or program strategy ought to be based on a coherent idea about why those activities will produce the desired results seems pretty straight-forward.  In fancyfunderspeak, it’s often called a theory of change or a logic model.  But there’s a debate simmering between the theory of change proponents, like Paul Brest of the Hewett Foundation, and the skeptics, like Albert Ruesga at the White Courtesy Telephone blog.

Brest’s basic argument (as he articulates in “The Power of Theories of Change”) is that it’s tough to expect donors to donate if the grantee can’t explain why their work is likely to lead to the desired social change.  The skeptics’ rebuttal seems to be that this constitutes funder overreach and that requiring an explicit theory of change creates a great deal of anxiety and extra work without any real added benefit.

Ruesga, for instance, argues that a clear theory of change is implicit in most grant proposals (“Debunking Theories of Change” Part I and Part II).  His worry is that requiring an explicit theory of change is likely to either confuse grantwriters or send them down rabbit holes of formal logic and dissertation-worthy socio-political models (and yes, in case you were wondering, he includes Carl Hempel citations and linked syllogisms to make his case).  If that’s really what a funder is seeking, I’m likely to going to side with the skeptics.  But plenty of grant proposals, in my experience, really aren’t built on a coherent rationale for why these activities might produce those results.  Waving off a requirement for an explicit theory of change would be like dropping the expectation for a grant proposal to clearly describe the goals of a project or to outline how the grantee will evaluate success.  If you are one of those funders who doesn’t care about evaluating grant outcomes, more power to you, but you ought not drop the requirement for reporting outcomes simply because you think most grant proposals already cover this.  Sure, some grant proposals will include clear goals, a cogent theory of change, and an evaluation mechanism, but many don’t.

I think I get the skeptical instinct here.  Funders can – and sometimes do – go way overboard.  But asking prospective grantees to explain why they believe their work will succeed seems pretty reasonable to me.

Now, there is a much deeper – and I think much more interesting – debate lying barely beneath the surface here about how involved funders themselves should be in crafting the strategic vision around a movement or social change effort.  Who is the steward of the strategy, and what does that mean for the relationship between funders and nonprofits, is a landscape-shaping discussion.  The question has a sibling, or at least a cousin, that gets at the extent to which funders should focus on organization- and capacity-building versus actual social change campaigns.

And these two questions both beg other, even more difficult and more important questions, like why do so many nonprofits doing such important work allow themselves to become so dependent on such challenging funding sources.  A foundation can ask for whatever the heck it wants, after all; if you don’t like their requirements then you can pitch elsewhere.  If you find the expectations of the philanthropic community to be unreasonable, then you might be better off developing a different business model altogether.

As a donor, a funder, or as a potential campaign partner, for that matter, I care a lot that you’ve got a coherent rationale for why this particular suite of strategies is likely to produce the change we are targeting, or, if we are feeling more risk-tolerant and experimental, why we think these untested approaches are worth the effort. If we really are getting stuck on the idea that grant proposals should clearly explain why the organization believes its proposed efforts will produce the desired outcomes, well, that’s not encouraging.

And a hat tip to Tactical Philanthropy for the links.