This Adam Mosseri talk about how Facebook uses data to make decisions is a little dated but his observations are still extremely useful. His key insight: clear metrics and strong data-driven feedback loops can be powerful, but they have their limits as well. Facebook often uses solid empirical data to make decisions about their website design, their products, and the workflows that users experience on Facebook. They can test two versions of a website design, for example, and if design option A produces higher engagement than design option B it’s an easy choice.
But Mosseri also explains how an excessive fidelity to data-driven decisions can privilege incremental and uninspired changes at the expense of innovation and ambitious thinking. Facebook sometimes is aiming not only for high levels of engagement but for more fundamental changes in the way people interact with it and with each other. Facebook’s Timeline, for instance, inspired anger and fierce resistance among many Facebook users and sharp derision from the press, and the use of a conventional data-driven decision process would have killed it before it got very far, but Timeline is now a central and deeply-valued part of the Facebook experience.
Most nonprofits don’t seem to rely much on data for their decision-making about their websites, email newsletters, programs, and fundraising efforts, and when they do those efforts aren’t often carefully crafted and executed (some do, of course, but for every one that does there are many, many more that don’t). The remedy isn’t to swap all the intuitive and qualitative decision-making for analytic feedback loops, but to find a good balance. “Data informed, not data driven,” as Mosseri says.