A Year of Leaning into Learning

In May 2024, the first-ever cohort of Evidence for Change Fellows gathered on Stanford’s campus to reflect on a year spent engaged in a fellowship designed for teams from California-based nonprofit organizations, share lessons learned, and plan next steps. The last time the group had all met in person was in September, to kick off the fellowship. 

In that 8-month stretch of time, fellows participated in workshops and received one-on-one coaching to develop evidence and data plans and put those plans into action. Fellows returned to campus in May now as experts, focused on building a learning culture within their organizations and using evidence and data to constantly test, iterate, and drive the impact of their work. 

The cohort included eight organizations working across the state of California on a range of pressing social issues–each of them grappling with big questions as to how they can better leverage data and evidence. Every organization applied the lessons and tools of  the fellowship in a different way.

• Trabajadores Unidos Workers United sought to identify which component(s) of their work is leading to class consciousness (the ultimate measure of impact according to their theory of change) and how they can promote that across more of their membership.
• The California Work and Family Coalition (CaW&FC) wanted to standardize and structure their data to make advocacy and organizing efforts more systematic.
• AHRI Center developed a theory of change for their internal culture as well as their external programming, and developed metrics to start capturing how effectively they are progressing against that theory of change. 

Though each team focused their energy differently, some shared themes emerged:

1. Theories of change and data have a bad rap! 

Theories of change are often static, abstract models developed for a funder that, instead of driving organizational priorities, instead simply sit on the shelf never to be referred to again. Data collection is often thought of as a hugely burdensome process associated with grant reporting and accountability, primarily focused on outputs rather than outcomes or impact. Data collection efforts might therefore be driven by output-oriented questions (How many youth came to after-school programming?) rather than impact-focused inquiries (Are those youth thriving as a result of the programming?). 

Throughout the fellowship, organizations focused on developing theories of change able to serve as dynamic tools essential for internal learning and focusing and energizing data collection. 

As simple as an ‘If we do x … then y will happen…’ statement, the theory of change guides what a team needs to collect data on to learn about the impact they’re having and how their work is contributing to it. Perhaps it even reveals what more an organization could do to drive that impact. And, despite the bad rap it gets, data collection can be fun! Sometimes it means gathering pictures of trees and song lyrics, rather than adding rows to a spreadsheet.

2. Developing a learning culture and shifting mindsets across the organization to align around what impact-focused work entails is crucial. 

A theory of change can only be dynamic and data collection seen as a gift (rather than a curse) within an organization-wide learning culture! Because of the status quo structure of output tracking, both leadership and program teams are not always incentivised to track their impact and to experiment, learn, and try new things. This manifests in many ways, from how teams are empowered to make decisions, try new approaches and learn, to how performance is measured and the questions that leaders ask of their staff. 

3. Since grant reporting won’t go away, having systems set up to make reporting as easeful as possible so teams can focus on data collection for learning and improvement is crucial. 

There are a wide array of fancy (and often very expensive) platforms out there, and no one-size fits all. It is amazing, however, how much can be done with ostensibly simpler and (often much cheaper) tools like Google sheets and Airtable. Before even starting to think about what platform to choose, asking and answering key questions that emerge out of the theory of change is crucial. What do we need to learn? Why? When? Therefore, what do we need to measure?

As we set out to open the Evidence for Change call for applications once again, the Stanford Impact Labs team looks to this next year of the fellowship with the following learnings in mind:

• Leaning into theory of change and learning culture work is crucial. It’s also hard to prioritize this once Fellows are back in the day-to-day grind of nonprofit life, so we need to optimize on-campus workshop time to give teams ample time to do the work of thinking broadly and then putting plans into place.
• This approach of learning and testing impacts the whole organization. It can’t be siloed off to just a data team, for instance. Fellows therefore need involvement of leadership and participation from teams across their organization.
• This work never ends! The Evidence for Change fellowship is 10 months long, which is not long enough. We hope to provide a set of tools and frameworks that organizations can continue to apply going forward and work with teams eager to engage in this work well beyond the close of the fellowship.