Is it the existence of the Higgs boson? (sometimes). Is it the physics behind the death of Spiderman’s girlfriend Gwen Stacy? (this is the case for many nerds, but not for me). No, it’s the July Atlantic article on governmental use (or non-use) of program evaluation findings in program funding.  Written by John Bridgeland and Perer Orszag, two fellows who have worked under the administrations of Barack Obama and George W. Bush, this article ponders whether our current financial situation (squeezing social programs) could lead us to utilize Evidence! to lead to more efficient and effective programs for the country.  Such was the story of the Oakland A’s, for those unfamiliar with Moneyball, (the film, book, and baseball lexicon). Being in a tight financial spot, the A’s general manager hired a statistician to use player data to build the team roster, rather than scouts’ traditional belief. Did they win the World Series? (SPOILER ALERT!) No. But they did a lot better than everyone expected.

So the question is, can the government do a lot better than everyone expects?  Bridgeland and Orszag don’t really come to a conclusion on that, but they provide a lot of case studies of how the government has put money, time, and human resources into evaluation with mixed results.

Observation 1: Lack of funding for evaluation.  If this surprises you, I’m not sure you have been paying attention.  In my experience working with community organizations, health departments, and churches, minimal resources are available for carrying out meaningful evaluation.  This doesn’t mean we don’t want to have rigorous evaluations; in fact, much of what I hear is that people want to know if what they are doing is working.  They want to know that their work is making a difference.  As the authors note “less than $1 out of every $1,000 that the government spends on healthcare this year will go toward evaluating whether the other $999-plus actually works.” We are making grand changes on our health systems, and it will be a challenge to know what is working.

Observation 2: There are people in government and other relevant entities that are dedicated to using data for funding decision-making. Again, we see this every day in our interactions with various grant administrators, and from listening to the organizations we serve in almost all 50 states now.  Many are supportive and require purposeful examination of program data.  Not just how many people came to your groups or got tested, but what types of changes likely resulted of your program? How long do these changes last?  Under Bush, the Office of Management and Budget created and administered a Program Assessment Rating Tool, and assessed about 1,000 programs, including those addressing youth violence, teen pregnancy, nutrition, and school-readiness. That is commitment!  But…

Observation 3: Assessment results were pushed to the side by the decision makers – Congress.  Now, this is the most frustrating piece of information that can keep your nerd self awake at night. The authors give several examples of programs that the assessments showed were not performing, yet they continued to be funded because they were pet projects for congress members.

There are many reasons why all this would be frustrating to everyone, program participants who aren’t getting prepared for school, taxpayers who are paying for ineffective programs, and program staff who are working so hard to make a difference but their program design is ineffective (due to both internal and external factors alike) and needs real adjustment.

As a community based organization with a strong research and evaluation department, and a high value for using data for decision-making, this is clearly upsetting. We get mixed messages from our leaders in government. We are being told that data drives every decision. Yet we are seeing that politics can be the determining factor in whether programs get funded.  I see this as a call to action.  Social service organizations need to build up in two ways, evaluation and political relationships. We need to be aware that politics influence funding, and that will likely not go away. Play moneyball and build your network.  Will the government get smarter about evidence driven funding? Definitely.  While there will always be politics in funding, evidence is becoming more and more influential.  As government is taking cues from Hollywood, channeling Jerry Maguire may be our best bet: “Show me the data!” Or something like that.

Now it is your turn… tell us how you use data for decision-making in your work setting.

Written by: Emily Klukas

Follow @commissionCBA

Share This Link on Facebook

Share This on Twitter