My goal is better intervention.
The schools in my community promote leadership district-wide using the 7 Habits of Happy Kids; so I have three household members bringing home scaffolded messages from kindergarten through middle school, at least. (My oldest is in eighth grade, so high school is yet to be revealed to us.) Working together, the kids and I can usually list these seven little mantras in about 45 seconds flat. It takes a little longer for us to put them in their correct numerical order. And we agree to disagree on which ones are the most important (but it's definitely Seek First to Understand, then to Be Understood).
Habit #6 is good, too, and will make a guest appearance in a future blog post or two.
The visual below is my embracing Habit #2: Begin with the End in Mind. The purple dot represents the goal of my work.
To this goal of better intervention, I offer my evaluation capacity building skills. If I were to say, though, (and I'm speaking from experience, here) that my goal is "increased evaluation capacity," some eyes can glaze. In the non-academic world, talking about "second order organizational change in evaluation utilization," or even "applying research to action," does not always connect.
Better intervention is better. I can ask any potential client, "what do you do?" and whatever the response (Seek First to Understand), I am usually in a position to say, "I want to work with you to find some ways to do {that} better."
"We can find -- or generate -- evidence that we can use to improve that work."
That's the goal.
Now that we are all clear on that purple dot,
...let's talk about evaluation use.
As an evaluator, I embrace participatory methods and feel most comfortable with the utilization-focused evaluation approach. I believe that it is the use of evaluation results, that has a direct and positive effect on the quality of interventions aimed at ending sexual violence.
See the green arrow below.
As much as I enjoy it, it's not enough to collect data and describe results. That's one of the reasons I left the Ivory Tower. At the beginning of a project (Put First Things First), my first set of questions always includes questions about use, and users:
Who is going to use this evaluation? Or toolkit? How?
No, I mean who specifically? Is it you? Your organization's board? Your volunteer coordinator? Your grants manager? Your funder?
What is the name of the person who will read the report? What kinds of things do they find credible? What kind of decisions do they need to make? What's the rest of the puzzle they're working on? Where will this new information fit into their bigger picture?
You get the idea.
It's then critical to get those people involved in the early conversations. Similar to evening meal preparation, the purpose is to get buy-in and ensure satisfaction.
What do you want to eat for dinner? Will you eat meatloaf? (No! No! Maybe.) Or fishsticks (Yes. No. Yes.) Or tortellini? (Yes. Yes. Yes). If you've ever been responsible for feeding children, you know the trick of getting them involved in the food prep as a way of getting them to eat the meal.
Wait. Did I just say that getting people to use evaluation is like getting kids to eat their dinner?
When it comes to evaluation, there are things that can be done (things I am trained to do) that I like to think increases the likelihood that evaluation will be used towards the goal of better intervention.
My work focuses on three primary capacity building strategies. Shown below in primary colors, they are: professional development, resource and support allocation, and organizational change.
Three major evaluation capacity building strategies in three primary colors.
My business plan is based on the idea that if we work on a couple of the "primary color things," everyone's evaluation capacity grows -- that we all learn, and are better able to use that evaluation process to do better intervention.
My friend Tina Ritzler and other colleagues in the field have done research describing four distinct components of evaluation capacity, which are related as shown below.
This part of the picture is really important. Because it brings me to the BIG IDEA: SOCIAL CHANGE. My work is aimed at more than just training or educating or cranking out reports. It's community change. And this picture shows it well.
If all energy is focused solely on professional development, there is no way to get to the better-intervention end goal. Individual-level factors such as "awareness of the benefits of evaluation" or "motivation to conduct evaluation" or "skills to conduct evaluation" are an important part of getting to better intervention. But it is the organizational changes in leadership, demand, structures that develop a learning climate and the resources to support it that make the difference.
The orientation of the arrows indicates the difference between first order and second order change. The arrows that point down represent first order change, which is generally an easier row to hoe. Momentum is kind of with you on this. Second order change is where things get real. It can be tough. It's usually not easy to push change through those orange and green arrows.
It is important, though.
““First-order” change is change that is consistent with prevailing values and norms, meets with general agreement, and can be implemented using people’s existing knowledge and skills. A change becomes “second-order” when it is not obvious how it will make things better, it requires people to learn new approaches, or it conflicts with prevailing values and norms. Second-order changes require leaders to work far more deeply with staff and the community. They can disrupt people’s sense of well-being and the co-operation and cohesion of the school community. They may confront and challenge expertise and competencies and throw people into states of “conscious incompetence.” Different perceptions about the implications of change mean that a change that appears to be a solution to one person can appear to be a problem for another.”
Watch what happens when that green arrow is taken out of the model. Let's say, for example, that I conduct an evaluation, and write an evaluation report for a client, but then nobody uses it. Say that report sits on a variety of wooden and electronic desktops, unread. The work we've done does not make intervention any better. We do not get closer to a world free of sexual violence.
It happens. My clients are busy. I am busy. When we catch a break and meet a deliverable, we're all moving on.
But this website will be here. And can be a place to share bits and pieces of those outcomes of our evaluation. I'll try to put it up here in ways that encourage use.
I have talked about this before: to colleagues at the National Sexual Assault Conference in 2013, almost at EVAL 2014 (my Frontier Airlines flight CLE ->DEN was cancelled due to Ebola scare), to my friends at the Society for Community Research and Action in 2015, at EVAL 2016; and I've blogged about it, too, here.
