On short term “fly in, fly out” evaluations

Some years ago, I was asked to be the external evaluator on a project involving a small, rural community bordering the Rio Grande river. The community had received funding from the US government to support a goat co-op that would produce products (eg: milk, cheese, meat) that could provide a sustainable livelihood for its members, and also counter the attraction of drug money so prevalent in the area.

The original evaluation plan involved two site visits. On the first visit, I would work out an evaluation plan with the co-op members who would then collect the data over the next 6 months. The second visit would involve data analysis and results. It all sounded reasonable until I arrived for the first visit, only to discover that the two “leaders” (who had written the grant proposal) were at war with each other. Both were non-Spanish speaking ‘anglos’ in a primarily Hispanic setting. I was also taken aside by the model herd manager and told that I, as a woman, could not speak with the men. So, what to do?

Though efforts to mediate the ‘war’ failed, I was able to involve all co-op members in a set of bi-lingual evaluation workshops which yielded some positive results. But in the end, my efforts were undermined locally once I left the site and the co-op dissolved in acrimony. I learned that short, fly in fly out evaluations, so typical of development work, are fraught with pitfalls and subject to local politics that an evaluator cannot control. I concluded that the only way to do this kind of evaluation is to work with a partner who lives in the area, one who is known and trusted locally and can be regularly on site. An outsider can offer expertise and perhaps objectivity, but cannot know the subtleties of local history and relationships and is thus vulnerable to manipulation by those who want to control the outcome.

(The details are published in Whitmore, E. (1998) ‘We need to rebuild this house:’ The role of empowerment in evaluation of a Mexican Farmer’s Cooperative, Chapter 13, in Kassam, Y. and Jackson, E.T. (Eds.). Knowledge Shared: Participatory evaluation in development cooperation. (pp. 217-230) West Hartford, CT: Kumarian Press)

Elizabeth (Bessa) Whitmore
Professor Emerita
Carleton University

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s