Location: University of Alberta and Alberta Land Institute
Research Team: Peter Boxall, Stephanie Simpson, and Curtis Rollins
Duration: March 2012 – March 2013
Objectives
Assess outcomes relating to Natural Advantage Program NAP participation, including action completion rates and referral access rates, and compare them with outcomes associated with non-participation in order to provide a general assessment of program success.
Identify factors linked to the adoption of actions and referred sources of assistance, focusing on how the NAP performed in promoting different types of actions to different types of individuals.
In addition, we use our findings from, and our reflections on, the evaluation process to:
3. Develop recommendations on how to improve the design of an extension effort such as the NAP.
4. Develop recommendations on how to conduct an external evaluation of an extension program.
Background
The Natural Advantage Program (NAP), was a free, intensive extension service offered by Ducks Unlimited Canada to Alberta-based producers between 2007-2009. The program provided participants with personalized reports identifying: 1) Stewardship practices that could be implemented on their farms; and, 2) Sources of financial and/or technical assistance to help implement those practices.
In late 2011, two types of surveys were developed for the purpose of this evaluation: a general survey designed to collect demographic and farm information from all participants, and a personalized survey designed to assess 2007-08 participants’ completion of the specific actions and referrals recommended to them by DUC. Surveys were then distributed to 215 NAP participants, with 137 participants (63.7%) returning completed surveys.
Respondents from 2007-08 reported completing 52.6% of the 683 practices, and accessing 28.4% of the forms of assistance, recommended to them in their NAP reports. Action completion rates (but not assistance access rates) were significantly higher than those reported by 2009 respondents, who did not receive the NAP service due to its early termination. When asked why they did not comply with recommendations, 2007-08 respondents most frequently stated factors relating to resource constraints, irrelevance and forgetting, in that order.
Consistent with the conservation practice-adoption literature, the NAP saw greater practice adoption amongst producers who owned their land, and who planned to transfer it to their families. Farm size, watershed group membership, and interest in conservation also increased the likelihood of adoption. The actions most likely to have been adopted were those that required little investment of time or money while causing little inconvenience to the producer. Actions causing greater inconvenience or requiring a greater investment were still somewhat likely to have been adopted if they also provided visible, private benefits.
Data Analysis Methodology
Descriptive statistics of farm and respondent characteristics were compiled. To determine in a broad sense the generalizability of this review’s findings to the greater population of Alberta producers, these statistics were compared to provincial data, largely from Statistics Canada’s 2006 Census of Agriculture (2006b). Descriptive statistics were also compared between the 2007-8 and 2009 groups in order to determine whether the 2009 group was a suitable comparison against which to assess the impacts of NAP participation.
Responses were aggregated and subjected to Principle Component Analysis (PCA). This PCA verified that each of the 15 items included in the scale correlated significantly with the appropriate higher-level factors. Oblique rotation was used to allow for correlation between the factors (the lifestyle and conservation factors are predicted to be correlated) and the number of factors to extract was set to three. A Cronbach’s alpha coefficient was also calculated for each of the three factors to determine whether the scale reliability of each factor was sufficient for exploratory research.
To assess stated action completion and referral access rates, descriptive statistics were calculated using 2007-8 participants’ personalized survey data. To assess very generally the NAP’s impact on a producer’s likelihood of acting on NAP recommendations, these descriptive statistics were compared to action- and assistance-related data provided by 2009 respondents in the general survey.
To provide insight into the factors contributing to 2007-8 participants’ decisions to follow NAP recommendations, stated reasons for non-compliance were compiled using personalized survey responses. Additionally, two econometric models (probit and ramdom effects), chosen for their complementary strengths, were estimated to identify the relative impacts of farm- and action-level factors on the likelihood of action adoption.
Deliverables
A final report presenting an evaluation of the effectiveness of an agri-environmental extension program, and making recommendations on how to design future programs and conduct external evaluations of such programs was delivered at the end of the project.