1.1 Assessing Impact
While program evaluations are common in the education sector and guidelines well established in the past (see for example, The program evaluation standards, 2nd edition, 1994), recent trends in the social sciences literature document a shift towards how such evaluative exercises might provide better and more timely information to the organizations they work with, particularly in terms of realizing the organizational mandate or goals. Such thinking focuses more on the organization’s quality, its worth in terms of meeting client or participant needs, its significance or importance to a community or group, as well as how potential lessons might be learned. In other words, it is more of a value driven exercise than one driven by quantitative, final outcome measures (Stufflebeam, 2007). The term impact assessment has therefore become a more common way of framing how such value-oriented outcomes might be considered and reported. Marula et al (2003) for example, suggest assessment is better described as “… analogous to a reflective process through which social change actors and advocates articulate their change goals and formulate the criteria with which they will evaluate the successes and failures of change efforts. This in turn guides the actors in rethinking their change efforts, influencing whether and how their further efforts should be modified” (p. 58 as cited in Lall, 2011, p. 5).
Other trends in impact assessment include the use of participatory research methods (McGregor, Clover, Sanford & Krawetz, 2008) that emphasize a need for reciprocity– including co-researcher roles– processes of shared knowledge creation and dissemination, and realizing socially just outcomes. A related field of research that is particularly important in the Canadian context is the ethics of conducting research that involves Aboriginal, First Nations, Inuit and/or Métis populations. Ball and Janyst (2008) represent many Canadian social science researchers in pointing to the importance of developing mutually negotiated protocols prior to beginning the cycle of research with Aboriginal communities, the need to develop research methods that are inclusive of Aboriginal perspectives, that are conducted with Aboriginal peoples in partnership, that incorporate Aboriginal cultural practices into their research processes, including processes of analysis, knowledge construction and dissemination. “Valid, useful findings and the larger goal of restorative social justice can flow only with partners as active participants in generating and interpreting data and shaping plans for knowledge mobilization” (p. 45). The consequences of doing otherwise, they argue, are to reify the colonial past in which Aboriginal peoples were ‘subjects’ and ‘objects’ of inquiry and to maintain systems of marginalization and exclusion. In the context of this study, such observations are incredibly powerful, given the purposes of the AES Network and the legacy of harm that educational systems have had on Aboriginal peoples.