«It takes a village ... Considering the limits of a knowledge-based school-centered quality evaluation of citizenship education projects»

Participation experiences, particularly in the school context, have been considered a crucial opportunity for developing personal and social resources for citizenship involvement. However, the impact of these experiences needs to be considered. The IEA Civic Education Study (1994-2002) is an example of a large-scale study that made an enormous effort for evaluate quality and efficacy of civic education in various countries; however, the very nature of the research results in over-emphasizing civic knowledge. This paper intends to critically analyze the limits of a knowledge-based approach to school citizenship education projects. Assuming an ecological-developmental perspective, that views life contexts as locus for action and interaction with different others, we argue that both the design and evaluation of school-based citizenship education projects should extend beyond the school to include other contextual dynamics and others active actors who support the students’ learning from participation.

Participation experiences, particularly in the school context, have been considered a crucial opportunity for developing personal and social resources for citizenship involvement. However, the impact of these experiences needs to be considered. The IEA Civic Education Study (1994)(1995)(1996)(1997)(1998)(1999)(2000)(2001)(2002) is an example of a large-scale study that made an enormous effort for evaluate quality and efficacy of civic education in various countries; however, the very nature of the research results in over-emphasizing civic knowledge. This paper intends to critically analyze the limits of a knowledge-based approach to school citizenship education projects. Assuming an ecological-developmental perspective, that views life contexts as locus for action and interaction with different others, we argue that both the design and evaluation of school-based citizenship education projects should extend beyond the school to include other contextual dynamics and others active actors who support the students' learning from participation.
Democracy vitality depends on the involvement and participation of citizens, but research has been revealing decreasing levels of participation in the civil society, particularly in conventional political activities, and growing tendencies for political skepticism and apathy particularly (but not exclusively) in young people (Putman, 1995;Delicado, 2003;. In Europe, where the institution of the European Union as a novel form of political governance is creating new challenges, there has been a particular emphasis on the need to "prepare young people for a new social, political and economic order" (Torney-Purta et al., 2001, p. 12), even more since research has shown that youngsters' political behaviors and attitudes are predictors of civic participation in adulthood (Miller & Kimmel, 1997). Therefore, the role of the school has been stressed out and the impact of dimensions such as non-authoritarian classroom environments on political attitudes and behaviors led researchers to advocate that "learning about citizenship is not limited to explicit instruction of teachers about young's rights and duties" (Torney-Purta et al., 2001, p. 21). These notions have also permeated the educational reforms that, since the eighties, European countries have experienced with a systematic focus on citizenship education (Bento, 2000).
Obviously, this is not a new phenomenon: conceiving the school as a vehicle for students' personal and social development is as old the school itself (Roldão, 1999); however, the curricular forms and strategies used to accomplish these goals have varied intensively. These European proposals have included cross-curricular infusion or dissemination of values, themes or competences; specific subjects or curricular spaces; transformation of the school ethos; creation of democratic institutions such as students' councils; extra-curricular projects, sometimes involving community service; … (Menezes, 1999). However, in spite of this enormous diversity of experiences, there is a scarcity of research that considers the actual impact of these curricular devices in students' political dispositions and behaviors.
A notable exception has been the IEA Civic Education Study (1994)(1995)(1996)(1997)(1998)(1999)(2000)(2001)(2002), a large scale cross-national study that examined civic knowledge, attitudes and behaviors of young people at age 14 and upper-secondary students, thus generating a data-based approach to questions like the engagement of citizens in democracies (Torney-Purta et al., 2001). The IEA -International Association for the Evaluation of Education Achievement -is an international consortium of research institutions in more than 50 countries and is well known for conducting studies in areas such as mathematics and sciences or literacy. The goal of the IEA Civic Study Education is to identify and examine, in a comparative framework, the ways in which young people are prepared to undertake their role as citizens in democracies (more or less recent ones), focusing not only their curriculum but also their opportunities for civic participation in the community. Responding to the expressed need of many countries for vital empirical data, these studies generally contemplate both a systematic analysis of written or prescribed curricula and the development of reliable and valid tests administered to nationally representative samples of students in the target grade(s). Results are seen as indicators of the quality and effectiveness of represented educational systems, and usually generate a widely publicized ranking of the various countries based on students' levels of knowledge and skills. In the case of the Civic Education Study nearly 90,000 14 year-old students in 28 countries (and approximately 50,000 upper secondary students in 16 countries) were tested regarding their civic knowledge and attitudes. There was an intense effort in the construction of "a meaningful, reliable and valid international test of student knowledge about democratic institutions, principles, processes and related topics despite differences in the political systems in different participating countries" (Torney-Purta et al., 2001, p. 44). This was indeed a major achievement of the study, but it should be noticed that is also important to consider the consequences of over-emphasizing civic knowledge in evaluating the quality of citizenship education projects 1 .

Limits of a knowledge focus on assessment and evaluation
As Bond stresses "as society shifts from an industrial age, in which a person could get by with basic reading and arithmetic skills, to an information age, which requires the ability to access, interpret, analyze, and use information for decision-making, the skills and competencies needed to succeed in today's workplace are changing as well" (1994, p. 2). As a matter of fact, schools are nowadays expected to help students develop useful skills and competencies for real life situations. Therefore, schools have to support the development of basic psychological processes, which will help subjects manage increasingly demanding situations. As a mere example, possible strategies could include having students performing various roles in the classroom that will make them express and defend their own opinions, or assuming responsibilities of in-and out-of-classroom -both of which appear to be associated with higher levels of political participation. The Delor's Commission on Education for the 21 st Century also emphasized the need for the development of competencies essential for the never-ending process of actualizing, deepening and enriching knowledge and adapting to a changing world . Additionally, existing research in the political domain also shows that knowledge, in contrast to democratic competencies, hardly contributes anything to the aims of socially committed participation (Oesterreich, 2003).
Moreover, when the evaluation of educational quality stresses knowledge about the political world, the message being transmitted to teachers, students, parents and other educational actors has several implications: i. an emphasis on contents rather than change processes, a focus on "what" and "how much" students have learned, which might favor educational strategies such as direct instruction, rote and passive learning; ii.
widening the gap between the one who knows (the teacher) and the ones who learn (the students); iii.
a clivage between (school-)knowledge and action (in the real world) resulting in low opportunities for students to put their knowledge into action, (re)interpreting existing knowledge in a self-constructed discourse; iv.
disregard for students' experiences (spontaneous or intentional) in-and out of school as an opportunity for learning.
Furthermore, the focus on knowledge rests on the tacit assumption that there is a direct relationship between knowledge and action -an assumption that is under severe questioning in several domains of human development (Menezes, 1999). Moreover, even if we could argue that a person "who has a democratic orientation will not have much influence in society if s/he has no adequate idea of society and its basic institutions and regulations", we could certainly agree that "somebody who understands what is going on in a society will not necessarily be a good democrat" (Oesterreich, 2003, p. 1-2). Therefore, if we recognize that citizenship is not a defined fact, a fixed social category, but the product on an ongoing social and personal construction emerging from public debate (Benhabib, 1999;Dahrendorf, 1994), the focus of learning should not be on contents but on processes (Gentilli, 2000). Hence, "if the goal is to promote the personal empowerment and social pluralism on which the essence of democracy relies" it is important to recognize the learning value of action and experience (in or out of school environment) "but only if it is intentionally designed and systematically supported: the quality of participation experiences, both in terms of meaningful involvement, of integration with (different) others, and opportunities for personal integration" (Menezes, 2003 b, p. 430). However, this shift only makes sense if quality evaluation of citizenship education evolves from a content-based assessment of students.

Evaluation as an instrument of quality improvement and empowerment
Is quality evaluation an episodic activity or a transversal tool of educational systems that involves all actors and experiences? Is quality evaluation simply a method to collect information on "what" students are learning from existing opportunities or does it serve as an instrument for change and improvement? As Lipsey & Cordray (2000) underlie it is important that evaluation procedures are not only an X-ray of the existing situation but help explain why and how certain results were (or weren't) obtained. For this understanding to be achieved, the active involvement of the actors (the teachers, the students, the parents, the community …) is absolutely essential. In fact, the model underlying the IEA Civic Education Study (Torney-Purta, Lehman, Oswald & Schultz, 2001) is a good point of departure for this approach; based on Brofenbrenner's ecological-developmental perspective (Figure 1), it embodies the influence of diverse life contexts in the thinking and acting modes of young people regarding the social and political environment. The ecological-development model, which privileges a holistic approach of the subject, assumes human action as fully connected to the ecological context where it takes place, in order to the promotion of youth's autonomy and empowerment. It assumes that human development is something that doesn't occurs naturally by a maturation process but by a systematic and intentional intervention that promotes psychological processes essential to achieve life competencies. In citizenship education this is even more relevant. As Putman (1995) stresses, civic involvement refers to a broad range of activities, which a person might assume in a more formal or informal way and in a variety of contexts. And if it is true that that school is a privileged locus for citizenship development, it's also true that students are also living relevant out-of-school experiences, in their families and communities (Sprinthall & Collins, 1988). The consequence is that the evaluation of theses experiences should therefore be extended beyond the schools to include other contextual dynamics, e.g., political parties, environmental organizations, religious groups, Scouts, and so on, and others active actors who support the students' learning from participation (Torney-Purta et al., 2001; Amadeo et al., 2002;Menezes, 2003 b;Ferreira, Ribeiro & Menezes, 2003). Consequently, an alternative way of evaluating citizenship education projects should concentrate on the quality of participation experiences and relationships between the teachers and other significant adults and the youngsters. It should be taken into account that research shows how participation is not good in itself , and that it could even reinforce intolerance and out-group bias (De Piccoli, Colombo & Mosso, 2004). Life contexts must give young people a chance to try new roles and responsibilities, with more active involvement, that will be pertinent to their identity construction. Hence, these experiences of interaction should allow for learning by doing, but it is also relevant to note that they should be include opportunities for systematic Public speech and practices (values, institutions, symbols, media, local and national narratives) Changing contexts (school, family, peer groups, local community) Student reflection, analysis and insight, in the context of a significant relationship, if our aim is to originate a personal integration of the experience. In this sense, quality evaluation of citizenship education projects must include a thorough analysis of the quality of experiences regarding opportunities for action and interaction with different others, balanced with intentional moments where the youngsters make a systematic and principled analysis of what is going on. In order to achieve this task, the active involvement of all the participants is absolutely essential, particularly of the goal is not only to consider the impact of these experiences but also to devise ways in which the existing practices could be improved and constitute a relevant and empowering civic opportunity for all of those involved. If they say it takes a village to educate a child shouldn't we involve the whole village in evaluating what is going on?