Center for Civic Education

Research and Evaluation
Project Citizen and the Civic Development of Adolescent Students in Indiana, Latvia, and Lithuania

Executive Summary

Thomas S. Vontz, Kim K. Metcalf, and John J. Patrick

ERIC Clearinghouse for Social Studies/Social Science Education and the Adjunct ERIC Clearinghouse for International Civic Education in association with Civitas: An International Civic Education Exchange Program

We the People... Project Citizen is an instructional product for adolescent students, which was developed and published in 1992 by the Center for Civic Education at Calabasas, California. Since then, it has become very popular. Today, Project Citizen is used by teachers and their students in all 50 states of the United States of America and more than 30 countries in different regions of the world.

Project Citizen involves students in the selection and investigation of important public issues in their community. They work cooperatively to propose, justify, and defend resolution of the issues. Thus, Project Citizen engages students in learning experiences designed to affect positively their civic development, which involves three basic components of democratic citizenship: civic knowledge, civic skills, and civic dispositions. Is Project Citizen an effective means to the civic development of students?

The study reported in this monograph, conducted at Indiana University, Bloomington by the Social Studies Development Center and the Indiana Center for Evaluation, was designed to evaluate the effects of Project Citizen on the civic development of adolescent students in Indiana, Latvia, and Lithuania. This inquiry began in August of 1999 and ended with the publication of this monograph in November of 2000.

Questions of an Inquiry to Evaluate Project Citizen

This inquiry was conducted in response to two sets of major questions (items one and two) and two ancillary questions (items three and four), which are listed below:

  1. What are the effects of Project Citizen on the civic development of adolescent students in Indiana, Latvia, and Lithuania?

    1. What are the effects of Project Citizen on the achievement by adolescent students in Indiana, Latvia, and Lithuania of particular kinds of civic knowledge?

    2. What are the effects of Project Citizen on the beliefs of adolescent students in Indiana, Latvia, and Lithuania about their achievement of particular civic skills?

    3. What are the effects of Project Citizen on the achievement by adolescent students in Indiana, Latvia, and Lithuania of particular civic dispositions?

  2. What are the relationships between the effects of Project Citizen on the civic development of adolescent students in Indiana, Latvia, and Lithuania ,and particular contextual and personal factors?

    1. What are the relationships between the effects of Project Citizen on the civic development of adolescent student Lithuania and particular demographic factors?

    2. What are the relationships between the effects of Project Citizen on the civic development of adolescent students in Indiana, Latvia, and Lithuania and particular programmatic factors?

    3. What are the relationships between the effects of Project Citizen on the civic development of adolescent students in Indiana, Latvia, and Lithuania and particular instructional factors?

    4. What are the relationships between the effects of Project Citizen on the civic development of adolescent students in Indiana, Latvia, and Lithuania and particular school-type factors?

  3. Between the political units of Indiana, Latvia, and Lithuania and independent of participation in Project Citizen, are there differences in the civic development of adolescent students in this study?

  4. Is Project Citizen differentially effective across the political units of Indiana, Latvia, and Lithuania?
Development of the Civic Development Inventory

An instrument was developed, The Civic Development Inventory (CDI), by which to gather data in response to the questions of this inquiry. The CDI (see Appendix A) was derived from an instrument constructed at the Center for Civic Education in Calabasas, California. However, the CDI was conceptualized and developed at the Social Studies Development Center of Indiana University in terms of three components of civic development: civic knowledge, civic skills, and civic dispositions. Further, the civic dispositions component included five factors: propensity to participate in civic and political life, political interest, commitment to responsibilities of citizenship, commitment to constitutionalism and rights of citizenship, and political tolerance.

Validity of the factors or dimensions of this instrument was developed by use of expert review and factor analysis with varimax rotation. The reliability of the instrument's measurement of each validated factor was demonstrated by use of Cronbach's alpha. The resulting reliability coefficients are civic knowledge, .57; civic skills, .80; political interest, .52; commitment to responsibilities of citizenship, .51; political tolerance, .70; commitment to constitutionalism and rights of citizenship, .69; propensity to participate in civic and political life, .78. Clearly, the weakest constructs or factors were political interest and commitment to the responsibilities of citizenship, both of which failed to reach a reliability coefficient of .55. The strongest factors were civic skills and propensity to participate in civic and political life, which both had reliability indicators higher than .75.

Research Design and Methods

The research design of this inquiry involved 102 classroom groups and 1,412 students in three political units: Indiana in the United States of America, Latvia, and Lithuania. There were non-randomly selected treatment classes (51 with 712 students) and comparison classes (51 with 700 students). Indiana had 20 pairs of classes (275 treatment class students and 267 comparison class students); Latvia had 13 pairs of classes (139 and 126 students); and Lithuania had 18 pairs of classes (29R and 307 students). Every student responded to a pretest and a posttest, the Civic Development Inventory (see Appendix A), which was constructed to gather data relevant to the research questions of this inquiry. Further, every teacher involved in this inquiry responded to the Project Citizen Teacher Questionnaire (see Appendix C).

Personal and contextual data about students and teachers, gathered through responses to items of The Civic Development Inventory and Teacher Questionnaire, were used to demonstrate comparability between treatment and comparison classes of students. This evidence for equivalence between pairs of classes warranted the claim that the treatment, Project Citizen, explained the positive differences in civic development between treatment and comparison groups, not existing differences in personal and contextual data associated with individuals in the paired classes. To ensure the preprogram similarity or comparability of treatment and comparison classes, multivariate analysis of variance (MANOVA) was used. The results (see Table 4.2) indicated that there were no significant differences between treatment and comparison classes across selected student, teacher, or school characteristics within each political unit - Indiana, Latvia, and Lithuania - and across all three political units of this inquiry.

Data pertaining to the first set of research questions of this inquiry - about the effects of Project Citizen on the civic development of adolescent students in three political units (Indiana, Latvia, and Lithuania) - were analyzed through the use of two-way univariate and multivariate analyses of covariance (ANCOVA and MANCOVA). The classroom group, not the individual student, was the unit of analysis in order to avoid misrepresentation or exaggeration of positive program effects. Project Citizen is a classroom-based instructional treatment offered to individual students nested within classes. Thus, the impact of the treatment on an individual student was impossible to separate from the effective- ness of the teacher and the characteristics of the other students of the class in which the individual experienced the program.

In this study, the 102 classes of students (51 treatment and 51 comparison) were the units of analysis in order to avoid misleading claims about the positive and significant effects of Project Citizen on the civic development of students. For each class (in both the pretest and posttest), mean student performance on each of seven factors (civic knowledge, civic skills, and five civic dispositions) was calculated and aggregated by class. Differences in means between treatment groups and comparison groups were analyzed to determine statistical significance across political units by two-way ANCOVA and MANCOVA.

The second set of research questions pertained to personal and contextual characteristics that might have contributed to explanations of significant differences in Project Citizen effects on students, which were revealed by analyses of data in response to the first set of questions. The student, not the class, was the appropriate unit of analysis for this facet of the inquiry. Thus, stepwise multiple regression techniques were applied to four sets of data pertaining to various personal and contextual factors.

The two ancillary questions of this inquiry (items three and four) addressed differences across and between the three political units - either in the effectiveness of Project Citizen or in the baseline level of students' civic development. Two-way analysis of covariance was used to analyze concurrently data pertaining to the first major set of research questions and the two ancillary research questions.

Findings

Project Citizen appeared to affect students' civic development positively and significantly across three political units: Indiana, Latvia, and Lithuania. The positive effects on students' civic development were consistent across the three political units, which suggests that the effectiveness of Project Citizen was not dependent upon or mediated by the country in which it was used.

Civic Knowledge. Project Citizen had a positive and statistically significant effect on the civic knowledge of students across the three political units of this inquiry: Indiana, Latvia, and Lithuania. After accounting for pretest differences, the mean posttest score of treatment classes was much larger than the score of the control or comparison classes (1.43 and .92, respectively; see Tables 6.1 and 6.2). This difference was statistically significant at the .05 level.

Civic Skills. Project Citizen had a positive and statistically significant effect on the self-perceived civic skills of students in Indiana, Latvia, and Executive Summary Lithuania. After participating in the program, students in treatment classes perceived themselves to possess more civic skills than students in comparison classes, who were not exposed to Project Citizen. After accounting for pretest differences, the mean posttest score of treatment classes on civic skills was significantly larger across the three political units of this inquiry than that of the comparison classes (3.17 and 3.04. respectively; see Tables 6.4 and 6.5). This difference was statistically significant at the .05 level.

Civic Dispositions. There was a statistically significant and positive effect of Project Citizen on students' propensity to participate in civic and political Life. After accounting for pretest differences, the mean posttest score of treatment classes on one civic disposition, propensity to participate, was significantly larger across the three political units of this inquiry than that of the comparison classes (2.21 and 1.89, respectively; see Tables 6.7 and 6.8). This difference was statistically significant at the .05 level. Project Citizen did not have a statistically significant impact on four civic dispositions: political interest, commitment to responsibilities of citizen- ship, commitment to constitutionalism and rights of citizenship, and political tolerance (see Tables 6.7 and 6.8).

Consistency of Project Citizen's Effects Across Political Units. The positive effects of Project Citizen on students' civic development were not dependent upon the political unit - Indiana, Latvia, or Lithuania - in which the instructional treatment was experienced. Effects were largely consistent across the three political units indicating that they were neither enhanced nor mediated by the political unit in which students experienced Project Citizen. The program appeared to be equally effective across the three political units of this study (see Tables 6.2, 6.5, and 6.8).

Differences Between Political Units in Students' Civic Development. Independent of their participation in Project Citizen, students of this study in Lithuania demonstrated a significantly higher level of civic knowledge than students of this study in Indiana and Latvia. Further, students in Latvia and Lithuania had significantly more political interest than students in Indiana. by contrast, the Indiana students exhibited a significantly higher level of self-perceived civic skills than their counterparts in Latvia. However, differences in students' civic development between the three political units of this study, apart from their participation in Project Citizen, were neither extensive nor profound (see Tables 6.3, 6.6, and 6.9).

Personal and Contextual Variables. The statistically significant and positive effects of Project Citizen on the civic development of students were generally not related to or explained by various personal and contextual factors examined in this study. There were five exceptions: the student's perceived level of participation in Project Citizen, mother's level of education, type of issue selected for investigation, implementation of the proposed policy, and curricular implementation of Project Citizen. The student's self-perceived level of participation or involvement in Project Citizen was the variable most strongly related to gains in civic development. A higher level of the mother's educational attainment was also associated with greater gain in the student's civic development.

Some of the explained variance in student gains in civic development could be attributed to the type of curricular implementation of Project Citizen; that is, use of the instructional treatment in an extra-curricular format or in a combination of curricular and extra-curricular formats resulted in greater student gains in civic development than use of the program solely in the regular curriculum. Further, students gained more in civic development when they investigated an issue in the school instead of the larger community outside the school. Finally attempted implementation of the students' resolution of a community-based or school-based issue was associated with substantially more gain in civic development. (See Table 6.10.)

Recommendations

This evaluation of Project Citizen suggests that the program can be used to promote the civic development of adolescent students in various countries in different parts of the world. These findings about the program's instructional effectiveness, however, are not definitive. More research is needed to investigate strengths and weaknesses of Project Citizen. In subsequent research about Project Citizen effects on students' civic development, the conceptualization, design, methods, and instrumentation of this study might be used, with appropriate modifications, to conduct inquiries that could confirm, alter, or expand findings of this inquiry.

In particular, curriculum developers, teachers, and researchers might collaboratively explore means to improve Project Citizen’s impact on students' civic dispositions. The related-research literature indicates that civic dispositions tend to be resistant to change as a consequence of "one-shot" and short-term exposure to an instructional treatment. Thus, it is notable that Project Citizen had a positive impact on one civic disposition, propensity to participate. A broader impact on civic dispositions might be achieved through pointed, detailed instruction about such factors as political tolerance, commitment to constitutionalism and rights of citizenship, commitment to responsibilities of citizenship, and political interest. It seems that long-term, in-depth instruction targeted directly to dispositional change is a key to improving Project Citizen’s impact on a broad range of civic dispositions in addition to propensity to participate in civic and political life.

Findings of this inquiry suggest additional means to enhance Project Citizen's impact on students' civic development, such as the following recommendations:

Involve all students in the class maximally as participants in all aspects of the program.

Emphasize school-based public policy issues, but not to the exclusion of community-based issues that strongly attract the attention and interest of students.

Encourage students to attempt- implementation of the policy they proposed to resolve a public issue.

Implement the program through a combination of curricular and extra-curricular activities.

Avoid brief and irregular involvement of students in Project Citizen; rather, integrate the program as fully as possible into the curricular foundations and extra-curricular activities of the school.

Expand the civic knowledge component of Project Citizen and strengthen connections and interactions of civic knowledge, civic skills, and civic dispositions through instructional activities of the program.

In general, this study found Project Citizen to be worthy of continued use in various educational settings in different parts of the world. The continued implementation of the program as a means Co students' civic development in a democracy should be investigated through subsequent research, which might be assisted by the conceptualization, instrumentation, design, and methods of the inquiry reported in this monograph, " Project Citizen and the Civic Development of Adolescent Students in Indiana, Latvia, and Lithuania."

In the meantime, as educators around the world await new findings from subsequent research, they have justification, based on findings reported in this monograph, for using Project Citizen to achieve positive instructional outcomes: significant gains in the civic development of students.

This publication is available from:

ERIC Clearinghouse for Social Studies/Social Science Education
Indiana University
2805 East Tenth Street, Suite 120
Bloomington, Indiana 47408-2698
800-266-3815
812-855-3838
fax 812-855-0455
ericso@indiana.edu
http://www.indiana.edu/~ssdc/eric_chess.htm

<<Center for Civic Education homepage

Send comments regarding this page to web@civiced.org