Assessing Our Work
By Mark Frederick, Indiana State University
In the Spring, 1993 edition of Peer Review, Caryn McTighe Musil, then AAC&U’s Vice-President for Diversity, Equity, and Global Initiatives, presented “Faces/Phases of Citizenship,” a developmental learning map” that could easily serve as a rubric for measuring college students’ growth, learning, and development (GLD) in the broad area of citizenship. With that model, an opportunity came to be where higher education could expect a reliable answer to the simple question “are our students developing meaningful citizenship skills and if so, to what degree?” And it seems that whenever an opportunity to measure something meaningful presents itself, there will be a social scientist that jumps into the fray to take on the challenges of operationally defining an important construct, collecting data, and reporting results to an educational environment that might or might not be “in the mood” to receive them. But unlike Jack Nicholson’s Character in the movie “A Few Good Men” where he states “You can’t handle the truth,” those of us in higher education can ill afford to not “handle the truth.” And the truth as to how and to what degree college students are becoming better citizens is being revealed, to the limits of the instrument, with the University Learning Outcomes Assessment (UniLOA).
The UniLOA measures student behaviors along seven broad domains, one of which is Citizenship. To be sure, the UniLOA is a self-report instrument and most of us would prefer to employ direct measurement of student GLD, the resources just aren’t available for us to hire trained observers who will follow students “24/7” to accurately record their behaviors. Ultimately, we have to rely on self-report, if for no other reason, than it’s the only method of gathering information we can really afford in higher education. While there is understandably some question as to the reliability of self-reported information, we’ve found remarkable stability and consistency in results patterns over time as students from across the country have reported their behaviors when they are administered the UniLOA. To that end, and with more than 50,000 student respondents, we’re confident that we have an accurate picture of the current state of citizenship development among college students across the nation.
Unfortunately, the news isn’t particularly good. Of the seven UniLOA domains, citizenship scores the very lowest and individual items for the domain are well below the mean of other domains’ items. Of the ten citizenship domain items, “I engage in the political process through voicing viewpoints” holds the honor of being not only the lowest-scored domain question, but indeed, one of the lowest scored questions of the entire UniLOA survey. The second lowest-scored item suggests that students don’t contribute financially to causes they believe in. On first blush, low levels of behavior would make sense among perpetually “broke” college students, but it would seem that they miss the point that “contributing” is a process rather than a relative value of the amount of a financial gift and giving just a little begins a process of learning “how” to contribute, regardless of the gift’s size.
Other items on the citizenship domain that score disappointingly low include students’ tendency to not vote as frequently as we’d like, that those that do vote, many fail to adequately research candidates before voting or even reporting that they are able to identify good political leaders based on those leaders’ stated values, voting record, platform or political philosophy. In addition, students report not keeping themselves informed of current events or are even aware of current issues within their own communities.
The sensitivity of the UniLOA to accurately measure behaviors appears to be strengthened as we contrast scores from 2008-2009 to those of 2007-2008. Scores were appreciably higher in behaviors such as researching candidates, being aware of current events, and voting in the previous year, a phenomenon we attribute to the heightened focus on political issues in light of the presidential election year that resulted in an appreciable and long-overdue degree of “diversity” occupying the White House.
Unfortunately, the UniLOA didn’t exist when Caryn’s article appeared in 1993 so the instrument hasn’t provided longitudinal data to know if we’re doing better, worse, or just the same in supporting citizenship GLD, but we have a baseline of functioning reflecting college students’ citizenship behaviors today and against which we can contrast behaviors in the future. Hopefully initiatives like the American Democracy Project, a commitment to engaged learning through community involvement, as well as other national and local projects will serve to eventually improve college students’ citizenship behaviors; that is, if higher education in general sees the value of developing citizenship. Whether or not higher education in general (based on what institutions DO, not what the merely SAY they do) determines that citizenship is an essential college learning domain is yet to be answered.
A full UniLOA report can be accessed at this website.