Skip to content

Assessing Our Work

By Mark Frederick, Indiana State University

In the Spring, 1993 edition of Peer Review, Caryn McTighe Musil, then AAC&U’s Vice-President for Diversity, Equity, and Global Initiatives, presented “Faces/Phases of Citizenship,” a developmental learning map” that could easily serve as a rubric for measuring college students’ growth, learning, and development (GLD) in the broad area of citizenship.  With that model, an opportunity came to be where higher education could expect a reliable answer to the simple question “are our students developing meaningful citizenship skills and if so, to what degree?”  And it seems that whenever an opportunity to measure something meaningful presents itself, there will be a social scientist that jumps into the fray to take on the challenges of operationally defining an important construct, collecting data, and reporting results to an educational environment that might or might not be “in the mood” to receive them.  But unlike Jack Nicholson’s Character in the movie “A Few Good Men” where he states “You can’t handle the truth,” those of us in higher education can ill afford to not “handle the truth.”  And the truth as to how and to what degree college students are becoming better citizens is being revealed, to the limits of the instrument, with the University Learning Outcomes Assessment (UniLOA).

The UniLOA measures student behaviors along seven broad domains, one of which is Citizenship.  To be sure, the UniLOA is a self-report instrument and most of us would prefer to employ direct measurement of student GLD, the resources just aren’t available for us to hire trained observers who will follow students “24/7” to accurately record their behaviors.  Ultimately, we have to rely on self-report, if for no other reason, than it’s the only method of gathering information we can really afford in higher education.  While there is understandably some question as to the reliability of self-reported information, we’ve found remarkable stability and consistency in results patterns over time as students from across the country have reported their behaviors when they are administered the UniLOA.  To that end, and with more than 50,000 student respondents, we’re confident that we have an accurate picture of the current state of citizenship development among college students across the nation.

Unfortunately, the news isn’t particularly good.  Of the seven UniLOA domains, citizenship scores the very lowest and individual items for the domain are well below the mean of other domains’ items.  Of the ten citizenship domain items, “I engage in the political process through voicing viewpoints” holds the honor of being not only the lowest-scored domain question, but indeed, one of the lowest scored questions of the entire UniLOA survey.  The second lowest-scored item suggests that students don’t contribute financially to causes they believe in.  On first blush, low levels of behavior would make sense among perpetually “broke” college students, but it would seem that they miss the point that “contributing” is a process rather than a relative value of the amount of a financial gift and giving just a little begins a process of learning “how” to contribute, regardless of the gift’s size.

Other items on the citizenship domain that score disappointingly low include students’ tendency to not vote as frequently as we’d like, that those that do vote, many fail to adequately research candidates before voting or even reporting that they are able to identify good political leaders based on those leaders’ stated values, voting record, platform or political philosophy.  In addition, students report not keeping themselves informed of current events or are even aware of current issues within their own communities.

The sensitivity of the UniLOA to accurately measure behaviors appears to be strengthened as we contrast scores from 2008-2009 to those of 2007-2008.  Scores were appreciably higher in behaviors such as researching candidates, being aware of current events, and voting in the previous year, a phenomenon we attribute to the heightened focus on political issues in light of the presidential election year that resulted in an appreciable and long-overdue degree of “diversity” occupying the White House.

Unfortunately, the UniLOA didn’t exist when Caryn’s article appeared in 1993 so the instrument hasn’t provided longitudinal data to know if we’re doing better, worse, or just the same in supporting citizenship GLD, but we have a baseline of functioning reflecting college students’ citizenship behaviors today and against which we can contrast behaviors in the future.  Hopefully initiatives like the American Democracy Project, a commitment to engaged learning through community involvement, as well as other national and local projects will serve to eventually improve college students’ citizenship behaviors; that is, if higher education in general sees the value of developing citizenship.  Whether or not higher education in general (based on what institutions DO, not what the merely SAY they do) determines that citizenship is an essential college learning domain is yet to be answered.

A full UniLOA report can be accessed at this website.

3 Comments Post a comment
  1. Mark,

    Your points about measuring citizenship development at universities, resonates with themes of a piece I wrote yesterday, before I attended the screening of a new movie about Hubert Humphrey at the University of Minnesota last evening.

    Measuring citizen agency not only can insure universities live up to their ideals. I would say it might also result in citizens seeking proof that public leaders see citizens as leaders, too. In fact, with the piece, I suggest a movement parallel to yours: one in which citizens demand measurable proof that politicians play nice in behalf our country — by co-producing solutions, not defending their positions of power.

    This informs the work of Nonpartisan Productive Dialogue (transitioning to a new name: DynamicShift). Perhaps it will resonate with the movement you are developing.

    Here it is:

    “How to Prove that Democracy Works”

    Andrea Grazzini Walstrom
    Founder, Nonpartisan Productive Dialogue


    November 12, 2010
  2. Andrea,

    Advancing an agenda designed to promote positive college students’ behaviors across the broad domain of citizenship would best be accomplished by coordinating the work of many concerns, including the Nonpartisan Productive Dialogue project and the American Democracy Project.

    Your suggestion of a “parallel movement” seems to be critical to fashion a broad response plan by all concerned if those interests are able to agree to and clearly articulate desired goals and outcomes.

    I wonder who or what concern might “play point” by providing umbrella guidance and coordination of the work of the many in the years to come…….


    November 15, 2010
  3. In partnership with the Center for Democracy and Citizenship, we are building just such a movement on the work we’ve done in our Civic Agency initiative. To read more about it, please visit this website:

    If you’d like to get involved, please contact me.


    November 15, 2010

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: