Liked Posts

On research and evaluation of a learning game: Alicia Menendez

imageimageOur daily development blog series for the Game Changer Chicago Design Lab’s alternate reality game The Source continues with the perspective of Alicia Menendez, Associate Professor in the Harris School and the Department of Economics at the University of Chicago, and a Principal Research Scientist at the National Opinion Research Center (NORC). She writes about research on games and learning.


Most days, I can see the teens that play the alternate reality game called The Source through my window. They go up and down 60th Street, heading over to some activity on campus or returning to the main game space. They seem engaged, enjoying themselves, and having a good time.  It makes me smile. We are all happy and that is great but… are the players learning anything this summer? Are they developing new skills? Could this have an impact on their grades next academic year? 

My name is Alicia Menendez and I am an economist at the University of Chicago and a member of the Center for Interdisciplinary Inquiry and Innovation in Sexual and Reproductive Health (Ci3).  A large component of my research work focuses on human capital - education and health - and I have been working on evaluating the impact of programs for several years.  When my colleagues Melissa Gilliam (Professor of Obstetrics/Gynecology and Pediatrics Chief, Family Planning) and Patrick Jagoda (Assistant Professor of English and new media studies) told me about The Source, I was instantly interested. Could an alternate reality game be an effective way of learning and develop non cognitive skills? This summer we are piloting the idea.

Our initiative is interdisciplinary. Team members come from the humanities, biological, and social sciences. Our skills and perspectives are different but we are all committed to a serious evaluation of this innovative approach.  We want to measure different aspects of the game using quantitative and qualitative methods. This summer we started by conducting a very exploratory evaluation of The Source to learn as much as possible about the game players, the interactions among them, and the effects that the game may have on them.

With the knowledge and consent of the youth playing the game and their parents, we have collected information about them before the game started. We created a survey that gathers demographic information, questions about personality and self esteem, preferences and attitudes towards STEAM learning, measurements of altruism and reciprocity, attention tests, self control measures, etc. For the length of the game, we constantly monitor the participants’ individual and team progress. Once the game reaches its end and again 12 months later, we will collect additional information among participants. All those data, along with administrative information, will be used to evaluate the experience, to try to understand what works and what needs improvement, and to quantify the effects of the program.   

Among other things, we have all been very keen to learn about the effects that badges may have. All the players of The Source can earn digital badges if they complete the requirements.  However, this form of incentive system is currently at a very experimental stage. We would like to learn how participants behave when offered badges as a reward and what characteristics of the players can increase the probability of completing all tasks in order to earn badges.

I am looking forward to working with all the data we are collecting. I hope this will be a summer of learning for everyone involved with The Source.