milgames thread – Do Serious Games Work?

Micheal Peck asks the following on the milgames list (see links area) and gets quite a varied response.

The question of “has there been a research effort to quantify how much better they are” might be addressed by some here. My direct experience with several “serious wargaming” projects is that despite the effectiveness, there is an inertia due to learning curve on the part of educators and students who are not gamers that makes this more of an “innovation diffusion” problem than an if they were really so effective they would be used a lot more”. If you have not read Everett Rogers seminal work “Diffusion of Innovations” I can’t recommend it highly enough. The 5th edition is available for Kindles.

Back to the questions – the questions that need to be answered in addition to the ones below include:

“How do I convince educational leadership to replace “legacy” classroom lecture time with “game time” in the negative sum curriculum game” (ie every new version of a curriculum these days is supposed to get sailors back to the fleet faster).

“How do i integrate embedded shipboard training with classroom “game training” to establish a “continuous learning environment” that keeps inertia after the student leaves the classroom? How does this integrate officer and enlisted training?”

“How do i convince the “metric heads” that educational experience can be derived from games that are not based on VV&A model stacks that has been millions in the making?”

How do i overcome the fundamental bias against gaming in the workplace as “not serious” even when everyone agrees it its beneficial? (We don’t have overhead to invest in “real” personnel development initiatives, let alone give the “perception of impropriety” associated with “playing games at work”.

These are the sort of cultural issue Rogers deals with in Diffusion of Innovation (in the negative case of – innovation will not Diffuse, even if obviously “better” if there are even modest cultural impediments.

When it comes to the effectiveness of serious games, most of the evidence I encounter seems to be anecdotal. The designer will tell me, “People liked it” or “Someone told me after the game that they learned more this way”.

I’m taking a look at this issue, and I’m curious about the current consensus regarding games in the defense arena. There has been a lot of hype and buzz about games. Assuming that games are cheaper and more convenient than other training methods (a questionable assumption?), it seems to me that games boil down to three possibilities:

1. They are less effective than other training methods for a given requirement, but their low cost and convenience makes them the superior option.

2. They are equal in effectiveness to other training methods, but their cost and convenience wins them the tie-breaker.

3. They are more effective than other methods, and they’re cheaper and more convenient. A triple crown.

And just for fun, let’s flip the question. When would you absolutely not use a game for training?


Michael Peck
Training & Simulation Journal

About Paul Vebber

"If you read about something, you have learned about it. If you can teach something, you have mastered it. Designing a useful game about something however, requires developing a deep understanding of how it relates to other things."
This entry was posted in I was too lazy to categorize this post. Bookmark the permalink.

5 Responses to milgames thread – Do Serious Games Work?

  1. Jon Compton says:

    Seems to me that there’s quite a bit of literature already out there that goes some way to showing the effectiveness of immersive/gaming experiences for learning. It’s been demonstrated pretty clearly that they increase contextual synthesis and dramatically increase peer-learning. Thus, I’m somewhat at a loss as to why we’re still asking these questions. As for convincing metric heads, you might as well try and teach a fish to walk. When all you have is a hammer, the world looks like a nail.

  2. Paul Vebber says:

    Mike is trying to collect the actual studies for an article and has been coming up blank. There are a lot of articles with discussion of anecdotal evidence, according to him, but very little he has been able to find that is rigorously executed. If you are aware of such stuff, he would be very happy to here from you!

  3. Jon Compton says:

    Since most of these studies were conducted by educators for educators, I doubt you will find anything done at the level of rigor he’s looking for. But I’d suggest searching for studies on Active Learning rather than gaming.

  4. Paul Vebber says:

    This report from 2005 is one of the better ones I’ve found, old, the only survey-type paper I’ve found with a fair amount of looking…

    Click to access GetTRDoc

    Summary of findings:

    This review of empirical research on the effectiveness of instructional games leads to the
    following five conclusions and four recommendations. The conclusions are:
    * The empirical research on the effectiveness of instructional games is fragmented. The
    literature includes research on different tasks, age groups, and types of games. The research
    literature is also filled with ill defined terms, and plagued with methodological flaws.
    9 Although research has shown that some games can provide effective learning for a variety of
    learners for several different tasks (e.g., math, attitudes, electronics, and economics), this does
    not tell us whether to use a game for our specific instructional task. We should not generalize
    from research on the effectiveness of one game in one learning area for one group of learners
    to all games in all learning areas for all learners.
    o There is no evidence to indicate that games are the preferred instructional method in all
    o Instructional games should be embedded in instructional programs that include debriefing
    and feedback so the learners understand what happened in the game and how these events
    support the instructional objectives.
    o Instructional support to help learners understand how to use the game increases the
    instructional effectiveness of the gaming experience by allowing learners to focus on the
    instructional information rather than the requirements of the game.
    The following four recommendations may help the instructional gaming industry produce
    more instructionally effective games.
    o The decision to use a game should be based on a detailed analysis of the learning
    requirements and an analysis of the tradeoffs among alternate instructional approaches.
    o Program managers and procurement personnel should insist that game developers clearly
    demonstrate how the design of a game will provide interactive experiences that support
    properly designed instructional objectives (see for example, Gagn6 & Briggs, 1979; Merrill,
    1983; 1997 for guidance on the proper design of instructional objectives).
    * Instructors should view instructional games as adjuncts and aids to help support instructional
    objectives. Learners should be provided with debriefing and feedback that clearly explains
    how their experiences with the game help them meet these instructional objectives.

  5. Skip Cole says:

    This paper has some good insights on this:

    The take away lesson for me is that games can provide good motivational experiences, but may be no better than other forms of teaching when judged by other criteria. But having people be involved in making the games has definite, demonstrable benefits.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s