Sealund’s Serious Games Blog

What comes first – the game or the learning?

Consider the Zen of fun. Watch people of any age engaged in having fun, playing any type of game, and you can see how engaged they truly are, full present in that moment, that behavior. We know that engagement enhances learning accuracy and retention. The perspective from which Serious Games developers approach the “engagement factor” depends on what we have done (what habits of thought we have learned) previously – developed purely entertainment experiences or engaged-learning experiences. From either perspective, the development teams had to follow a certain logical path to create effective Serious Games.

The logical path is this: Games are great fun. Fun is one aspect of a pleasurable experience. The pleasure centers in the brain stimulate us to want more of the pleasure they register. The brain’s pleasure centers connect closely with the brain’s emotion centers. We remember longer and more clearly experiences linked with emotions. So the “rush” of gaming (see previous blog: Game-play Rush & Retention) becomes a component of instructional design for Serious Games.

The answer to the question of whether the entertainment or the learning comes first is: the learning. Learned behavioral outcomes are the “beef” of business requirements for Serious Games. The game is one technique—a powerful one—for achieving those requirements.

Purpose Directs Priority

Job performance improvement is job one for Serious Games. The results of Serious Games are measured at least twice – once in the scoring of the game, and again at least once on the job with performance measurements. Best practices suggest that post-training performance be measured shortly after the training and then at several-month intervals until full mastery is demonstrated consistently (which may require re-training if the employee hits a below-mastery plateau). These measures are important for employee performance rewards and, before the rewards, for coaching by supervisors. (For a classic consideration of job performance measurement purposes and strategies, peruse the International Society for Performance Improvement’s Measurement Counts! series of articles.) Entertainment game developers don’t—and shouldn’t have to—consider these Serious Games purposes, even when they’re developing Serious Games.

Serious Games open the door to exploration, competition, and achievement

That’s what the instructional designers are for: to ensure that the learning-driven architecture (design) is strong within the game. This means that the game’s adrenaline rush reinforces those behaviors the business demands, the business-required behaviors are scored, and the relevant outcomes are measured. Instructional designers do the architecture, indicate where the doors (into the learners’ memories) go, and perform effectiveness inspections. Game developers make the world of the game so engaging that the learners are eager to go through those doors – explore, take risks, compete, win! Game developers and learning game developers can learn from one another and jointly bring value to Serious Games.

Differentiators Between Entertainment and Serious Games

How can you tell a soundly designed Serious Game from one primarily for entertainment? Both have engagement, challenge, competition and rewards. Serious Games, however, demonstrate their purposes in the players’ paths to the rewards – and in the nature of the rewards themselves. The paths take players through challenges related to their work, either directly or metaphorically. And the rewards may go beyond the satisfaction of a high score to actual workplace rewards. It’s the instructional designer’s job to make sure the game puts the emotion-tickling challenges at the crucial learning points and that rewards reflect business values.

It takes a village to create Serious Games. The collaborators include business analysts, instructional designers, game developers, systems engineers, 3D animators, quality assurance specialists and project managers. Each has their role and the roles overlap into teams. It is imperative that the architecture of instructional design is the framework for learning. The development teams win the “game” only when they hit the business requirements squarely on target by following sound instructional design principles and best practices for Serious Games.

What do you think? Please share your comments.

Serious Games = Serious Learning!™

For more information about Sealund’s Serious Games, please visit our website at www.sealund.com

Sealund’s Performance Improvement team has been researching the empirical evidence of the effectiveness of Serious Games, with which to assure our clients that Serious Games achieve serious performance. We know from experience that workplace performance, based on retention of knowledge and transfer of knowledge and skills directly to the job, depends on the learners’ depth of engagement in learning experiences. So it makes sense that learners’ intense engagement in Serious Games, such as simulation-based games, would enhance both retention and transference of newly learned abilities – and provide a valuable ROI, as well as ROE.

Little Empirical Corporate Research Published

Bottom line – our review of published research demonstrated the high level of Serious Games’ effectiveness when selected as the appropriate instructional medium.We also want to share with you some of the complexities we discovered in the research itself. As we investigated the empirical research, we found that some researchers found transference of soft skills to the workplace difficult to measure, and that some analysts’ statements about the implications of findings are ambiguous. These predicaments may result from the fact that the bulk of published research has occurred in academic environments, business schools whose students’ on-job effectiveness could not be measured. Corporations that analyze training’s transference to the job as a means of performance improvement rarely publish their findings publicly.

Serious Games make training a success

Statistical and Anecdotal Research Findings

Particularly interesting for its observations on the significance of instructional design in achieving optimal learning outcomes, one 2005 meta-study reviews “the outcomes claimed in journal articles that report empirical work, indicating the usefulness of the frameworks, and the necessity to consider the role of affective learning. The article ends with some comments on the relationship of instructional design to effective games and learning outcomes.” [Classification of Learning Outcomes: Evidence from the Computer Games Literature. Harold F. O’Neil, Richard Wainess, Eva L. Baker. Curriculum Journal, Volume 16, Issue 4, December 2005.]

In one specific (if very early) study, business school students who completed a series of business simulations modules commonly reported that the simulation was the first time they truly understood the financial modeling and analysis concepts they had studied theoretically. In the simulations, they could apply those theories, base decisions on them, and experience the consequences. [What’s in It for Me? Over, Under, and Around Using a Computerized Business Simulation. James Estes, University of South Carolina. Journal of Experiential Learning and Simulation, Volume 1, Number 1, 1979.]

Simulation and gaming give learners the chance to apply theories and techniques in business-oriented simulations. One study found that “especially [for simulations] involving quantitative techniques, it is often difficult to isolate the decision making process from the mechanics of the simulation.” The computer-based simulation designed for this study was meant to alleviate that problem. This study’s conclusions report that “students have always reacted with enthusiasm. IMS [Inventory Management Simulation] has exceeded our expectations as a teaching device.” Empirically, “two groups of students in an operations management course were given the identical exposure to inventory management except that one group ran IMS and the other did not. Both groups were given a multiple-choice exam, which included five basic knowledge questions on independent demand inventory systems. The group using IMS scored a mean of 4.23 (S=1.23) out of 5 correct and the group not using IMS scored a mean of 2.89 (S=1.37)out of 5 correct. The difference between the means is significant at the 0.0006 level.” [An Inventory Management Simulation. James A. Pope. Old Dominion University, Department of MIS/Decision Sciences. Journal of Experiential Learning and Simulation, Volume 3, Number 3 & 4, 1981.]

Another meta-review of research on Serious Games’ effectiveness, found: “Some games provide effective instruction for some tasks some of the time, but these results may not be generalizable to other games or instructional programs. No evidence indicates that games are the preferred instructional method in all situations. Instructional games are more effective if they are embedded in instructional programs that include debriefing and feedback. Instructional support during play increases the effectiveness of instructional games.” [The Effectiveness of Instructional Games: A Literature Review and Discussion. Robert T. Hays, Naval Air Warfare Center Training Systems Division. 2005]

Dispute Over Research Results Interpretation

One analytical review of many studies gives a clear picture of what is unclear in the existing research. “Substantial disagreement exists in the literature regarding which educational technology results in the highest cognitive gain for learners. It was found that across people and situations, games and interactive simulations are more dominant for cognitive gain outcomes. …When students navigated through the programs themselves, there was a significant preference for games and interactive simulations. However, when teachers controlled the programs, no significant advantage was found. Further, when the computer dictated the sequence of the program, results favored those in the traditional teaching method over the games and interactive simulations.” [Computer Gaming And Interactive Simulations For Learning: A Meta-Analysis. Jennifer J. Vogel, David S. Vogel, Jan Cannon-Bowers, Clint A. Bowers, Kathryn Muse, Michelle Wright. Journal of Educational Computing Research. Volume 34, Number 3, 2006.]

In February, 2008, the Serious Games listserv (hosted by Digital Mill) saw a lengthy exchange about empirical research into Serous Games’ effectiveness. Many participants expressed dismay at the number of studies asserting that learning games are no more effective than other media, when their own experience contradicts that. One listserv member, himself an author of one of the studies cited in the previous section, explained those assumptions: “The reason you’re seeing all those negatives regarding games goes back to the 1983 article by Clark. His meta-analysis resulted in the often-misunderstood statement that you cannot learn from media. People often misinterpret that statement to mean that if you expect people to learn, you better not use media. What the statement really means is that media is simply the delivery vehicle, and it is the instructional methods and strategies built into the specific implementation of that medium that will determine whether learning will occur. [Richard Wainess, Ph.D., Senior Educational Researcher, CRESST/UCLA, National Center for Research on Evaluation, Standards, and Student Testing, University of California, Los Angeles]

Importance of Analysis & Instructional Design

The bold-print information in all citations above reinforces what instructional designers already know: people learn best when the training medium is appropriate to the subject matter and they’re in control of the learning process. Because learner-control is a key characteristic of Serious Games, proven instructional best practices support empirical and anecdotal findings that, for the types of subject matter appropriate to them, experiential Serious Games are more effective than other media. That’s why at Sealund, we begin every project with analysis of the client’s subject matter and required learning outcomes. We want to ensure that each learning solution uses the appropriate medium or blend of media for the project’s content and goals.You may find these resources useful for your own review of research:

• The Association for Business Simulation and Experiential Learning website contains a number of research papers on the effectiveness of simulation-style Serious Games in the Bernie Keys Library.
• The Education & Information Technology Library contains peer-reviewed and published articles.
• A Google Scholar search on “serious games effectiveness” (or “learning games effectiveness) returns a long list of articles.

What do think? Please share your comments.

Serious Games = Serious Learning!™

For more information about Sealund’s Serious Games, please visit our website at www.sealund.com