Via: Game Developers Conference 2010 - Serious Games Summit
SGS Session: The Games for Learning Institute: Research on Design Patterns for Effective Educational Games
Speakers: Jan L. Plass (Associate Professor and Co-Director, G4LI, NYU), Ken Perlin (Professor, NYU), John Nordlinger (Senior Research Program Manager, Microsoft Research)
Session Description
What makes games fun and engaging, and how can these concepts be measured? The Games for Learning Institute (G4LI), a multi-institutional collaboration of 14 game design and education faculty from 8 universities, conducts research on design patterns for effective educational games.
This session will provide an update to the G4LI collaborators and research agenda, present educational games developed by G4LI, and describe the research methods used to evaluate these games in middle school settings. For these studies, G4LI researchers have developed an instrumented game design architecture that allows extensive user tracking, research methodologies to measure fun and engagement with a combination of biometric and behavioral data, and methods for integrated assessment of learning outcomes and learning strategies. This session will prove an overview of these activities.
Idea Takeaway
Attendees will learn about:
- The common research approach taken by the 14 members of G4LI at 8 universities
-The research methods incorporating behavioral data from user logs, biometrics, and user reports
-The instrumented game architecture that allows for the design of games and the integrated assessment of learning outcomes and learning strategies
-Summary of research results to date
About G4LI
The Games for Learning Institute (G4LI) is a first-of-its-kind, multi-disciplinary, multi-institutional gaming research alliance that will provide the fundamental scientific evidence to support games as learning tools for math and science subjects among middle school students. It is great example of how technology can play a role in changing how students learn and give teachers new tools to create dynamic and effective curriculum.
Bringing Game-Based Learning to Scale: The Business Challenges of Serious Games
Excerpts
Beginning with Jim Gee’s recognition of bona fide learning processes in his and his son’s video-game playing, and propelled to public recognition through Ben Sawyer and David Rejeski’s launching of the Serious Games movement and its attendant conference, the use of video games to teach academic content has now reached a level of national interest.
Game-informed learning is now the basis for a new urban school serving grades 6-12; numerous colleges and universities offer game design courses; engineering professional societies have launched member sections devoted to games; foundations have recently poured millions of dollars into research on games and learning ($50M from the MacArthur Foundation alone), and the venerable National Academies has begun to hold workshops on the topic.
Why Learning Games Are So Hard To Find – It’s Not What You Think
Presuming, for the moment, that games can teach, the question arises as to why a wildly popular medium in other spheres has not gained a greater foothold in both formal and informal education.
Several initial hypotheses can be discarded almost immediately – for example, that there exist no games that teach academic subjects.
Indeed there are many, ranging from games covering physics (e.g. Crayon Physics, Physicus, Coaster Creator to math (e.g. DimenxianM for algebra, NIU-Torcs for numerical methods) to mechanical engineering (Time Engineers), to network engineering (Mind Share) to biology (Virtual Cell, Cell Saver, Immune Attack) to ecology (Resilient Planet, River City, Quest Atlantis, Wolfquest ) to many others.
A secondary hypothesis is also contraindicated: that games covering academic
content, particularly math and science, are doomed in their attempts to attract an audience because the subject matter itself is so inherently dull.
Another hypothesis offered to explain the near-invisibility of “games that teach” is that a truly fine exemplar – one that captures all the production value of a AAA entertainment title, thereby commanding public attention and generating high profits – has yet to be built.
The lack of a $100M academic game is, as far as the author knows, a true fact. Production costs for academic learning games tend to be much more modest, typically no more than $5M and most commonly under $1M.
In his presentation, “Serious Game Production,” Noah Falstein, whose commercial credits include Star Wars: Empire at War and Indiana Jones: The Last Crusade, as well as several “serious game” titles, explains what one obtains for increasing levels of development cost: more scope (more game levels and story lines to explore), higher quality graphics (3D rather than 2D, higher realism, larger world or variety of places to inhabit), more extensive playtesting and production value (better software engineering for faster game response time, higher frames per second, professional sound, fewer execution bugs, more powerful physics engine – leaves flutter in the wind; items break according to resolved forces instead of stored animations).
He gives the following examples to illustrate the cost hierarchy, indicating what each price point was able to buy.
From Falstein’s list:
$5K: Example Happy Neuron (brain teaser)
2D
simple puzzle-type game (brain exercise)
Based on a prior game’s mechanics
Less than 2 months’ development time
$15K (approx): Example: Airport Insecurity (Airport security practices)
2D
original game design
more complex game mechanics (game outcomes modeled on real TSA reports)
6 months’ development time (estimated)
$200K: Freedom Fighter 56 (Hungarian revolution)
2D, detailed art
original game design; graphic novel with embedded mini-games
15 months’ development time
$2M: Re-Mission (first person shooter, cancer treatment)
3D
Detailed but not high realism graphics
original game design
User testing and multiple redesigns to ensure product met effectiveness benchmarks (change in patient attitudes and health)
18 months’ development time
$10M: Gears of War (3rd person shooter)
3D
original game design
Highly realistic graphics
Up to 8 simultaneous players
$10M (cheap by entertainment game standards)
3 years’ development time
$50-$100M: World of Warcraft (fantasy)
3D
Fantasy art that spans continent-sized regions and takes years to exploreDetailed visual effects – light filtering through trees, footprints dissolving in the beach sand, mist rising from the ground
Tech services that respond to and support 8.5 million simultaneous users, on hundreds of different software/hardware configurations interacting with each other inside the game (as of 2009, over 10 million users)
5 years’ development time
What is instructive about this list is the relative independence of cost from engagement – measured either in terms number of users, or hours spent. World of Warcraft, the most expensive game on the list, has over 10 million players currently.
However, Happy Neuron, at an original $5K development cost, has since expanded into a suite of 35 similarly-themed quick ‘n easy brain teaser games with 35 million “exercises completed”. Each of its 125,000 users is therefore, on average, completing 280 game exercises.
Whyville is also rendered in cost-effective 2D. Its cartoon-drawn characters, some of which are no more than mere ovals, have attracted 5 million players, from humble beginnings of $30,000 and one in-world activity.
Thus, for orders of magnitude lower production cost, simple Flash-based games can be very addictive, and with a profitable business model they can grow to significant size.
With respect to science education, many of the features that can be purchased at the tens-of-millions-of-dollars level have yet to be equated with improved learning outcomes. Characteristics that have been equated with improved learning outcomes include learner control (ability to navigate through the game under one’s own choice, instead of being “led along”) and in-game user help systems. The former is also a large contributor to user engagement, as measured quantitatively in, and the latter is largely responsible for the exceptional learning outcomes of low achievers when they learn via game.
Classroom research has shown that cooperative learning improves learning outcomes by 25% over solo learning; if we extend this concept to games, then multiplayer design should prove to produce superior learning outcomes over single player design. Note however, that none of these proven or potential learning-related features are show-stopping cost drivers. Even the most complex of these, the multiplayer structure, has already been adopted by several medium-budget commercial learning games: DimenxianM from Tabula Digita, Making History from Muzzy Lane Software, and Whyville from Numedeon.
And, to make the cost issue even less germane, a slew of authoring products are now becoming available to drive game development costs down further. These include Blender, a free high end graphics tool; Multiverse, a free high end game development system, used to make customized virtual worlds; DARPA’s Real World, in which warfighters can easily create simulations of their own battleground experiences; and Torque, a low-cost game engine popular in academic circles.
Tools are also being created that will dilute development costs by spreading the effort among many co-developers: Medulla, being developed by the Federation of American Scientists, will enable structured multiperson collaborations on 3D environments, managing authentication, peer review, contractor payments, consumer ratings, volunteer participants and more.
In yet another development, the increasing adoption of standardized file formats for 3D objects, e.g., Collada, will enable copy and paste across 3D environments, so that game developers are no longer faced with the cost of having to develop new graphic assets from scratch.
Most of this work on free and easier-to-use game creation tools has been underway in just the past 5 years, but in aggregate, it should significantly reduce game development costs.
In conclusion, it appears it is not lack of product, or boring subject matter, or a more modest expenditure on game development that keeps learning games from attracting a significant audience.
Instead, this white paper proposes it is a series of business-related issues: finding a sustainable business model, creating or obtaining a distribution network, and achieving consumer acceptance.
Failed Attempts at Scaling – Why The Two Obvious Approaches Did Not Work
As described below, the two obvious approaches to bringing learning games, or Serious Games, to a national audience have failed primarily for business reasons unrelated to lack of product, content, or initial cost.
In the first approach, someone attempts to “scale up” a game developed in an academic research environment. This approach almost universally fails, for two reasons: 1) the grant on which the game was developed flatly does not provide funds for commercial hardening, marketing, sales, distribution or other functions that would be required to allow the game to be provided at commercial scale and 2) even when such funds are externally provided – as in the Department of Defense (DOD) experiment described below – the game is so poorly designed from the outset to meet these challenges, and its academic developers so unskilled at the tasks required to get the product to a level of commercial acceptability – that failure is also almost inevitable.
The second obvious approach – using the commercial know-how, investment capital, and leverage of a large company and pushing academic content through their existing pipeline, has also failed. Large entertainment game companies have difficulty seeing how their existing business model of enormous up-front investment, followed by mass uptake by an audience of millions in a few months following release, would work for educational offerings.
In 2003, Microsoft held a Higher Education Leaders Symposium, in which it concluded that high quality educational games would be “exceedingly expensive to build with a business model geared toward a mass market of consumers”. The question of “who will pay for development” weighed heavily in those proceedings, since the market Microsoft was experienced with was not the market that would buy educational games, and the obvious market of educational institutions (colleges, schools) was not seen as having deep enough pockets.
Established educational software companies do exist but limit themselves to the K-6 market, an age bracket where parents still make software purchasing decisions for their kids. Even so, these companies have not been spectacularly profitable and are not in a position to take on risky investments in academically-developed software.
The Three Business Challenges: Sustainability, Distribution, and Customer Acceptance
Sustainability
Developing a sustainable revenue model from paying customers is one of three major dilemmas facing the scale-up of serious games, the others being distribution and customer acceptance.
Economic sustainability is certainly the issue on the minds of commercial game companies that have toyed with the Serious Games space. It is also the major question for academic developers. Scaling up from academic origins requires a process of product hardening and ongoing maintenance that is consistent with commercial quality software production (versioning, user testing, marketing, etc). Some of the new developers are beginning to take on these functions or provide them as a service to others. However, they need a sustainable source of revenue to cover these recurring costs. A one-time grant, that covers only initial game development, will not suffice.
Sustainability requires revenue that is tied to product use, rather than product development. Somehow, the users must pay.
Distribution
Distribution is the second major challenge faced by those wishing to bring immersive education to scale. As noted above, commercial game manufacturer and textbook companies have the needed distribution channels but do not have business models that are compatible with selling Serious Games.
The academic researcher/game developer, by himself, has no virtually channels at all. Posting one’s game on the equivalent of “myuniversity.edu/facultymember/course1201/lecturenotes/~game” is not distribution. In the absence of a significant marketing push to drive traffic to the otherwise obscure website, no one will go there.
Unfortunately, marketing is a task well outside most researchers’ expertise and the grants’ funding allowances.
Thus the establishment of new distribution channels, well suited to this new product genre, is a major challenge for expansion of the field.
Customer Acceptance
Consumer acceptance is the third challenge, after ensuring the product can be reliably sustained and actually be distributed to the customer. The white paper examines several known factors relating to consumer acceptance.
These three business issues: sustainability, distribution, and consumer acceptance are, in the author’s opinion, the three primary factors limiting the scale up of immersive education to million-user levels.