Earlier tonight, a small group of my friends gathered to play games together. I made a comment at dinner about how I should’ve brought Guitar Hero, which I haven’t played in a while, and my friend Caralyn, who is less a gamer than others present, warily asked if we would be playing board games or video games. The plan was to play board games, I said, and she replied, “Oh good, I know how to play those.” Being the huge nerd that I am, I commented that this was a pretty interesting comment to me, considering that video games tend to have only a limited number of input mechanisms—the buttons on the controllers—whereas board games have a theoretically limitless number of input methods. “Whatever,” she said, “I haven’t played video games since Duck Hunt.”

My friend isn’t the only one who thinks of playing modern video games as somewhat akin to reading a foreign language (except that when you butcher a foreign language, at least you don’t have to watch a tiny avatar of yourself being helplessly destroyed). As Ernest Adams suggests in a recent Gamasutra article (link via Kotaku), action games are typically only playable by hardcore gamers, inaccessible to newcomers and disabled players. What is it about some games that makes my friend think she knows how to play them, while others seem impenetrable? I’d like to consider for a moment that these sorts of examples might point to a certain “video game literacy” that players are expected to possess, and which might have fairly unique implications for game design.

I use the phrase “literacy” with some hesitation, knowing the kind of baggage it carries. Video games represent not a verbal language, but a specific medium, generally with a major visual component (hence the “video” part of the name). Would a theory of video game literacy resemble other theories of visual literacy, then, such as comics literacy? And if so, which theory of visual literacy should it most resemble? Just to break this down a little, let me be clear that we’re potentially talking about a few different things when we talk about visual “literacy”:

1. A learned understanding of symbolic narrative references. Matthew Pustz’s definition of “comics literacy” encompasses this. In Comic Book Culture, he argues that one must be comics literate to recognize that a rose in an alley represents Batman’s deceased parents. This is more akin to recognition of Biblical and mythological allusion than to what we more commonly understand as being the divide between literacy and illiteracy, however.
2. A learned understanding of syntactic conventions. Neil Cohn addresses this in his theories on “visual language.” He writes about a learned ability to recognize how sequences of images (e.g., a layout of panels on a page) provide a linguistic system analogous to verbal language.
3. An innate recognition of basic signs. Paul Messaris (my advisor) explains in Visual Literacy how research in perceptual psychology demonstrates that some of our ability to process images is universal and innate, unlike learning a language. This is because some images are iconic, meaning that they resemble what they represent, while most words (aside from pictograms and onomatopoeia, for example) are purely symbolic, arbitrarily matched up with what they represent.

You can use these definitions of visual literacy to describe video game literacy, but only up to a point. The tricky thing about games is that there’s more than just the signs that you read off the screen: there’s also the “language” of input or control, matched (or not) with what happens on the screen.

Going back to the example of Duck Hunt, then, we can see all of these kinds of literacy at work. You still need some acquired understanding of cultural context and history of the medium, similar to that described by Matthew Pustz: even if the image of the hound is not enough to signal a hunting trip, previous experience with games would give you the knowledge that moving objects and a plastic weapon in your hand indicate that you should shoot the targets. You also need some formal literacy related to somewhat abstract conventions and syntax, akin to that described by Neil Cohn: you must keep track of your own ammunition with the icons displayed on-screen, and previous experience with games would reassure you that the scores appearing in the place of shot ducks can safely be ignored.

Finally, at the most basic level of interpretation, Duck Hunt is particularly accessible, requiring less knowledge of acquired literacies, because it is iconic in both the basic units of imagery and input. The cartoonish graphics are reasonably iconic, so you know you’re looking at ducks who visibly die when shot. The input is reasonably iconic because Duck Hunt doesn’t use a regular controller, but something that resembles an object in real life; the action you perform with it is not arbitrary, but analogous to how you would use that object’s real-life counterpart (i.e., pointing and pulling the trigger).

Arguably, of course, all video game play is iconic to some extent: pushing a joystick or pressing a directional pad to the right, for example, moves a character right on the screen. I would still argue that this language is even more frequently symbolic, however: pushing the X button does something that does not at all resemble button-pushing onscreen. Video games with custom hardware and gesture-based technology, such as Guitar Hero and many Nintendo Wii and DS games, make games more accessible by making input styles more iconic.

Admittedly, this is a fairly long-winded and theoretical way of arriving at a pretty obvious point: some people feel like they don’t “get” games because it’s not obvious how to play them. I think it’s important, however, to make distinctions between different forms of literacy, between iconicity in graphics and in gameplay, and between the very idea of ‘iconic’ and ‘realistic’ as might be applied to games.

I think it’s unnecessarily confusing to use the term “literacy” as it has been used to refer to recognition of traditions and narrative symbols. Nevertheless, the inability to recognize these components is far more problematic in games than in other media. As a “literate” person, you might not get as much out of a Batman comic, a Shakespeare play, or a Tarantino movie if you don’t know what the subtle allusions refer back to, but you’ll be able to get through the whole thing and at least recount the basic plot. If you approach a game without any knowledge of the rules and without any experience with other games of the same genre, you might not get past the first thirty seconds of play. Knowledge of references to older games is sometimes intimately connected to knowing what one can do in a game. After all, a lot of the basic input language of games is based more on tradition than any reasonable, formal purpose.

Even in the case of a relatively iconic gameplay style, such as in an arcade shooting game, it’s easy to miss the flashing message warning you to shoot off-screen to reload. Several months of participant observation research in arcades suggest to me that people who haven’t played a point-and-shoot game since Duck Hunt tend to keep clicking at the screen even after running out of ammo. Similarly, a person who has never played fighting games before will not walk up to an arcade cabinet without posted instructions and try pushing the joystick down, forward-down, and forward, with a button push at the end—but I’ll bet many people reading this could tell me one character to pick (in any of several games) to make this move produce a projectile attack, and even recite what this move is traditionally called. People who have played games for years tend to take for granted certain conventions that remain mysterious to the uninitiated.

While I’m no more interested than the next gamer in giving up the game genres I love, I’m inclined to believe that growth and outreach mean embracing more iconic gameplay and leaving behind tradition for its own sake. This is not the only tradition we may need to loosen up on: common wisdom seems to suggest that video games need to become more realistic in their graphics in order to sell more and reach out to broader audiences. The startling popularity of the graphically inferior Wii console, which happens to also be associated with more cartoonish games, should already be putting the graphics myth in its place. This is not to say that graphics are unimportant, but that iconic graphics are sometimes good enough (a good cartoon is still iconic) and that iconic input may sometimes be better.

Arguing for more iconic gameplay probably opens up a whole other can of worms, of course. Critics are already quite upset about players being able to bring about the realistic death of virtual people by pressing a few buttons; they will be none too thrilled when someone gets the bright idea to use the cord between the Wiimote and Nunchuck attachment to strangle some virtual person. For what it’s worth, though, Caralyn survived her Duck Hunt experience to grow up into a vegetarian (without a hunting license). Something tells me we might be able to lure her away from the board games, too, if we get a game of Wii Sports going.