A few months ago, I started taking notes for a post titled “Quality Should Not Be a Dirty Word.” This was initially prompted by reading that Ed Norton (an actor whom I like) would be starring in the next Hulk movie (a franchise I think could be fun), but that the movie would be directed by the fellow who did The Transporter 2 (a fairly abysmal movie). The disappointment I experienced made me want to write a blog post, and it seemed geeky enough to fit in here, but then I realized that it seemed somewhat out of bounds for an academic blog: Media researchers aren’t supposed to make evaluative judgments like this. That kind of reaction is for fans—though, when you think about it, it’s not like media researchers’ tastes don’t influence what they write about. Thoughts of the Hulk behind me, I suddenly started taking notes on the relative lack of research and reflection on the how aesthetic standards are formed and applied, including by academics (at least since Bourdieu closed the book on it for many since he described taste in terms of class values).

I saved the post as a draft and never really got back to it. The argument in my notes, anyway, was this paragraph:

Perhaps our goal, then, should not be to calculate a universal rubric for quality or engagement, which results in theories that seem reductive, overreaching, or awkwardly cobbled together, like the (admittedly interesting and ambitious) GameFlow model does for games. Rather, we should identify what it is that individuals and communities find rewarding about their favorite works. Not, “Why is Star Wars so good?” but “What is it that people find good about Star Wars?” Closely related to this is the question, “What do people do with star wars?”

I ended up following up on this idea somewhat in my recent post on how we should evaluate “art” in games, suggesting that we break our evaluation down into formal elements (like focusing on “atmosphere” versus “storytelling” in games) and practical results (like lasting historical relevance, or inspiring immediate political action).

Outside of games, another specific example for this (to be described in the aborted essay) was the example of “worldbuilding,” which Cory Doctorow describes as “the science fiction writer’s practice of meticulously (or not) drawing up the contours of a fictional place—from the physics to the biology.” Cory’s not a fan of this practice. An essay he links to on this topic suggests that M. John Harrison is not a fan either; he calls worldbuilding “the great clomping foot of nerdism. It is the attempt to exhaustively survey a place that isn’t there. A good writer would never try to do that, even with a place that is there.” And, actually, I’m not a big fan of reading other people’s worldbuilding either (though I recall from my high school days that worldbuilding was the most fun part of designing homemade roleplaying games). Nevertheless, we must recognize that for some, at least, worldbuilding not only holds certain appeal as a formal element, but may be evaluated as effective or ineffective by audiences. We can see differences in fannish appraisal of quality in worldbuilding when we contrast responses to how this was handled in the original Star Wars trilogies and the material that followed with the prequels, or in viewing fan reaction to different Tolkien books or the comparison between the books and the movies. Here, discussion of quality is not monolithic and oppressive, defined by the wealthy classes and handed down from on high, but posed in context, framed by questions of use: quality for whom and according to what? Socioeconomic class clearly plays a part, as Bourdieu discusses, but that may not tell the whole story of how taste is made in more specific instances.

There you have most of my notes for “Quality Should Not Be a Dirty Word” (minus some additional babbling about Stuart Hall and jazz). I’m not going to finish that essay, though, because I have much else to do, and, more importantly, because Jason Mittell already did a fine job of it and posted it to his blog. “Lost in a Great Story: Evaluation in Narrative Television (and Television Studies)” not only reflects on media researchers’ shyness about issues of evaluation, but attempts to apply a culturally framed evaluation to a particular media object, the television series Lost. He gets away from the all too typical academic approach to television which cloaks author opinion with terms like “complexity” or “nuance” (and is dead on in pointing out this practice), and instead suggests some specific formal factors that keep people coming back to the show.

I do think the analysis could go a bit further. A noteworthy counterargument comes in the comments that follow from Ian Bogost, who suggests that fans’ faith (that all the plot twists will fit together as if planned) seems is quite likely to be misplaced. (Though I disagree that leaving giant questions lingering between seasons is a way of fleecing suckers in the audience, which I find analogous to arguing that each chapter of a book should leave no major questions dangling). More niggling to me is the question of why I should accept the argument that Lost‘s use of the techniques described here is superior to Heroes‘s use, which seems attributed to some unidentified mass of fans and critics. Overall, though, I think this is a noteworthy step toward discussing issues of quality assessment in media research, and I’m sure glad some other Jason took the time to do a good job on it.