Week Summary: 26/1-30/1

This week we dug into the world of comics, film and role playing games (Both Live action and pen and paper) in search of interesting input, ideas and in order to expand our vocabulary and understanding of ways of communicating meaning to the viewer/player.

All three of these subjects gave unexpectedly positive results especially when it comes to understanding the concept of Diegesis. It seems as though insights gained from these different areas will blend together well and form a good framework for the rest of the thesis.

The game study is coming along fine, and we have, as of today, analyzed 12 games - and patterns are really starting to emerge when it comes to charting the design space.

McCloud on How Words and Pictures are Combined in Comics

After a quick penetration of McClouds "Understanding Comics", one thing that could be directly related to our project is the variations of interplay between words and pictures in comics. Words and pictures can be regarded as the information channels in comics (together with shape of panels and space between panels).

According to McCloud, words and pictures are combined in the following ways:

Word specific combinations: "pictures illustrate, but don't significantly add to a largely complete text."

Picture specific combinations: "words do little more than add a soundtrack to a visually told sequence."

Duo-specific combinations: "both words and pictures send essentially the same message."

Additive combinations: "words amplify or elaborate on an image or vice versa."

Parallell combinations: "words and pictures seem to follow very different courses - without intersecting."

Montage: "words are treated as integral parts of the picture."

Interdependent combinations: "words and pictures go hand in hand to convey an idea that neither could convey on their alone."

This could be related to the notions of conflicting, redundant and complementary multi-sensory displays that Erik elaborated on in a previous post. I guess every single of the combinations above have their optimal uses for different purposes (immersive, narrative, etc.) - most interesting for us should be the interdependent interplay between information channels, to provide an experience as rich and immersive as possible for the player.

Ramblings: Symbol Interpretation

-I don't know if this will be of much use to us, but I found it quite interesting

This is a system for dividing symbolic (textual) interpretations into three distinct iconic meanings (from "Beyond Role and Play"):

1. Imaginisation - the player fills in the information gaps using his/her own imagination.

2. Diagrammatisation - the symbol is interpreted systematically and logically (no information is added by imagining) .

3. Allegorisation - The obvious is ignored and the symbol is interpreted as a metaphor.

These categories are perhaps applicable to other types of symbolic communications (other than textual) and might be of some help.

Article: Games as Playground and Space

This article discusses game environments from both a narratological perspective and - perhaps more interesting for us - a gameplay perspective. Among many things, the author discusses environmental affordances, and coins the expression 'Game Ego' to explain the part of the player that acts within the game on a motoric/sensory level. Throughout the whole article, the author exemplifies with games and draw comparisons to childs play, movies and novels.

http://portal.acm.org/citation.cfm?id=1234341.1234353&coll=Portal&dl=GUIDE&CFID=19883058&CFTOKEN=75684218

Digesis by Communication

Excerpt from the book "Beyond Role and Play" concerning diegesis construction. Article by Mika Loponen & Markus Montala:

"... every player's diegesis is different by definition (since it is constructed in the players mind). In the best possible situation the diegeses are equifinal. Two interpretations are equifinal when they are similar enough to cause indistinguishable consequences. Equifinality is easiest to achieve if all players have a good understanding of the genre, style and world of the game "

The opposite of equifinality is called arbitration - this occurs in all forms of communication when people have different interpretations of the meaning of symbols (language).

The authors go on saying that: "...it can be said that the more the diegeses are constructed through icons and indices the more (life)-like a game is."

To use an example from Mirror's Edge, let's look at Runners Vison (an object colored red in the environment) and how it can be interpreted by players' diegetically in at least 3 different ways:

Indexical interpretation: Faith actually sees some objects as red (allthough they are not), due to some special mental state or through some exotic technology.

Symbolic interpretation: Faith picks out important objects in the environment through her own perception and the red color is just an arbitrary way of showing this.

(Iconic interpretation: Someone has actually been nice enough to paint stuff red to help Faith.)

After discussing indexical vs symbolic interpretations of signs the authors state that:
"The more a game is based on indices, the less chance there is for misinterpretations, and the less need for non-diegetic communication."

However, the authors identifies at leas two mayor problems with a fully indexical game world.
1. The player need to have an extended knowledge of objects in the game world, its back story and so on, in order to fully interpret indexical symbols.

Example: In an FPS with no HUD a player has to know that an assault rifle holds 30 bullets and a revolver holds 6 in order to best time his/her reloads.

2. Every player will still have a unique interpretation of each symbol, and the designer can never guarantee that all player make the right (or even the same) interpretation.

-I found this way of using semiotic concepts to describe game world symbols very interesting and useful to us.

Game Audio Connected to Diegesis

I stumbled upon an interesting Gamasutra article proposing a framework for game audio. The IEZA framework splits the audio design space into a) a diegetic/non-diegetic dimension and b) an activity/setting dimension, resulting in the sound categories below (quoted from the article):

The Interface category expresses the activity in the non-diegetic part of the game environment. In many games of today this is sound that is synced with activity in the HUD, either as a response to player activity or as a response to game activity.

The Effect category expresses the activity in the diegetic part of the game. Sound is often synced to events in the game world, either triggered by the player or by the game itself. However, activity in the diegetic part of the game can also include sound streams, such as the sound of a continuously burning fire.

The Zone category expresses the setting (for example the geographical or topological setting) of the diegetic part of the game environment. In many games of today, Zone is often designed in such a way (using real time adaptation) that it reflects the consequences of game play on a game's world.

The Affect category expresses the setting (for example the emotional, social and/or cultural setting) of the non-diegetic part of the game environment. Affect is often designed in such a way (using real time adaptation) that it reflects the emotional status of the game or that it anticipates upcoming events in the game.

As the notion of diegesis feels more and more relevant for the project, this article could serve as a good base for argumentation and inspiration.

http://www.gamasutra.com/view/feature/3509/ieza_a_framework_for_game_audio.php?print=1

Ramblings: Media tracks - Games vs Film.

In this "article" summarizing the book "New Vocabularies in Film Semiotics", I came across an interesting view on film as a media. According to the author, film has five tracks that can be used to convey meaning to the viewer:

1. Moving images
2. Dialogue
3. Noise (environmental sounds)
4. Music
5. Text

All these tracks are shared with Games as a medium. However, games have at least 3 more tracks at its disposal, that I can think of:

6. Responsive behavior (the game world responds to your actions and you are forced to respond in return),
7. Haptic feedback (Rumble)
8. Non-diegetic elements (the HUD and menus).

This might be something worth looking further into IMO....

HUD Elements Classification

As the results of the objective game study grow larger to involve more games, trends of more or less conventional ways of designing traditional HUD elements layered on top of the application can be noticed. One possible classification of traditional HUD elements can be done by looking on the placement and visibility strategies. From this classification of HUD elements, three categories of elements can be found.


Static HUD elements are always visible and have a fixed placement on the screen. Here, elements such as meters indicating health level, digits indicating ammo, cross hairs and stance indicators (standing/crouching) can be found.

Pop-in-out HUD elements, in lack of a better term, have a fixed placement as well, but are only visible when considered relevant. Usually, this seem to include concrete information indicating a possible action ("Pick up weapon", "Press X to ..." etc). However, some games such as The Darkness and Legendary uses this to pursue a HUD that is transparent during non-hostile game sequences, by fading out the health meter and ammo digits whenever the player is not involved in combat.

Dynamic HUD elements are only visible when considered relevant, and don't have a fixed on-screen placement. These elements seems to typically convey location-bound information, such as angular direction indication of attack, direction indication to teammates, points of interest, etc.


No examples of constantly visible non-fixed HUD elements have been found so far, perhaps not very surprising considering the usability aspect of GUI elements constantly changing position.

Ramblings: Film language terminology

Here are some useful pieces of terminology used in film language and semantics dealing with the conveying of meaning:

In Film:
The Perceptual Level - This level consists purely of shapes, colors, textures and movement.

The Diegetic Level - Everything narratively denoted - Characters, objects, time, space, actions and events occurring in the "film world".

The Connotative Level - involves all the ‘symbolisms’ and metaphorical, analogical, and associative meanings that attach themselves to events and objects (or to relationships between
objects) outside of films, i.e., in culture.

The Subtext Level - involves hidden and suppressed meanings of symbols and
signifiers, framed by one of the “great narrative structures that obtain outside of films”. For example, the use of Freudian symbolism or the hero symbolizing a messiah (as in the Matrix) .

The Cinematic Level - accounts for the use of formal film/video techniques to produce particular artistic/formal/expressive results. For example, the use of a "wavy lines transition" to symbolize a character remembering a previous event.

In Semantics/Language
Five general ways of producing meaning, when using language:

We tell people how things are (Assertives)

We try to get them to do things (Directives)

We commit ourselves to doing things (Commissives)

We express our feelings and attitudes (Expressives)

We bring about changes in the world through our utterances (Declarations)

- I don't know if this is of much use, but I found, at least the film part, quite useful. Interesting to note is that films, in most parts, only work with assertiveness and rarely employ any of the other meaning conveyors - But that games do.

Book on film language

I think I have found the perfect book for us concerning film language: "Film Language: A Semiotics of the Cinema" by Christian Metz. Unfortunately, it doesn't seem to be available from any where but amazon.com. Perhaps they have it at the library.

The UI Design Space - Initial thoughts

Looking at the results of the objective game study so far, some trends can be identified. The notion of diegesis seems relevant to take into the picture.


The visual non-diegetic dimension of the UI design space seems to be highly exploited by FPS games in general; traditional HUD-elements layered on top of the game in various forms, menus used for different purposes. Blur filters, red glow when the player is damaged and fading the screen when dying seem to fall into a 'grey area' - is it always what the player character really experience? The visual diegetic dimension is less exploited - a few games use diegetic information elements, mostly in the form of highlighting interactive objects. Environmental effects seem to be rare. Possibly, this will change as more games are studied.

In the auditory dimension, diegetic UI elements mainly seem to exist in the form of heart beats and pulsating sounds to indicate a critical health level. Non-diegetic audio used to convey information mostly exists in the form of dynamic music indicating battle, used by a few of the games studied so far.

Controller rumble is more or less exclusively used in a diegetic way; vibrations when a weapon is fired, when taking damage, etc.


In the search for unconventional, innovative ways to get rid of traditional, immersion-breaking HUD elements, there seems to be a lot of unexplored ground in the diegetic visual dimension as expected, as well as both the diegetic and non-diegetic haptic and auditory dimensions.

Cinematography

Last Friday I contacted a professor of Cinematography at Gothenburg University, asking for help in coming up with useful books or articles on "Film language" (how information is conveyed to the viewer).

Still have not heard back from him though...... Perhaps he is occupied with the ongoing film festival.

Week Summary: 19/1 - 23/1

Meetings have been held with both Anna and Staffan. The monday was spent in Stockholm with Anna, for some general discussion and introduction to the project. During our tutoring meeting two days later, Staffan gave us tips about authors and terms worth checking out, among other things.

Our literature study so far has helped us expand our terminology, and we've found quite a few really interesting articles (see previous posts!) related to the objective of our thesis.

The objective game study has been started; so far we've analyzed The Darkness, Clive Barker's Jericho and Hour of Victory. We can already see that the results of this study will ease the mapping of the different information channels used by games today.

As for the study of other art forms, Erik have ordered "Understanding Comics" by Scott McCloud which will most likely serve as a superb source of inspiration. We are still looking for literature on how information is conveyed in movies.

The Auditory Dimension, Part II

It seems like most written material covering a deeper exploration of audio as an information channel comes from the 'games for visually impaired' community. The authors of this Gamasutra article reflects on game audio by looking at the results of developing a blind-accessible first-person 3D game. Might spur some ideas for us.

http://www.gamasutra.com/view/feature/3493/playing_by_ear_using_audio_to_.php

The Auditory Dimension of Games

The article "Audio Games: New perspectives on game audio" provides some potentially usable terminology for the auditory dimension of the game design space. The article discusses two different design strategies; a) using auditory icons and earcons, or b) dividing human listening into casual listening, semantic listening and reduced listening. These strategies are combined into a so-called "sonic triangular vocabulary", a format common to the visual triangular vocabulary used by Scott McCloud (author of "Understanding Comics").

When it comes to the auditory dimension of interfaces, conventions seem to exist only to a very small extent. In order to explore the possibilities of using audio as a carrier of information, I think that this sonic triangular vocabulary - and its comparability with McClouds visual counterpart - might be useful. The link to the article follows below.

http://portal.acm.org/citation.cfm?id=1067343.1067361&coll=Portal&dl=GUIDE&CFID=18462546&CFTOKEN=50433615

Ramblings: Multi sensory displays.

The article about Multi Sensory Displays (MSD) previously posted has a nice way of dividing MSD's into three categories.

1. Conflicting: The information that reaches the player through the different channels is confusing to the player.
Example: The player see a muzzle flash from his gun, but at the same time the the gun makes a clicking noise indicating that it has run out of ammo - What conclusion shall the player make in this case?

2. Redundant: Two separate information channels give the same information, increasing player confidence, performance and immersion.
Example: When firing his gun the player see a muzzle flash, hears a bang and feels the controller vibrate.

3. Complementary: The different information channels give complementary information - Resulting in a richer experience.
Real world example: By touching an object in the real world you will get a lot of information that complement your visual perception of the object - Such as surface texture, temperature, weight and so on.

I think we should focus on how to provide complementary MS feedback to players and not just redundant feedback (as is often the case with games).

Ramblings: Diegesis in Movies and Games.

The Concept of Diegesis (described in a previous post) is something that manifests in similar manners in both movies and games (at least if we consider action/FPS games). Both media rely (almost) entirely on visuals and audio to convey information to the viewer/player.

Therefor games and movies share many non-diegetic elements that do not, to any real extent, brakes immersion or engrossment. These elements would be things like the musical score - while being a strictly non-diegetic element, a musical score often adds to immersion and engrossment rather than diminishing it. The same might be said of sub-titles - giving important information to the viewer/player that adds to his understanding and appreciation of the experience.

However - consider someone hitting the menu button on the DVD remote during a crucial scene in a movie, or a player opening an inventory screen during an epic fight with a monster. These are non-diegetic elements that really breaks the immersion for us.

I believe that in the thesis we should really look at the differences between the non-diegetic instances that breake immersion and the ones that does not. And also to consider the usability aspect of diegetic and non-diegetic HUD solutions. Navigating in Mirror’s Edge without the little cross-hair dot would probably be a lot harder, although the dot itself is a non-diegetic object.

Article on Rumble

I found an article/column on Gamasutra about how rumble can be used for conveying information to the player to a higher extent than giving simple feedback for shooting, being shot, etc. The author mentions Ocarina of Time, Wii Sports and Rez as games that use vibrations on a higher extent than low-level tactile feedback.

http://www.gamasutra.com/view/feature/3652/persuasive_games_texture.php

Objective Game Study: The List

The selection of games to study is made using Metacritic. Of all Xbox360 FPS games, the game scoring lowest is Hour of Victory, with a score of 37. Bioshock and The Orange Box rank highest with a score of 96. The interval ranging from the lowest to the highest scoring game is divided into ten sub-intervals, and from each interval one game is initially selected as object for studying.

37-43: Hour of Victory [37]
44-49: Legendary [49]
50-55: Conflict: Denied Ops [52]
56-61: (Alone in the Dark [58])
62-67: Clive Barker's Jericho [63]
68-73: Call of Juarez [71]
74-79: Mirror's Edge [79]
80-85: The Darkness [82]
86-91: Left 4 Dead [90]
92-96: Bioshock [96]

Diegesis

This article on Gamasutra discuss the term Diegesis and how it relates to games. If nothing else the Diegesis concept can be usefull as part of the terminology used in the thesis.

Simply put most sound effects that you hear in a game would be considered diegetic - since the sounds would be perceived by any game character as well as the player (i.e exists inside the game world). Most game music however would be considered non-diegetic since it exists external to the game world, and would not be audible for a character in the game.

Most classic hud elements and almost all menus are therefor to be considered non-diegetic elements.

Engagement - Engrossment - Total Immersion

During our tutoring meeting this Tuesday, Staffan mentioned engrossment as a term worth checking out.

The following article is based on a user study of gamers, in which immersion is mentioned as a three point scale where the lowest level is 'engagement', followed by 'engrossment', and the highest level is 'total immersion'.

http://portal.acm.org/citation.cfm?id=985921.986048&coll=Portal&dl=GUIDE&CFID=19056125&CFTOKEN=32395460

Work Progress

Today we start our objective study of the "design space" when it comes to HUD solutions and player-feedback paradigms in console FPS games. The first game up for analysis will be The Darkness - We'll both be playing the demo available from XboxLive this evening and then compare our findings tomorrow. Later we'll probably be playing one game each every night in order to be more effective (our current game list has 42 titles on it!).

Also, today, I placed an order for the book Understanding Comics (McCloud), this book will hopefully spark some new ideas about how information can be conveyed through a medium more limited than games are.

- Erik out

Article: Flow in Video Games

An article on flow in games, and games and player interaction as information systems. It covers information processing on a cognitive level, and connects flow to information systems, which makes it seem highly relevant.

http://portal.acm.org/citation.cfm?id=1371216.1371223&coll=Portal&dl=GUIDE&CFID=18428489&CFTOKEN=57378857

Article: Invisible Walls

An article about maintaining immersion in games and problems with it, touching on the subject of removing HUD elements among many things. Might not be credible enough for referencing, but worth reading.

http://gamecareerguide.com/features/593/invisible_.php?page=1

Way finding in games

Here is an article discussing player navigational aids. Something that feels relevant, since most modern FPS include some sort of navigation or way finding function as a part of their HUD.

http://portal.acm.org/citation.cfm?id=1367956.1367962&coll=GUIDE&dl=ACM&CFID=18398977&CFTOKEN=12539952

Time in games

This article looks at a model for understanding how time can be understood in a game context. This article might be relevan when dealing with save/load functions in a game menu. If nothing else it provides a vocabulary for discussing time in game contexts.

http://portal.acm.org/citation.cfm?id=1231894.1231902&coll=GUIDE&dl=ACM&CFID=18398977&CFTOKEN=12539952

Problems with embodiment

The following article deals with problems concerning player embodiment in first person game worlds and might be of some relevance, especially if we are considering games like Mirror's Edge, where embodiment is a key factor for increasing player immersion.

http://portal.acm.org/citation.cfm?id=1234341.1234359&coll=GUIDE&dl=ACM&CFID=18398977&CFTOKEN=12539952

Multi-sensory game interfaces

Here is an interesting article about multi-sensory game interfaces.

http://portal.acm.org/citation.cfm?id=1378337.1378341&coll=GUIDE&dl=ACM&CFID=18398977&CFTOKEN=12539952

It seems very relevant to us since it deals with immersion and player performance in Quake 3 Arena comparing different combinations of multi sensory game feedback.

How FPS gamers experience the game situation

Article covering a user study of how FPS gamers experience the game situation. Chapter 4.1.1 "Interfaces" covers players' opinions on both HUDs and menus.

http://portal.acm.org/citation.cfm?id=1146816.1146827&coll=GUIDE&dl=ACM&CFID=18398977&CFTOKEN=12539952

Peripheral information - Weiser's article

Mark Weiser's article about calm technology and the use of peripheral information:

http://www.johnseelybrown.com/calmtech.pdf

Signifiers and Affordance

In the article "Signifiers and Affordance" (interactions dec 08) Norman talks about the importance of including signifiers in designs. Signifiers are things like an empty train platform - signifying that the train has just left and a flag, showing wind strength and direction. Non of these things are specifically designed with these signifying properties in mind but are still very usefull to us in every-day life.

This idea of "unintentional" signifiers is something that might be very usefull in a HUDless game environment and something I think should be emazised in our thesis.

More interesting articles

Here are a few more interesting articles:

This one is about interactions in the game FABLE

This one is about something called Play-Centric-Design

Master's Thesis: Game Environments as Psycho-Symbolic Allegory

A thesis on the psycho-symbolic potential of game terrain:

http://www.gamecareerguide.com/thesis/masinter.pdf

Book list

A selection of interesting books from: http://wordsonplay.wordpress.com/2008/07/30/the-secret-books-of-game-design/

1. "Understanding Comix" by Scott McCloud

2. "Flow: The Psychology of Optimal Experience” and/or "Beyond Boredom and Anxiety" by Mihaly Csikszentmihalyi

3. “The Design of Everyday Things” by Norman

Upcoming HUD-free game: Afro Samurai

Upcoming game Afro Samurai will be "entirely HUD-free", enforcing "a few elegant touches like this - enemies' health showing in how bloody their clothes are", according to Eurogamer:

http://www.eurogamer.net/articles/afro-samurai-hands-on

Gamasutra article: Off With Their HUDs!

An article published on Gamasutra trying to answer the question "How do you convey necessary information to the player without utilizing a traditional HUD?".

http://www.gamasutra.com/view/feature/2538/off_with_their_huds_rethinking_.php

Interesting articles

The following articles from Interactions Magazine nov/dec 08 might be worth checking out.

http://portal.acm.org/citation.cfm?doid=1409040.1409042

http://portal.acm.org/citation.cfm?id=1409040.1409044&coll=GUIDE&dl=GUIDE&idx=J373&part=magazine&WantType=Magazines&title=interactions&CFID=18265550&CFTOKEN=59223937

List of HUDless games

The following is a link to a list of HUDless or near HUDless games that might be worth checking out in detail.

http://www.mobygames.com/game-group/hudless-games/offset,0/so,1d/

The Blog is UP!

The Blog for our masters thesis in interaction design is now up and running!

Stay tuned!