Towards guidelines

Expanding upon the "Determining the UI strategy" post of last week, we would like to rethink the Spatial Interaction category and add a Navigation Significance category.

As graphical hardware constraints gradually disappear, 3D game spaces grow more detailed. As 3D game spaces grow more detailed, the amount of affordances that can be deduced from them grow larger. However, the amount of deduced affordances is always larger than the amount of affordances actually available to the player (in Mirror's Edge, only some doors can be opened; in Bad Company, only some parts of the environments can actually be destroyed). In these cases (level 2), the UI designers should aim to amplify the affordances actually available to the player, and suppress the affordances not available yet deducable from the environment. If full diegesis is not a design goal, UI designers can work with spatial information such as the runner vision in Mirror's Edge or shiny shaders on pickupable objects - if diegesis is important, signifiers might be a better tool such as smoke coming from barrels that will explode if the player shoots at them.

The significance of navigation within the geometry affects the importance of spatial UI elements. In games in which navigation is of importance (level 2 and 3), it is also important to maintain the player's sense of direction within the game environments. The importance of diegesis affects the design approach to this; if diegesis is important, the use of signifiers is appropriate (example: red smoke in Bad Company to indicate rally points), if diegesis is of less importance UI designers could also experiment with spatial information (example: the red beacons in Mirror's Edge time trial mode to indicate checkpoints).

New level-sketch

Here is a new sketch of the level model with a few more layers added. The avatar is placed in between the spatial and internal layer.

Determining the UI strategy

For quite some time, we've struggled to find a model that we can use to separate different FPS games from each other and see what game characteristics that affects the UI strategy suitable for the game - a ground for UI design guidelines. In this post, I will describe one approach to this.

There are at least five factors that in some way have a direct impact on the UI of a FPS; Level of Competition, Fiction Persuasiveness, Spatial Interaction, Rules Transparency, and Level of Vertigo.


The level of competition of the game determines whether an overlay 2D HUD might be a reasonable strategy or not. In games based heavily on narrative (level 1), designers might wanna go for an approach that pursues something that draws close to the notion of immersion, by working with diegesis and avoiding a traditional HUD breaking the illusion. However, the more competitive elements the game experience builds on, the importance of being able to make strategic gameplay decisions grows for the players; in pure multiplayer games (level 3), it is also important do make these decisions fast. Hence, ambiguous information should be avoided in these situations, which is most easily achieved with an overlay HUD.


The fictional persuasiveness of the game determines how important the notion of diegesis is to the game UI. In games with fiction that attempts to deliver a game world nearly identical to the real world during some time in history (level 3), slightest non-diegetic element might break player immersion. In games that delivers plausible worlds that are quite different from our own (level 2), diegesis might be just as important, but can be allowed by using artifacts and entities that builds on the game fiction (sci-fi, post-apocalyptic, etc). If a game does not try to hide the fact that it is an artifical world (level 1), the level of diegesis might not be an issue at all.


We think that when the 3D geometry affects gameplay, it also affects the UI regarding interface elements with spatial qualities. If the geometrical aspect of level design has no direct impact on gameplay (level 1), there is no or little need for UI elements presented spatially in the 3D space. However, as the importance of the geometry grows for the player (level 2, 3), the UI should support this by providing short-term navigational guidance and strengthening environmental affordances. If full diegesis is not a design goal, UI designers can work with spatial information such as the runner vision or checkpoint beacons in Mirror's Edge - if diegesis is important, signifiers might be a better tool.


The rules transparency of the game impact on how affordances is conveyed to the player, and the diegesis of the learning/tutorial approach of the game. If the game does not make the slightest attempt to hide the fact that it is governed by a rule framework (level 1), affordances can be conveyed by fairly straight-forward approaches such as pure textual feedback ("Press x to pick up weapon.") throughout the whole game. If this strategy is not desirable (level 2), designers should make an effort to restrict non-diegetic affordance references to a tutorial part of the game, and convey the rules in a somewhat more diegetic way ("You must find the fire escape ladder to gain access to the building" instead of "Find the exit to proceed to next level").


Finally, the level of vertigo determines how the game should make up for the fact that the only links to the perception of your in-game avatar are graphics, audio and rumble. If the gameplay is sensory-poor (level 1), this link is of less importance. However, as the gameplay draws on sensory-rich elements such as height or speed (level 2), the player needs to be linked closer to his/hers game-ego. This can be achieved by exaggerating (meta-representations, according to our previous terminology) and fortifying these senses visually, sound wise, or by rumble; for example blurring the edges of the screen (used in racing games, and Mirror's Edge) to indicate speed, or exaggerating traffic noises when jumping between rooftops.

Conceptual view of the UI

I'v been playing around with a conceptual sketch of the FPS UI and it's components as consisting of several layers (in the above image these are the overlay(er), the avatar-layer and the world-layer). I thought this could be a nice way to describe how we view the UI in an upcoming presentation/report.

The design space revisited, once again

Inspired partly by the work of Jesper Juul mentioned in the previous post, we have once again taken a look on how the design space can be modified even further. In the last proposal, we laid forth a UI design space with two dimensions; the spatiality in relation to the 3D game space, and the existential qualities within the fictional game world. Jesper Juul speaks of the game space as a meeting between fiction and rules. As mentioned before, there are cases (from an UI point of view) when fiction and rules do not overlap but are still in the game space, resulting in information outside the fiction part of the UI but within the rule space; for example runner vision in ME. In a similar way, there are cases when fictional and rule-based entities is presented outside the (3D) game space, such as overlay GPS systems or the GTA4 cell phone.

An alternative take on the UI design space would be to describe it as the collision of the 3D game space, the fictional space, and the rule space. The old design space expanded with a rule dimension, sort of.


What's good about this? Traditional HUD elements today resides only in the rule space, outside the 3D game space and the fiction space. To pursue an immersive/engrossing experience UI designers should focus on pushing the UI closer to the core of the design space.

Another advantage is that the model can be exploited from different directions depending on the nature of the game. A competition based experience is a lot about rules, a mimicry-heavy game experience is a lot about the fiction, and pursuing vertigo is a lot about feeling that you are one with the 3D game space, which means that depending on a games focus, the model can be exploited from different directions. For example, if a game is heavily competitive an overlay HUD might actually be preferable while signifiers might be regarded as vague information, while in a heavily mimicry based situation spatial and non-diegetic representations might break the immersive experience. Exploitation of the earlier design space differed a lot depending of the nature of the game.

This UI design space also illustrates the role of the game UI designer as we see it, being involved with both game designers (rule space), level designers (game space), and fiction/art director (fiction space).

The design space revisited

The design space we settled for about a month ago relies heavily on the notion of diegesis. One problem with the notion of diegesis is that there are some potential gray areas between what is diegetic and what is non-diegetic. Take for instance the blood stains appearing when taking damage in Killzone 2; having your face splattered with blood seems quite realistic given that half of the characters' body mass consists of blood (which is usually a convention in action games), but the visual effect would most certainly not be distinctive blood stains on your eye retinas. Or the Mirror's Edge runner vision; objects in the game world with the potential of aiding your traverse of the terrain the turns red, yet is this what Faith really sees?

This might motivate an effort to look at an alternative way of mapping out a design space in which diegesis is not a dimension itself, but rather an area within the design space.


If the spatial dimension (is the UI element presented in the game space or not?) of our design space was to be kept, and the diegetic dimension is replaced with the existential qualities of the UI element within the game world (does the representation exist in the fictional game world or not?), the design space would be represented by the model pictured above.

Information representations that is presented spatially in the game space and exist as entities in the game world, are diegetic representations. Take for instance the Doom 3 data terminals; they exist in the game world, they are also presented in the game space, and are hence very much diegetic. The same can be said for the Dead Space menus; the tech setting of the game (sci-fi) allows projecting information on virtual surfaces, hence the menus are not only a part of the game space but also of the fictional game world itself.

Representations can exist in the game world, but aren't necessarily visualized spatially for the player; these are meta representations. The Killzone 2 example above is an example of this; characters bleed when they get hurt, however the blood is visualized as stains on an overlay camera lense rather than in a realistic fashion. Other examples of meta representations would be the GTA IV cell phone (a 2D overlay representation of the character's cell phone), heart-beat audio indicating critical health level (you normally wouldn't hear your own heart beats, yet they exist and can be heard by others), ambient sounds such as thunder or breezing wind (usually just zone/area specific rather than actual spatial 3D audio sources), or the old Doom head indicating character health level and status (an overlay representation of the character's head, which is a representation of the status of the player).

Non-diegetic representations are disconnected to both the game space and the game world. This would be traditional overlay HUD elements such as health bars (there is no health bar visible for the in-game character), ammo digits representing how many bullets that are left in the clip, dynamic music, sub-titles, etc.

Finally, UI elements could be presented in the game 3D space without being an entity of the actual game world; spatial representations. Here, we find examples like the character outlines of Left 4 Dead (none of the game characters actually see the silhouettes of their teammates through walls, yet they are visible for the player), the highlighting of interactive/pickupable objects, Faith's runner vision in Mirror's Edge, Dead Space path guidance, or check point beacons in Mirror's Edge.


So what's the point of tearing down the design space just to re-assemble it in a slightly different form? Given our results and reflections so far, it seems like spatial representations and meta representations can be used to engross the player even deeper than what would be achieved with a completely diegetic UI! Spatial representations can be exploited to strengthen the player's in-game perception regarding external information - UI solutions such as runner vision in Mirror's Edge or the teammate silhouettes in Left 4 Dead makes up for the fact that players tend to loose their sense of direction when moving about in a virtual 3D space. Similarly, meta representations can be exploited to strengthen the player's perception regarding internal information; blood stains and heart beats, or even the old Doom head, helps players make up for the fact that the physical game interface (TV, speakers, controller) does not allow players to connect with their in-game avatar using the broad set of senses that is available to us in the real world.

The results of the interviews - early reflections

During the last two weeks we have conducted five interviews, trying to prove our hypothesis going something like this: a player engrossment/immersion model can be created using a taxonomy of games (competition, mimicry, vertigo) and frames (person/player/character), and this is reflected in the game UI and the diegesis-spatiality of conveyed information.


When talking to our interviewees, we tried to elicit what they consider to be an engrossed/immersed game experience, and what elements that breaks this experience. According to the model above, the results of an early analysis of the comments and states made by each person interviewed can be visualized in the following way:

Dimitry prefers authentic and realistic war themed FPS games, and although he plays competitive multiplayer games to a high extent (stating Red Orchestra as one example) he seems to be playing for a mimicry experience rather than the competitive aspect. He does not want to be reminded that it is "just a game", hence he moves between the person and the character mimicry frames (taking his real-world knowledge into the game-world). Dimitry prefers diegetic UI solutions and no HUD at all, although he do think that the HUD-heaviness of games such as Team Fortress 2 is ok as the game does not aim for a realistic experience.

Guy enjoys both pure gameplay and more immersed experiences; according to him, he plays story-heavy games like Bioshock for more of an experience, while (quote) ”TF2 [and other multiplayer games] are so absurd.. you’re playing them as a game rather than an experience”. Hence, we would say that in a mimicry situation Guy engrosses himself on a character frame level, while playing more competitive games in the player frame. Guy is one of the two interviewees that have played Mirror's Edge, in which he seeks an experience of being embodied as Faith in the game-world rather than enjoying the story told. HUD-wise, Guy feels that it is good to have more information in a competitive multiplayer situation, but finds it desirable to find alternative solutions to a traditional non-diegetic overlay HUD in single-player experiences.

Robert is as diverse as Guy, describing himself as a player both enjoying competitive multiplayer and story-based single player games. In single-player games, he is in it for the story and the game universe, while he plays competitive multi-player games for competing and feeling better than others. Robert has played Mirror's Edge as well, naming one of the reasons he enjoyed the game so much was because (quote) "it was just you and the game [no hud] and not a lot of other stuff you had to look at”. As with Guy, Robert playes competitive games by moving between the person and the player frame, while single player mimicry experiences is played largely in the character frame. Robert prefers games that strive for a fully transparent HUD because (quote) “the HUD does not help immerse you into the game and sometimes it even disturbs the experience”, however he also says that (quote) "in some games you really need to know stuff like how much ammo you got".

Mattias describes himself as a "classic gamer", having played a lot of Counterstrike in his past but have now moved on to more single player games. To him, an engrossing multiplayer experience is when he is really focused on what happens at the moment and what his teammates and rivals is doing, while he enjoys single-player games for the unexpected turns of the narrative and puzzle solving. This could be interpreted as that Mattias plays competitive games in the player frame, while in story-heavy games he moves between the character frame and the player frame. Mattias don't believe that the game experience is very dependent on the HUD strategy chosen by the developers - as he is a "classic gamer", he prefers explicit solutions such as overlay health bars as it helps him make in-game decisions, however he admits (when asked about it) that fully transparent HUDs is more realistic and is probably suited for games aiming for a high level of realism.

Henke mostly play single-player games, though not for experiencing the story but more as playgrounds. He really enjoyed Halflife (1), and mentions the ability to play around with the physics as one reason. He does mention Bioshock as a game he enjoyed because of the thrilling game-world, however he found the non-varied nature of the gameplay to be irritating. This points at Henke playing mimicry games in the player frame. As Mattias, Henke does not believe that the HUD play a large role in the engrossment/immersion of a game, stating that (quote) "I can be sucked into a game no matter if it has a good HUD or not". When asked, he believes that more information makes sense in multiplayer games, hence a more HUD-heavy UI strategy might be of importance in those compared to single-player experiences.

Altogether we're quite content with this non-quantative interview user study, as we've gained some knowledge. Our immersion/engrossment model seems appropriate, however it seems that it can be modified into the following:


In competitive game situations, players move between their person frame (personal goals, striving towards a sense of achievement) and the player frame (exploiting the game rules). In a mimicry game situation players seem to move primarily between the person frame and the character frame - carrying over real-world knowledge into the game-world. The same goes for vertigo-focused games. A traditional HUD seems to be most important when acting in the player frame, hence a 2D overlay UI strategy might be suitable for competitive games, while alternative strategies might be desirable in mimicry and vertigo situations.

Frames, Game Taxonomy and Diegesis

Caillois taxonomy of game pleasures - what pleases us about games and play - seems to provide helpful guidance in designing UIs for different types of games. This model can be connected to frames and diegesis, according to the sketched figure below.


First of all, the player frame is the frame we adapt to understand and make use of the rules of the game - how we reason as players of the game. Since rules are a vital part of every computer or console game (or indeed any other game as well), playing a game will always involve adapting the player frame in one way or another. However, depending on what elements (competition/mimicry/vertigo) the game uses to make us feel engaged and engrossed, playing a game will also involve adapting the person frame (reasoning as a person in the real world) and/or the character frame (reasoning as character in the game world). In other words, playing a game involves switching between frames, which is a rather strong argument against the possibility of achieving full immersion.

The competition element of games involves switching back and forth between the person frame (sense of achievement, competing against oneself or other individuals) and the player frame (making use of and exploiting the game rules to achieve ones goals). Here, rather than focusing on an UI that immerses the player into the game world, the UI should support the competitive spirit and make sure that the game rules are clear to the player.

The mimicry element of games instead involves the player frame (adapting the game rules in order to take part of the game world) and the character frame (being immersed into the game world). Here, the diegesis of the UI should be an important tool to push the player into the character frame rather than being reminded by distracting non-diegetic UI elements that the game is "just a game".

The vertigo element of games involves the person frame on a perceptual level, and the player frame on a game-rule level. The goal of pure vertigo games is to achieve a sensory-rich experience (skiing, mountain-climbing, etc.). In FPS games, this can be reflected in platforming, the chaos that follows intense battle and explosions, scare and fright, or as in Mirror's Edge applied to character movement itself by parkour. From a vertigo point of view, the UI should try to push the player into the person frame, to try to connect the game to the player's primary framework of senses.


In most FPS games, all three elements of competition, mimicry and vertigo exist, however one element will be in more focus than the others. Hence, the UI should support the element most dominant. In Mirror's Edge, this could be the element of vertigo (the act of parkour). In Battlefield: Bad Company single player, the dominating element could instead be mimicry (taking the role as one of the squad soldiers). In the multiplayer part of Bad Company, the dominating element is competition (competing against rivaling team, competing internally by getting higher scores than teammates).

Results: Game Study

Here are some of the results from the objective game study. First let's look at how the information conveyed to the player is distributed between different categories (Think of it as informational band width):

And here is how it breaks down into different "carriers"

Internally:
Externally:
Goal oriented:
And Non-diegetic info:

Charts: The Design Space and How it is Exploited Today

One possible charting of the UI design space of FPS games is to divide it into three dimensions: the information channels (how is the information conveyed?), the diegesis of the information conveyed (is the information diegetic or non-diegetic?), and the spatial quality of the information carrier (does the information representation exist in the game 3D space or not?). If this division is made, a four-field chart of each main information channel (visual, audio, haptic) can be produced. By putting examples (from the objective and subjective game study) into the charts, we can see how the design space is exploited by games today.

VISUAL INFORMATION


Game developers today mainly tend to use non-spatial visual information carriers (2D overlay elements, image filtering) to convey information. Spatial representations seem to be used to a far less extent; here, few conventions exist, hence exploration might reveal quite innovative ideas on how to information can be conveyed.

AUDITORY INFORMATION


As far as game audio is concerned, diegetic information represented in a non-spatial way seem to be the most exploited area (allthough there's probably still plenty of the area left to explore) when it comes information more sophisticated than mere auditorial feedback on explosions and weapon fire. Games such as Left 4 Dead (teammate dialogue, characteristic special infected sounds) shows that a lot of potential also reside outside of this area.

HAPTIC INFORMATION

Similar to the auditorial dimension, developers today tend to exploit controller rumble mainly in a diegetic non-spatial way. In most cases vibrations seem to be exclusively used for giving feedback on being hit by enemy fire and/or when the player use firearms. Looking at examples from other game genres, it's clear that there is a lot of unexplored space when it comes to the haptic dimension of the design space.

Draft of heat map diagram

To gain a better understanding of the design space in FPS games we decided to make so called heat map diagrams to illustrate the frequency of different design solutions - "messages" are listed on the Y-axis and "the messengers" are listed on the X-axis. Pleas refer to this post for a description of each "messenger".

Red cell = very common
Pink cell = common
Orange cell = quite common
Green cell = uncommon
White = non existent

Here is a draft of the "internal states" heat map - Things happening to the player.
The External states - thing happening in the game worldGoal oriented information
Non diegetic information - Things separated from the game world but relevant to the player.