A Semiotic approach to the HUD

This whole thing started with me attempting to do a Wenn diagram explaining Peirce's sign theory. For funnsies I tried to see if it all somehow could be applied to game HUDs. Turns out it can and works very well as a framework for analyzing HUD designs. The diagram brakes up into seven categories (A-G) and I will attempt to give one example from each.


A. Purely iconic signs: Signs that directly depict what they are representing. A photograph is a classic example of an iconic sign.

Example: The picture of Atlas in the bottom right corner, Note however that there also exists a indexial relation in this case. Also, the ammo icon in the bottom left directly depicts the kind of ammo equipped.

B. Purely indexical signs - These signs have a direct physical or causal connection to what they signify. Bread crumbs signify bread (Physical) and thunder signify lightning (causal). Almost all sound effects in games are indexical symbols.

Example: The sound signifying that shield-recharge has started in the Halo series.

C. Purely symbolic signs - All text and abstract signs are symbolic. These signs require that you know some "ground rules" in ordered to be interpreted (Like knowing how to read text or roman numerals).

Example: Text instructions in Killzone 2

D. Iconic/indexical signs - both directly depicts what they are symbolizing AND have a casual/ physical connection to something happening in the game.

Example: Blood stains in Killzone 2 - they directly depicts the blood that the player looses due to being wounded, the amount of blood is related to the amount of damage taken and the direction from where it gushes is related to the direction of the hit.

E. Symbolic/iconic signs - A sign that is depicting something, but symbolizing something else or something more general than the symbol suggests. Usually these kinds of signs are widely spread throughout our society. These signs are more abstracted than signs in from category A but still has some connection to their signifies unlike signs from category C.

Examples: A symbol depicting a floppy disk usually means "save" in general, not just "save to a floppy disk".

F. Symbolic/indexical - Thes signs are thing that change in some predetermined manner as an effect of changes in the game world - But still need to be explained to the player, unlike signs from D.Example: All the bars and indicators in Street Fighter 4 - what these bars represent and how they are connected to the actions of the player and the events in the game world is not apparent from the start.

G. Everything at once! - Maps are a classic example of a sign(system) that occupy all the three groups at once. The map is iconic in that it depicts the actual world (from a top down perspective), it is indexical in that the distances on the map correspond to real world distances according to the maps scale and symbolic in that it often use symbols that need to be interpreted using the maps legend.

Example: The map in the bottom left corner in GTA4 - The map is an accurate depiction of Liberty City (Iconic), it makes use of a lot of small icons to denote special locations (Symbolic) and it reacts to player actions by constantly rotating and scaling the map (indexical).

This is useful for analyzing HUD designs form a component level - I belive that for an immersive singe player experience HUD elements should be represented as either A B or D signs. For an engaging and fun multi-player experience however signs from C, E F and G can be used to grate effect.

Ramblings: Informational Content vs Carriers

In the analysis of our initial studies of how information is conveyed, it feels important to reflect on how information carriers are separated from their content - the information itself. While an information carrier itself may not have spatial qualities (exist in the 3D game space), the content can still consist of spatial information. This is the case with HUD maps, compasses and radars - the information carrier is a 2D overlay object, conveying information about the external surroundings.

In FPS games, information about internal status - non-spatial data such as health, ammo, etc. - is normally conveyed through information carriers without spatial qualities, such as 2D overlay health bars or effect filters, digits for ammo count, etc. In a similar way, it is not uncommon to provide external information through information carriers with spatial qualities, such as checkpoint beacons in Mirror's Edge, indicating pickup-able objects by highlighting the objects themselves, the character outlines when teammates are behind walls in L4D, etc.

A combination that is very interesting in my opinion is when spatial information carriers are used to convey information content about one's internal status. One of few examples of this is the use of NPC speech in L4D; besides of conveying information about the surroundings ("Boomer!", "Safehouse ahead!"), the teammates' comments can also refer to the player's health ("If you don't find first aid you're not gonna survive!"). It seems like this area is largely uncharted, yet I think this way of conveying information has great potential from an immersion point of view .

For literature on the separation between informational content and information carriers/vehicles, we might want to look into semiotics and philosophy.