Encoding Emotion

This post is in response to Brave and Nass's question "How does emotion play out in computer-mediated communication (CMC)?" in Chapter 4 of The Human-Computer Interaction Handbook...

Emotion certainly plays a significant role in how we communicate with one another by providing a "social context" that seems to define the ways we communicate with one another. A smile on your face indicates to me you are feeling positive and are therefore likely more approachable and receptive to conversation than an individual with a grimace. It seems strange that computers have remained rather removed from needing to understand and convey their user's emotional experience. In many ways, computers have mostly been viewed as a cognitive tool but since computers are now being more and more used as mediums for creativity, connectivity, and interaction among people, we need to consider ways emotion can be encoded into the experience.

Emoticons became necessary because the types of interaction made possible via the Internet could not alone provide the context necessary for the communication. There are still, however, many emotions that cannot or effectively be conveyed through emoticons alone (e.g. sarcasm). Avatars in 3D games have employed emoticon type gestures that allow their character to exhibit behaviors that demonstrate one's mood (e.g. World of Warcraft characters can cheer or dance to celebrate a significant event) but these too often fall short. Emoticons of this type are low-bandwidth emotion conveyers. What's missing is the continuous and high-bandwidth emotional expression one can perceive from direct interaction with one another. Video communication (e.g. Skype) is highly effective, but video communication cannot as easily be shared, transmitted, or processed as text.

An important realization I've had is that emotions are experienced; they are not static. Emoticons are attempting to use another mode or dimension of textual interaction to efficiently "package" emotion into the textual experience--that is difficult to do and which is why they still fall short. Perhaps another mode that could be adopted for conveying emotion textually could be to standardize colors to sentences that indicate tonality, pitch, or cadence (i.e. much like music notes written on a sheet of music). While not ideal aesthetically, this capture would allow emotion to be captured in the communication.

I will soon start a new job at a small company. One important requirement levied by the hiring company was for attaining highly effective communication skills across many mediums and modes given the fact the team is spread around the world. Ideally I should be sure to spend time with these individuals in person to help define a "social context" that will help me "understand them". This provides me the means to know how to communicate with them effectively. We will also set up a "portal" that is a 24 hours a day, seven days a week television screen with a webcam linked directly to other portals around the world. The idea is that the portal will provide for higher-bandwidth communication than email can currently provide. The portal will allow the us as users to feel connected to the other working spaces rather than disconnected and reunited for minutes at a time with video chat like Skype.