This project was the MSc dissertation in HCI at UCL. The research addressed people’s decision-making process when looking for iTV content, by combining affective and cognitive search mechanisms. The prototyped findings were presented and published at EVA London and the first ACM TVX2014 conference. After that, Imperica magazine reviewed the project.
This dissertation was lead under the supervision of DR. Paul Marshall at UCLIC for the duration of this dissertation project.
Interactive TV (iTV) is television combined with the internet. ITV adds a layer of ‘dialogue’ between broadcasted TV and viewers, actively engaging them in making more informed decisions. One of the challenges ITV faces is finding content. Current search solutions include keyword search, but fail to address affective user interaction as a possible way of finding content.
This dissertation explored the notion that emotion and cognition combined, could support the users to make an informed decision and enrich the browsing experience. This hypothesis was based on research suggesting that “People make decisions based on reason and emotion combined” (Bechara, Damasio, 2005).
The literature review was focused on current search engine mechanisms in most European ITV interfaces. Looking at non-keyword-based search mechanisms and discussed the use of personalised recommendations as a search outcome.
Due to the unique nature of this topic, I followed a Research through Design (RtD) approach throughout my thesis, as this approach is helpful when designing for innovation. I employed an ethnographic method called 'Cultural Probes' to determine user behaviour and the lifestyle of seven participants. The probes intended to evoke the users’ thoughts and imagination so that I can understand better the participants’ culture about iTV browsing.
I then combined RtD with a 'Collaborative-Design Workshop'. A design process, where participants and designers co-operate, sketching design solutions together. This type of study was both research and design-led, aligning well with RtD approach. A pilot study was designed before the main, to test and inform the latter.
To analyse the data gathered, I followed a thematic data analysis. This bottom-up qualitative method helped to surface three overarching themes.
People’s lifestyle is etched by mood directing our decisions, shaping behaviour habits. However, during the current state of iTV technology, there is no way to search by mood despite the fact that people's transient moods determine their decisions. It is then necessary to explore designing a search for iTV content supporting reason and emotion combined, to aid the users’ decision process. People are often in lack of words, nonetheless aware of what are looking for, we should also be able to browse for ITV wordlessly.
The ‘cultural probes’ study revealed that participants preferred to search iTV content using a blend of voice, body, gesture and image recognition as another intuitive wayfinding.
Two types of users were identified. Mood drives type of users who might have no linguistic example to search by keyword. If one does not know how they feel, they will describe their feeling with various items dragged in the empty canvas. The recommended result then appears in real time on the third stage of the suggested process. Another significant finding was the concept of genre wheel as a component with content changing as the user shifts knobs and sliders.
User preferred technologies include a blend of voice, body, gesture and image recognition. Html5 mood tagging, was then a proposed tag method to help record the user’s affective state, as this is felt before, during and after watching a certain video. A three-stage structure was preferred, when people know how they feel, they may type it in the search field. When people do not understand how they feel, they may describe it with colours dragged in the canvas, before the recommended result appears in real time.
Participants preferred to search both by mood and reason combined either orally or not. The resulting search engine should then support, both worded and tacit users’ browsing decisions.
They also expressed an interest in playful browsing and self-awareness tools. The preferred input technologies included voice, body, gesture and image recognition. The new prototype of ITV interface yields personalised real-time recommendations based on a record of mood tagged video content.
As such, the focus was leveraged from designing an affective mood board to engaging the user into a clear, memorable and physically immersive search experience.