Post-Test Session: Storytelling

At the end of each session, the visitors were involved in a debriefing (in one of the rooms of the museum) where storytelling was encouraged to further comment, analyse, and interpret events which occurred during the test. The subjects were asked to describe their experience looking at the video recording of the test. This created a common base of discussion and knowledge, and provided concrete data to express impressions and points of view. To facilitate the storytelling, we prepared a set of questions to stimulate the discussion at the four levels of evaluation. These questions spanned over the following dimensions:

• Global experience
• Satisfaction /engagement.
• Contents.
• 
• Design concepts.

As a general outcome of the evaluation, we can say that stories mostly addressed aspects related to the emotional and socio-cultural levels. Indeed the system created a rich sensory environment that the users perceived but were not able to describe in a more structured form than stories. Stories, being concrete and immediate, do not require abstraction or introspection, so the users were free to tell and to compare their previous experiences, re-creating a context to share with the facilitators. That is why we used storytelling as an expressive means to stimulate communication between people who were not familiar with each other and to encourage them to speak about personal feelings, experiences and impressions.

We were aware that this kind of evaluation cannot provide quantitative data about satisfaction and engagement (such as that obtained using the Differential Emotions Scale, the Semantic Differential Scale or the free labelling method (Kim and Moon, 1998)). However, since the system was oriented to entertainment and leisure, we decided to collect data that could give insights into the capability of the system to intrigue and attract the users. During the debriefing we collected a corpus of about 20 stories that were mapped on user requirements and system specifications. Here is one of those stories:

When we went to Avignone, to visit the Palais des Papes, we had a local guide, a teacher of a school party who explained in detail the artistic and historical features of every room. We were interested in her explanation, but the students (children of the primary school) got suddenly tired. During the visit we passed through a room, where there was a different kind of exhibition where strange and funny animals dangled from the ceiling. Pupils were very curious to know about them, but the teacher was prepared only on the Palais des Papes, so she passed through the room without paying any attention to the animals of the exhibition.

The story contains a number of relevant elements for the design and evaluation. It highlights that visitors have heterogeneous needs: most of the time their activity is "non-goal oriented" since they can be pushed just by curiosity or pleasure, and their behaviour is not predictable. They often do not know ahead of time, or with any specificity, what future state they desire to bring about. Therefore, the situations of use can be various and idiosyncratic, leading the visitors to frequently adjust their goals and objectives during the visit.

Taking these elements into account, we can infer that the HIPS tour guide and systems like it must learn to adapt to visitor inclinations as they arise. The solution designed into the system, which proved to be particularly appropriate in this respect, focused on establishing a closer link between environment and user interactivity through the introduction of adaptive mechanisms.