All posts by admin

Publication: Audiovisual Narrative Creation and Creative Retrieval – How Searching for a Story Shapes the Story

Sabrina Sauer’s article titled ‘Audiovisual Narrative Creation and Creative Retrieval: How Searching for a Story Shapes the Story’ is published in the Journal of Science and Technology of the Arts.


Media professionals – such as news editors, image researchers, and documentary filmmakers – increasingly rely on online access to digital content within audiovisual archives to create narratives. Retrieving audiovisual sources therefore requires an in-depth knowledge of how to find sources digitally. These storytelling practices intertwine search technologies with the user’s ideas and production cultures. This paper presents qualitative research insights into how media professionals search in digital archives to create (trans)medial narratives, and uses the notion of creative retrieval to unravel the dynamics of audiovisual narrative production. Creative retrieval combines ideas about the effects of media convergence on media content, theories about serendipitous information retrieval, and studies of creativity to argue that retrieval practices of media professionals who create audiovisual narratives are governed by organizational, technological and content affordances and constraints. The paper furthermore exemplifies the first stage of an ongoing research project in which a user-centered design approach guides open source self-learning search algorithm development to support creative retrieval.

Please read the article here:

Serendipity Society at the World Humanities Conference

Sabrina Sauer will be one of five researchers presenting work on serendipity at the Serendipity Society Symposium, hosted at the World Humanities Conference in Liège on the 9th of August 2017.

Symposium descriptionSerendipity in research is associated with the now classic experience of looking for a book on a library shelf, and finding another, even more valuable, book. However, our world is increasingly technical – traditional approaches to humanities research must integrate with the tools of scientific methodology and information technology. Nonetheless, there is a continuing need for insight and creativity, even in a world of controlled experiments and algorithms, and people continue to make unexpected and unpredictable discoveries. How does serendipity, a concept born in the humanities, clarify and contextualize the current push toward ‘innovation’? How do humanities researchers today experience serendipity? What is the nature of the ‘unsought finding’? How can we ensure future opportunities for serendipity? What technologies enable valuable, yet unpredictable, discoveries? This Symposium presses upon the boundaries between disciplines to illustrate how research in all fields will—and should—continue to be a human experience above all. 

Sabrina’s paper titled “Serendipitous search practices of media researchers: Developing techniques to elicit ‘the unforeseen’” deals with the relation between serendipity, creativity and search. It presents insights into the role of serendipitous search for media researchers’ unearthing of research ideas and insights. Media researchers increasingly rely on digital access to audiovisual archived material to collect data. This stipulates that to retrieve relevant material, researchers require an in-depth understanding of digital search. Using qualitative (focus group and interview) data, this paper draws conclusions about how media researchers experience and elicit serendipitous search, as a form of craft. These conclusions aid the development of new search algorithms that embrace serendipitous search as a source of innovation.

Read more about the Serendipity Society here!

Paper presentation: Getting the Bigger Picture: An Evaluation of Media Exploratory Search and Narrative Creation

Sabrina Sauer co-presented a paper with Berber Hagedoorn at the DHBenelux conference hosted by Utrecht University  on the 4th of July. The paper, titled Getting the Bigger Picture: An Evaluation of Media Exploratory Search and Narrative Creation, is partially informed by insights gained during MediaNow’s second user panel meeting ( 24th of May 2017).

Abstract: Digital Humanities centres on questions that are raised by and answered with digital tools in the Humanities. At the same time, it interrogates the value and limitations of digital methods in Humanities’ disciplines. While it is important to understand how digital technologies can offer new venues for Humanities research, it is equally essential to understand – and therefore, being able to interpret – ‘the user side’ of Digital Humanities. Specifically, how Humanities researchers appropriate and domesticate search tools to ask and answer new questions, and apply digital methods. Previous user research in Digital Humanities concentrates on assessing, for example, how and why Digital Humanities benefits from studies into user needs and behaviour (Warwick, 2012), user requirement research, as well as participatory design research (Kemman & Kleppe, 2014).

Exploratory search is crucial for Humanities researchers who draw upon media materials in their research. Audio-visual, online and digital sources are in abundance, scattered across different platforms, and changing daily in our contemporary landscape. Supporting researchers’ explorations becomes even more important when scholars study media events. A ‘media event’ is an event with a specific narrative that gives the event its meaning, and is in contemporary societies increasingly recognized as non-planned or disruptive. Disruptive media events, such as the ‘sudden’ rise of populist politicians, terrorist attacks or environmental disasters, are shocking and unexpected, making them difficult to interpret. This leads to problems for media researchers who analyse how narratives construct different political, economic or cultural meanings around such events. Previous research argues that media events should always be viewed in relation to their wider political and sociocultural contexts. Events, as they unfold in the media, may correspond to long-term social phenomena, and the way in which such events are ‘constructed’ has particular connotations (Jiménez-Martínez, 2016). Specific actors (newscasters, governments, institutions) use media events to build narratives in line with their own political, economic or cultural purposes. Media researchers also build narratives around events; prior research underlines the importance of visualizing, constructing and storing of narratives during the information navigation to contextualize material (Akker et al., 2011; Kruijt, 2016; De Leeuw, 2012). Offering media researchers the ability to explore and create lucid narratives about media events therefore greatly supports their interpretative work.

This paper proposes to add to this body of research by presenting the insights of a cross-disciplinary user study that involves, broadly speaking, researchers studying audio-visual materials, in a co- creative design process, set to fine-tune and further develop a digital tool that supports Humanities’ research through exploratory search. This paper focuses on how researchers – in both academic as well as professional settings – use digital search technologies in their daily work practices to discover and explore digital audio-visual archival material. We focus specifically on three user groups, namely (1) Media Studies researchers, (2) Humanities researchers that use audio-visual materials as a source and (3) Media professionals. These user groups are the foreseen end users of the tool, because they create audiovisual narratives for their respective work purposes. We set-up co-creative design sessions with 74 participants (group 1: 24; group 2: 40; group 3: 10) to observe and reflect on the practices of media researchers in terms of how they interact with search tools to explore, access and retrieve digitized audio-visual material, in order to interpret, and in some cases, re-use this material in new audio-visual productions.

Methodology: In our user study, we employ a user-centred design methodology to evaluate and fine-tune the exploratory search tool DIVE+ media browser. It offers events-driven exploration of digital heritage material, where events are prominent building blocks in the creation of narrative backbones (De Boer et al., 2015) and links a variety of different media sources and collections. DIVE+ offers intuitive exploration of media events at different levels of detail. It connects media objects, subjects (“concepts”), events, and persons to aid in the formulation of research questions, and to contextualize the former into overarching narratives and timelines. Our main research question throughout the case study is how does exploratory search support media researchers in their study of how media events are constructed across different media and instilled with specific cultural or political meanings? To be able to answer this question, we study how media researchers construct navigation paths via exploratory search and – by means of user studies – evaluate the role of narratives in (1) learning and (2) research. In this process, we compare DIVE+ to other online search tools.

The user study observes media researchers as they use DIVE+ to explore media events, across 3 stages: (1) during research question formulation (2) DIVE+ use; and (3) comparative user evaluations of the DIVE+ browser, compared to other online search tools. The collected data, consisting of both qualitative – observational and focus group – data, as well as logging data gathered during user testing, provides insights about how media researchers search and explore digital audio-visual archives. We utilize a case study approach, which combines grounded theory (that fosters an understanding of how researchers interpret and create narratives) with usability methodologies, such as work task evaluations. This, first of all, allows us to draw conclusions about how search tools and digital technologies co-construct the researcher’s professional practice. Second, the data helps us probe the question how the ‘digitality’ of search and retrieval shapes the practice of media research, and, in extension of this, creative processes.

The research presented in this paper takes an interdisciplinary approach: it combines insights from Media Studies, as well as from Information Studies and Science and Technology Studies and integrates ideas about narrative creation, search practices, and overarching notions about how users and technologies co-construct meaning. Therefore the presented research does not focus on how Digital Humanities’ tools have an impact on researchers’ practices, but rather analyses how researchers make use of search tools. We subsequently (1) draw conclusions about scholarly practice and the role of search technologies for digitized audio-visual materials therein; and (2) present lessons learned on how to optimize the search tool that is used, in order to improve its performance.

Report: MediaNow Second User Panel Meeting

MediaNow’s second user panel meeting took place at the Netherlands Institute for Sound and Vision on the 24th of May 2017. Eighteen people gathered for the meeting to discuss the status of and insights produced by the project so far.  After a lively round of project update presentations by the involved researchers, members of the user panel actively participated in the morning’s MAKE, TRY and ASK sessions.

The MAKE session focused on mapping search processes of media professionals, researching specific topics for audiovisual items, projects or programs. The media professionals collaborated in small groups to draw flow charts or search scenario’s, and shared their ideas with the rest of the group afterwards.

During the TRY session, the exploratory search browser DIVE+ was tested. Guided by a number of search tasks, users tinkered with and tested the browser. Feedback was collected by means of an online questionnaire, as well as shared afterwards during the ASK session.

The researchers thank the user panel for their very active participation! Insights provided by the created search scenario’s  will help validate the team’s earlier findings about the context and practices of professional audiovisual search, while the DIVE+ testing will lead to recommendations for the further development of the search browser.

Presentation: Habit, craft and creativity: How digital search habits shape the craft of professional audiovisual storytelling

On the 29th of June 2017, Sabrina Sauer will present a paper at the European Network for Cinema and Media Studies conference, NECS Paris. The paper is titled ‘Habit, craft and creativity: How digital search habits shape the craft of professional audiovisual storytelling’.


The increased digitalization of audiovisual materials allows media professionals, such as news documentation specialists and documentary filmmakers, to increasingly make use of online access to digital archives to find sources for their audiovisual stories. This paper presents empirical insights into the digital search habits of media professionals, and questions how habitual user-technology interactions configure the craft and creative practice of digital storytelling.

Conceptually, the paper frames creative practices of media professionals by focusing on the tension between perspectives on habitual work routines and craft; between routines that are afforded by socio-technical context and ideas about individual creative agency. Here, the tension between habitual work routines and creative agency is studied empirically. By analyzing how media professionals make use of digital search technologies to retrieve archived audio-visual material for re-use in new digital stories, it becomes possible to form an empirically grounded understanding of how habits relate to craft; how habitual search strategies relate to socio-technical constraints and affordances and how, together, these shape creative processes and products.

This paper is presented in the context of an overarching research project that takes a user-centered design approach to co-create new open source search algorithms together with foreseen end users. The qualitative methods that are part of this approach, such as focus groups and semi-structures interviews, allow an in-depth understanding of digital search habits of the included media professionals. Apart from drawing conclusions about the relationship between habits, craft and creativity, the paper thus also draws methodological conclusions about how user-centered design methods can channel observations about mundane, tacit and habitual user-technology interactions into new media innovations.

Paper: Generating descriptions of entity relationships

A full paper on Generating descriptions of entity relationships by Nikos Voskarides, Edgar Meij and Maarten de Rijke was accepted for publication at the 39th European Conference on Information Retrieval. The paper will be presented in April at Aberdeen, Scotland.

Abstract: Large-scale knowledge graphs (KGs) store relationships between entities that are increasingly being used to improve the user experience in search applications. The structured nature of the data in KGs is typically not suitable to show to an end user and applications that utilize KGs therefore benefit from human-readable textual descriptions of KG relationships. We present a method that automatically generates textual descriptions of entity relationships by combining textual and KG information. Our method creates sentence templates for a particular relationship and then generates a textual description of a relationship instance by selecting the best template and filling it with appropriate entities. Experimental results show that a supervised variation of our method outperforms other variations as it best captures the semantic similarity between a relationship instance and a template, whilst providing more contextual information.

Presentation ICT.Open 2017

Sabrina Sauer will present MediaNow at the ICT.Open conference on the 21st-22nd of March 2017. The ICT.OPEN event is organised annually by the Netherlands Organisation for Scientific Research (NWO) under the auspices of ICT research Platform Netherlands (IPN). The title of the presentation is MediaNow – using a living lab method to understand media professionals’ exploratory search.


Paper: Digital search strategies of Dutch television professionals: How search savviness shapes creative work routines and audiovisual content

On April 20-21, Sabrina Sauer will present a full paper at the conference Researching Media Companies Producing Audiovisual Content, hosted by Lillehammer University College, Department of Film and Television Studies. This event is sponsored by the International Association for Media and Communication Research’s Media Production Analysis working group.

Television broadcasters increasingly rely on digitized audiovisual material for reuse in the production of new audiovisual content. Access to, and an expert understanding of how to quickly find existing material, for instance in online archives, has changed working practices of professionals creating online as well as broadcast television content. This paper focuses on how Dutch private and public media companies that produce cross-media content search for and reuse digitized archival material, and draws conclusions about how digital search technologies influence work routines and creative production processes. What do professionals perceive as affordances and constraints in their search process, and how does navigating these affordances and constraints shape their work practice?

To answer these questions, the paper presents qualitative research insights collected during focus group sessions and 20 semi-structured interviews with professionals who work for public and private companies in news, entertainment and documentary television production. The collected data focuses on work routines, specifically search practices, interactions with (online) archives and how these routines shape audiovisual content.

The analysis particularly reflects on professionals’ descriptions of how affordances and constraints such as time pressures, genre conventions, audience profiles, and technological and budgetary pressures shape the creative search and production process. Apart from clarifying how digital search technologies shape work routines and audiovisual content, the analysis suggests ways to further grasp the complex relationship between work routines, technological innovation and creativity.

The paper concludes with an overview of professionals’ search strategies to create audiovisual content, and how professionals see future developments in this area. Their future visions form the starting point – in an overarching research project – for the development of new search algorithms for a large Dutch audiovisual archive. This prompts a methodological discussion: how can academic analyses of creative media production strategies help shape new search technologies.

Talk: Living Labs – channeling serendipity and the unforeseen into ICT innovation

Sabrina Sauer gave a lunch talk at UvA’s ILPS group on the 7th of October about how innovators can channel serendipity and unforeseen user ideas into ICT innovations using the living lab approach.


Living labs are public-private-civic partnerships that facilitate user-centered ICT development in daily life environments. In living labs, prospective technology users are invited to join R&D processes. As experts of their daily life settings, they are believed to bring new, serendipitous and unforeseen ideas to the table. Yet user inclusion in innovation is regarded with some ambivalence exactly because of the uncertain outcomes. Based on research into user involvement in living labs, this talk offers six suggestions on how to successfully embrace this uncertainty and discusses opportunities of living labs and serendipity in IR.

Paper: Seeking Serendipity: A Living Lab Approach to Understanding Creative Retrieval in Broadcast Media Production

A short paper by Sabrina Sauer and Maarten de Rijke on the role of serendipity in media professionals’ search practices was presented at ACM International Conference on Research and Development in Information Retrieval (SIGIR 2016) in Pisa, Italy.


This paper presents a method to map user needs and integrate serendipitous search behaviors in search algorithm development: the living lab approach. This user-centered design approach involves technology users during technology development to catch unexpected insights and successfully innovate. This paper focuses on the preliminary findings of a living lab case study to answer the question how this methodology reveals fine-grained information about users’ serendipitous search behaviors. The case study involves a specific user group, media professionals who work in broadcast television and use audiovisual archives to create audiovisual content, during the development of new search algorithms for a large audiovisual archive. Research insights are based on data gathered during one co-design workshop, and ten in-depth semi-structured interviews with media professionals.

Findings stipulate that these users balance socio-technical constraints and affordances during creative retrieval to (1) find exactly what is sought; and (2) increase the possibility of serendipitous, unforeseen search results. We conclude that modeling these search processes in terms of improvising with constraints and affordances enables an effective articulation and channeling of user-technology interaction insights into new technology development. The paper suggests next steps in the living lab approach to further understand serendipitous search and creative retrieval processes.