Quantifying Emotion: Survey Methods and Sentiment Analysis in Cartographic Design Research

Abstract. Despite the strong connection between art and emotion – and cartography’s position as both an art and a science – only recently has there been a strong call for cartographers to partake in research related to maps and emotion (Griffin and McQuoid 2012). Such research can typically be classified into one of three categories: (1) maps of emotions (i.e., visualizations of spaces as happy or fearful), (2) maps as tools for the collection of emotive data, and (3) affective responses to maps (Griffin and McQuoid). Of these, the first two have been the primary focus of most recent research in emotive cartography; this research has benefited greatly from the proliferation of participatory web-based technologies such as social media, smart phones, and fitness activity trackers.

The third category of affective map research – the study of emotional responses to maps – has received less attention, though some notable work (e.g., Fabrikant et al. 2012) has highlighted its importance. We suggest that such research on affective responses to maps might also benefit from an increased reliance on web-based data collection techniques. There has been, for example, a recent increase in the use of online crowdsourcing platforms such as Amazon Mechanical Turk for cartographic and related visual perception research (e.g., Heer and Bostock 2010; Limpisathian 2017). Such platforms decrease the cost of participant recruitment and survey dissemination, while facilitating easier testing of samples outside of the traditional undergraduate student subject pool.

Given the inherently personal and subjective nature of emotions, the use of crowdsourced samples to measure affective responses to maps poses an interesting dilemma. How might cartographers assess the emotive nature of maps and mapping products via participant samples that are both geographically dispersed and impersonal by design? As a case study, we present a user study recently conducted to measure participant responses to affectively-congruent (vs. incongruent) chorochromatic maps. By affectively-congruent, we mean maps with data topics whose emotive content is in harmony with the emotive nature of the colour scheme. For example, a thematic map about homicide with dull, muted, colours would be considered affectively-congruent, while one with bright, highly-saturated colours would be considered incongruent (Figure 1).

In this work, we discuss two contrasting methods for collecting and analysing emotive participant interpretations of these maps. First, with closed-ended survey questions, wherein study participants are asked to rate how much a map exhibits a type of emotion (e.g., positive). In our study, we used a modified 7-point version of the Affective Slider, a tool developed by Betella and Verschure (2016) for measuring emotion in web-based surveys. Thus, due to our 7-point scale, participants classified their rating of a map’s emotive content into one of 7 ranked categories. We use a between-subjects design and analysed participant responses using non-parametric tests.

The second method we discuss for analysing affective response to maps is the collection of free-form responses from survey respondents, followed by the classification of these responses using sentiment analysis. In our study, we asked participants to provide three words they would use to describe each map. We then categorized these responses as either positive, negative, or neutral using the ParallelDots Sentiment Analysis API. Thus, we did not ask participants to classify a map’s level of emotion specifically, but rather collected their more instinctive responses and categorized them later using text classification algorithms.

Following a discussion of the respective merits and drawbacks of these methods, we discuss how the selection of one of these contrasting approaches might influence affective cartographic research results. For example, in our study, while the Affective Slider proved a more efficient tool for analysing between-subject variance in emotive rankings of maps, sentiment analysis of free-form responses facilitated better analysis of within-subject variance. One participant, for example, listed both “exciting” and “depressing” in response to a single affectively-incongruent map, highlighting the confusion that can be triggered by a map with an incongruent design.

In closing, we discuss the multitude of challenges introduced by these methods – including the false precision sometimes provided by fine-grained survey scales and complex algorithms, and the paradoxical nature of personal data collection via anonymous participants. But despite these challenges we propose that crowdsourced surveys show strong promise for quantitative affective mapping research: future work is likely to result in new, exciting insights into user’s affective responses to maps and mapping products.