From Semiology of Graphics to Cultural Analytics: flaws in the mathematization of visible

Abstract. The goal of this conference is to study the legacy of Jacques Bertin’s Semiology of Graphics in the field of dataviz, and more precisely in Lev Manovich’s Cultural Analytics.

Published in 1967, Jacques Bertin’s Semiology of Graphics (Sémiologie graphique) aims to establish the representation of topographic mapping on a scientifically “proven” system. Bertin relies on an objective (Desbois, 2018) and rational system of signs called “visual variables” (Bertin, 1967). Evoking the old Gestalt Theory (Metz, 1971), the Semiology of Graphics wants to build an “efficiency” of communication at the expense of precision. According to Bertin, it is essential to simplify the information beforehand. There is also the idea of a moving and dynamic image that makes sense through tangible manipulation. Its matrix table system (“ordinable matrices”) allows readers to configure their own reading of data by appealing to their aesthetic feeling. This is what Bertin called “the painter’s eye” (Palsky & Robic, 2000).

It is striking that Bertin depreciates aesthetics to the detriment of efficiency and logical rules – a kind of “doxa” of the visible. While the Semiology of Graphics has greatly influenced cartographers since the 1970s, this paradoxical (repressed?) place of aesthetics in the construction of scientific images is still alive (Renon, 2016). We thus find many resurgences of this desire to mathematize the visible in contemporary information design practices, for example in dataviz. This rapprochement between Graphics and computer sciences (IT) had also been initiated by Bertin himself through his many collaborations with IBM.

We could consider that dataviz deviate from Bertin’s principles because they too often favour technical performance over effective understanding of information (Buellet, Gimeno & Renon, 2018). On the other hand, the primacy of the efficiency of computer code (the “efficient” processing of large datasets) may suggest that digital code would, in a way, have replaced visual variables. Caught in a tension between the abstraction of computer code (which is no longer manipulable, as was the case in Bertin’s ordinable matrices) and their visibility (their representation on the screen), dataviz is an interesting object of study to rethink the relevance of Bertin’s theories today.

We will focus our study on the field of Cultural Analytics. Initiated by the artist and computer scientist Lev Manovich in the early 2010s, Cultural Analytics compute human cultural productions (social media, works of art, etc.) in “patterns” to identify matches and recurrences (Manovich, 2018). Some of the most well-known projects (interactive experiments and exhibitions) include Phototrails / Instagram Cities (2012–2015) and Selfie City (2014). These real technical feats do not escape the impasses of topographic map neutrality denounced by John Brian Harley in the late 1980s: “As they embrace computer-assisted methods and Geographical Information Systems, the scientistic rhetoric of map makers is becoming more strident. The ‘culture of technics’ is everywhere rampant (Harley, 1989).” This transition from topographical mapping to the mapping of human cultural productions raises questions about visual variables: is a mathematization of the visible desirable or even possible?