1We use IT appliances for communicating, watching movies, paying our taxes, bills and doing shopping. Computers and software are one of the most important parts of our everyday life. Our IT appliances have become indispensable assets for the majority of contemporary individuals. It would have been hard to conceive our life if we had not had them at our disposal. “This new revolution is arguably more profound than the previous ones, and we are just beginning to register its initial effects” (Manovich, 2001, p. 19). As a matter of fact, we share with ICT (Information & Communication Technology) equipment a new kind of space, that is, the informational environment, the infosphere, “the global space of information, which includes the cyberspace as well as classical mass media such as libraries and archives” (Russo, 2012, p. 66). Recognizing it “brings upfront the need to reinterpret Man’s position in reality – that is Man’s position in the infosphere” (Ibid.).

2The present work aims at demonstrating that the software-based experience, namely the experience mediated by digital technology, therefore by software, is based on a radical break between the code and what I call “lived experience,” i.e. the use of the device – the experience made by ordinary users throughout management of ICT appliance. [1] This break entails deep consequences for our way of perceiving the world. As Floridi (2010; see also Floridi, 2011) claims, the digital revolution brings up questions about the relations between physis and techne, respectively understood on the one hand as nature and reality, and on the other hand as technique and technical practices. I want to emphasize that this revolution affects first of all the position and the role of human beings as perceiving and epistemic agents. If the experience becomes digital, its unity becomes problematic.

3The paper is organized as follows. I start (section 2) by analyzing a case study: the video game Assassin’s Creed: Odyssey III. A video game is a relationship between a human being (the user) and a machine that can go so far as to redefine all aspects of the world’s experience pertaining the user. Analyzing the video game experience is useful because video games are one of the most extreme and profound digital experiences – currently one of the most common ones in any case. In section 3, I examine the gap between code and lived experience. I claim that the code’s concealment is the necessary condition of the digital experience. The code must be absent: when I play a video game or look at a picture on my laptop screen, I do not see the code, I do not even know what code is creating that experience. In order to know the code and to understand it, I need a specific knowledge and a technique. However, the code is the source of my digital experience. Aiming at reaching the core of the break between code and lived experience, in section 4, I wonder about the very nature of software, about the ontological definition of software as an entity. In the last part of the essay, section 5, I argue that the break between lived experience and code is recomposed by imagination through the act of design.

Assassin’s Creed: Odyssey III [2]

4The video game Assassin’s Creed: Odyssey III (Ubisoft, 2018) is an open-world game in which the most important aspect is not a fixed and static goal (conquer a city, free a princess, kill the enemy, complete a path, etc.), but the exploration of a world – a historical world in this case: the ancient Greece and the Persian wars. The general narrative plot is articulated in an infinite multiplicity of possible alternative stories: love relationships, travels, encounters with other characters, knowledge of new parts of the context: the ancient Greece is represented in a mythical, lunar, rarefied way.

5The plot is complex, articulate and compelling. The game is a part of a saga. The protagonist is Layla Hassan, already present in Assassin’s Creed: The Origins (2017). The game opens with the Battle of Thermopylae. The Spartan king Leonidas leads his soldiers by urging them to fight against the Persians of Xerxes. After this initial scene, the set piece goes up to our days. During an archaeological expedition, Layla, an agent of the Assassin Brotherhood, finds the ancient spear that belonged to Leonidas and, with the help of a friend, Victoria Bibeau, manages to extract the DNA of two brothers who lived in the 5th century BC during the Peloponnesian war, Alexios and Cassandra. Therefore Layla must choose whether to follow the story of either the former or the latter. Her ultimate goal is to find the stick of Hermes Trismegistus and restore the final order of things.

6The game offers a vast territory that can be explored freely. The player is placed into a setting that he / she can know in the smallest detail, potentially going on forever. She / he can interact with his surroundings in many different ways and can engage in combat with any person or animal, using a vast arsenal of weapons.

7The most interesting characteristic of the game is that there are potentially endless development possibilities. There are two kind of possibilities offered to the player: (a) movements, that is the possibility of moving in the Odyssey world, crossing valleys, islands, caves, beaches, lakes, half-destroyed temples, cities, forests, etc.; (b) actions, namely the possibility of carrying out an action instead of another: killing (above all), loving, making friends, making sex, stealing money, or having conversations with philosophers, politicians or priests. The story is amplified also with new parallel narratives provided by the developers through the game’s website, such as the stories “The Lost Tales of Greece: Daughters of Laila” and “A Poet’s Legacy”. It is not a situation comparable to a book or a film that present closed, defined narrative evolutions.

8This video game is not simply a story, an object, a film, a toy, or a program, a formal system of rules making a machine work, but an all-encompassing experience replacing – or trying to replace – reality and directly involving the user, the subject who plays, or the subjects who play together, as in multiplayer games. In this case, digital technology creates new environments, and “those environments are at once either cognitive – in the sense of the space of knowledge – and applied – in the sense of the space of application of such a knowledge” (Russo, 2012, p. 67). This totalizing experience, in some cases, even comes to challenge the user as a subject: think about the multiboxing mode, where a single player can play more roles, impersonate more characters and watch the game world from multiple perspectives.

9This all-encompassing experience poses a philosophical issue. As Triclot (2013, p. 18) writes: “where are we when we play a video game? […] we are in an intermediate zone” (translation is mine). Triclot refers to Winnicott (2005) and his concepts of “transitional objects”. The experience of the game – the play – lies neither in the subject nor in the machine, but in their interaction. Defining this intermediate area is very hard because it requires the analysis of many different aspects: the pleasure produced by the play, the consumer dimension, the user’s involvement in terms of senses, body and emotions, the mutual adaptation of software and user.

10Playing is above all fun. As Caillois (1958) explains, following Huizinga (1951) [3], the experience of fun is based on four essential and irreducible impulses mixed up in different proportions: competition, chance, simulation, vertigo (fear of defeat, bankruptcy, death, etc.). These four categories are not fixed labels, but form a polarized, moving space. The game is a free activity, separated from the rest, which develops within a specific space, where the four categories enter into ever-changing relationships. The video game summarizes, takes to the extreme and transforms Caillois’ categories thanks to the digital technology. Triclot (2013) defines the relation between player and machine in the video game an hallu-simulation: the player finds himself / herself in an artificial world where every aspect is the result of a code, of software, except the player himself / herself. A hallu-simulation is a paradoxical combination between the freedom of fun and the determinism of an algorithm.

11The alternative universe produced by software is not purely mechanical or deterministic. The machine does not eliminate chance at all, but controls it, and this produces the player’s emotional reaction. In Tetris (Pazitnov, 1984), for example, the number of types of shapes or blocks is more limited than the number of ways these shapes or blocks can appear. This difference produces the concealment of the finiteness of the combinations making the blocks appear on the screen. The player is led to believe that these combinations are infinite, uncontrollable, unpredictable, even if they are not. This illusion is a hallu-simulation. An open-world game uses precisely this dissimulation effect to produce in the player a feeling of freedom and improvisation.

Code vs lived experience

12The Odyssey case study is useful to define the central issue that the present paper aims at investigating. A video game is a relationship between a subject and a meta-medium, the computer (Manovich, 2013), that can go so far as to redefine all aspects of the individual’s world perceptions. An open-world game defines another world. How is this experience possible? Is there a digital experience?

13The “softwarization” of the world is the most global and complex phenomenon of the last thirty years (1990-2020). If we want to understand contemporary society, our analysis cannot be complete until we consider software as the main cultural (and philosophical) factor. Today every part of our life, not only communication, has been expressed by software. Software never sleeps: Google, Facebook, Amazon, Twitter, eBay and the other main IT brands change their algorithms daily, and this is an automatic process. Many software change themselves by themselves: they have an autonomous “life” materialized in many servers all over the world. This “life” is the core of the global economy, culture, social life and politics, but “this ’cultural software’– cultural in a sense that it is directly used by hundreds of millions of people and that it carries ’atoms’ of culture – is only the visible part of a much larger software universe” (Manovich, 2013, p. 7 – emphasis added). Therefore “software is the invisible glue that ties it all together […]. software as a theoretical category is still invisible to most academics, artists, and cultural professionals interested in IT and its cultural and social effects” (Manovich, 2013, p. 8-9). It is a new kind of lived experience requiring an entirely new philosophical approach.

14The relationship with the media is essential. By redefining our media, the computer creates a new kind of world experience: “[…] the computer media revolution affects all stages of communication, including acquisition, manipulation, storage, and distribution; it also affects all types of media – texts, still images, moving images, sound, and spatial constructions” (Manovich, 2001, p. 19). Software is the core of this revolution: “[it] shapes our understanding of what media is in general” (Manovich, 2013, p. 121).

15Now, by transforming our media, software modifies our experience of the world. In this redefinition, however, software opens up a radical and profound fracture. The thesis of this paper is that this fracture involves an issue which is specifically a philosophical one. That is, the unity of the experience and the status of the digital experience. To solve this issue, we must ask ourselves another radical question: what is software?

16Let us go back on our case study. Odyssey III produces a very precise simulation of a physical experience. The play presents two layers: the first one is the code, which is an autonomous type of writing, ceaselessly reading and modifying itself; the second one is the lived experience, that is, the interaction between the user and the metamedium depending on the effects of the code. I say “the interaction between the user and the metamedium” and not “the interaction between the user and the machine” because the metamedium cannot be understood as a physical machine. Of course, a metamedium involves many physical machines, but it is not just a bunch of devices. Actually, the physical devices do not really matter; I can have the same experience (play my video game) on different devices. Then by “lived experience” I mean the interaction between the users and the metamedium conceived in both physical and abstract sense.

17Therefore, there is a difference between the code, i.e. the algorithm, used to play, which is a string of characters, a writing [4], and the relation between the game and the users, that is, the play itself. The first level is invisible: the computational process is hidden.

18For instance, one may think about the asymmetry between the discrete level of screen pixels and the vision I have when I play my video game, which is continuous (Salanskis, 2011, p. 113-114). This an effect of the digitalization consisting of two steps: sampling and quantization. Firstly, an image is translated into a set of screen pixels: it is fragmented in minimal pieces. “Sampling turns continuous data into discrete data, that is, data occurring in distinct units: people, the pages of a book, pixels” (Manovich, 2001, p. 28). Secondly, pixels are quantified; “[each pixel] is assigned a numerical value drawn from a defined range (such as 0-255 in the case of an 8-bit greyscale image)” (Ibid.). Sampling and quantization are the two main stages of the distance created by the code.

19The code is hidden not simply because it is invisible. The point is that writing and reading software are opposed to the experience of looking at an image. When I play the video game I live a single and consistent experience of fun and involvement. And yet, “behind the screen” there is something completely different: a string of 1 and 0. Where does the synthesis come from? Learning to play does not mean learning the algorithm. I can also become a champion without knowing the code at all, nor the possible different algorithms that “stand behind” the screen. I could also try, when I play, to make a strange experiment: to observe on another screen, next to the one which I am playing on, the string of code that develops as the game goes on. However, I would not get much: 1) if I were not a programmer, that string would not tell me anything – it could also be a Balzac novel, I could not distinguish the two; 2) even if I were a programmer, I could not be sure that it is precisely that string of code that runs the game because the same game scene – the same effect – can be caused by very different algorithms, as well as the same software can refer to different algorithms; 3) even if I were the creator of the game, I could not be sure that the string I see on the screen next to the one which I am playing on is just the string that produces that game, because it could be an old version of the program that I created, which has meanwhile evolved, has transformed itself – or thanks to other programmers.

20We can now better formulate the problem: what holds together these two opposite sides – the algorithmic manipulation and the lived experience – of one single event? Why and how does the user interpret those electrical stimuli produced by software in that precise way?

What is software?

21In responding to the objection, I want to highlight some crucial aspects:

22a) Software is not a tool. It is not a mechanism such as a car engine or an electrical system. Behind software, there is culture: a style of writing, an aesthetic vision, a social perception, a commercial objective. Software is a cultural system, in accordance to Geertz (1973), i.e. a network of meanings. This is the thesis of Finn (2017) which establishes a new “algorithmic reading” by creating concepts such as “algorithmic imagination” or “algorithmic aesthetics”. For Finn, the algorithm is not a simple symbols’ manipulation separated from the meanings; it is not a purely syntactic concept. On the contrary, the algorithm has not only consolidated cultural and historical roots, but also significant consequences on the way in which human beings act and conceive themselves and their world. The algorithm “is an idea that puts structures of symbolic logic into motion. It is rooted in computer science, but it serves as a prism for a much broader array of cultural, philosophical, mathematical, and imaginative grammars” (Finn, 2017, p. 41). That is why the algorithm is a “culture machine” which “operates both within and beyond the reflexive barrier of effective computability, producing culture at a macro-social level at the same time as it produces cultural objects, processes, and experiences” (Ibid., p. 34). According to Finn, we must distinguish three different levels of interpretation of the concept of algorithm: a) the computational reading, the pure mathematical structure theorized by Turing, Church and Gödel, and which can be summarized by the concept of effective computability; b) the cybernetic reading, developed above all by Wiener, in which the algorithm becomes the instrument for the control of information and communication applied to society, biology and physics; c) the symbolic reading, according to which the algorithm develops the idea at the root of magic, i.e. the use of language to produce effects in the world.

23b) Software is a living organism, and its life goes beyond the machine and the particular experience of the user. Software is an autonomous environment, with its own rules. Software often updates itself, works and develops without human intervention – the same engineers cannot figure out what is the behavior of the system created by them: “Wall Street traders give their financial ’algos’ names like Ambush and Raider, yet they often have no idea how their money-making black boxes work” (Ibid., p. 16). Web pages are automatically generated from databases by using templates created by webdesigners; furthermore, “information about the user can be used by a computer program to customize automatically the media composition as well as to create elements themselves” (Manovich, 2001, p. 37).

24c) Software has an artistic dimension. It is not just a combination of logical-mathematical signs and operations. The beauty of software is a crucial element: the more beautiful software is, the more it works and it is alive. Software designers are artists and strongly claim this role (see Sack, 2019). The pioneers of digital media such as Sutherland, Nelson, Kay, Agroponte, Engelbart have underlined the importance of the artistic factor in their inspiration (see Wardrip-Fruin, Monfort, 2003). Of course, “culture” is behind all technological artifacts. Technologies are the result of cultural transmission; they have specific styles and aesthetic visions. I claim here that software has always a tendency to go beyond its technical and logical-mathematical nature, and that this tendency produces a particular form of cultural contents.

25Given this, how does the opacity of software differ from that of other technologies? Let us suppose a human being traveling in his car. At some point the car breaks down. He / she stops and opens the car’s hood: what does he / she see? He / she sees the engine. With technical knowledge, he / she is also able to explain the car’s functioning and perhaps to repair it. In any case, the driver has, or can have, a direct perceptual experience of machine operations: combustion, petrol, internal mechanisms, etc. He / she can disassemble the machine piece by piece and show the direct connection between movement and engine.

26Now let us consider another human being while using his laptop. Let us suppose that he / she decides to open the laptop and look inside: what does he / she see? He / she does not see the machine working, but a cluster of circuits that only partially tell him / her something about how the laptop works. This is the point: the actual functioning escapes continuously. Let us suppose that he / she is a programmer and knows perfectly the software that runs on his/her computer. However, the connection between code strings and machine operations varies continuously. Let us suppose – recalling our previous example – that he / she could visualize on three different screens at the same time the electric impulses in the computer, the data stored in the memory and the code strings. Even in this case, he / she could not directly experience the connection between strings, data and electrical impulses. The user must presuppose this connection but cannot perceive and / or show it. Even if he / she dismantles the laptop, he / she will not find this connection. The user cannot show why an abstract symbol produces a physical effect.

27The reason for the discrepancy between code and lived experience is not in our ignorance. It is structural; it is a material a priori, to use a phenomenological expression. Software is not the computer’s instruction booklet, a technical knowledge detached from the machine. Software is the operating of the computer (or at least, an essential part of it). Software writing takes place in the computer. It is performative: it does what it “says” (writes / re-writes). This is why the illusion of transparency and immediacy of technology turns into a paradox.

28These considerations lead us to a crucial question: what is software? We have to understand the specific nature of software to find the roots of the break between code and lived experience.

29Let us start with a definition that has become a classic: that of Wirth (1976), according to which “algorithms plus data structure equals programs”. This definition is extended by Manovich: the intellectual work of programming consists “of two interconnected parts: [the first one is about] creating the data structures which logically fit with the task which needs to be done and are computationally efficient and [the second one is about] defining the algorithms which operate on these data structures”. Thus, software is “a combination of a data structure and set of algorithms” (Manovich, 2013, p. 207-208). To make a comparison with language: “we can compare data structures to nouns and algorithm to verbs. To make an analogy with logic, we can compare them to subject and predicates” (Ibid., p. 211).

30Is this first definition satisfactory enough for us? It is obviously not. It is a descriptive definition, not strictly philosophical. We must go further.

31From an ontological point of view, software is an enigma. Colburn (1999) defines software by coining the expression “concrete abstraction”, underlining its dual, almost contradictory nature. Abstractness and concreteness are essential properties of software. Colburn develops this thesis through two arguments:

32a) Computer programming is not a branch of pure mathematics – mathematics deals with formal abstractions, while programming deals with abstractions that do not eliminate the content but widen it, make it more expressive.

33b) There is a deep distance between the binary language and the physical states of the corresponding machine; “the characterization of physical state in machine language as zeros and ones is itself an abstraction; the kinds of physical state that this language ’abstracts’ are boundless. They may be the electronic states of semiconductors, or the state of polarization of optical media, or the states of punched paper tape, or the position of gears in Babbage’s 18^th century analytical engine”. This process of abstraction becomes more and more wide: “the whole history of computer science has seen the careful construction of layer upon layer of distancing abstractions upon the basic foundation of zeros and ones” (Colburn, 1999, p. 16).

34Colburn’s thesis is that we cannot reduce a side to the other either: a monistic approach is useless in this case. On the other hand, a dualistic approach cannot be causal: how can a string of numbers and operations give rise to a physical state? Colburn prefers to speak of “pre-established harmony” (Ibid., p. 17), namely a parallelism between code and machine established not by God, but by the programmer. “Programmers today can live almost exclusively in the abstract realm of their software descriptions, but since their creations have parallel lives as bouncing electrons, theirs is a world of concrete abstractions” (Ibid., 18).

35This conclusion is criticized by Irmak (2012) who prefers to define software an “abstract artifact”, that is an abstract object, built by the human mind for a precise purpose. Software has no spatial characteristics, but it is placed in a time, as well as in a historical period – it is created and can be destroyed, unlike Platonic ideas. Irmak develops his thesis through a comparison with music: “I think that both software and musical works are abstract artifacts which are created by software engineers or composers with certain intentions”, and therefore “computer programs are not types and thus the relation between computer programs and their physical copies cannot be understood in terms of the type/token distinction” (Irmak, 2012, p. 70). Nevertheless, Irmak’s conclusion also seems problematic or incomplete. In which sense the following statement “musical works and software come into existence by some human being’s act of creation” (Ibid., 71) can help us to understand the ontological status of software? Irmak’s thesis, although true, does not make us progress at all.

36The whole problem must be re-considered. A software ontology should start from what computer programming actually is, that is to say, a long and complex process articulated throughout numerous phases. I think there are two key concepts to be considered: problem solving and coding. The first one is the more creative part of the job. As many programmers say, “too sooner you start coding your programs, the larger it is going to take”, thus “first think, code later”. Programming is essentially problem solving. Thus, understanding and defining the problem are the core of this activity. We must know exactly what we want to do, outline the solution – organizing tasks, subtasks, units, etc. – and then select and design the algorithm. Even the procedure of writing an algorithm in a correct way is not easy at all – for instance, its fundamental part is specifying how a value will be assigned to an operation. The coding-programming depends on a problem solving matter. The choice of the programming language will be dictated by the nature of the problem (building a website, making up an app, etc.) and the languages available on the computer in use.

37Therefore, I call software a specific artifact defined by a) a problem, b) an algorithm (logic + control), c) a code (a formal language). By this I do not mean that the only way to define software is a functional or instrumental one. I am trying to isolate an elementary conceptual “cell” through which to build our definition of software. I am only saying that a basic form of software ontology – a discourse interested in defining software as entity before other technical, artistic, cultural and social determinations – should start from these three categories as connected parts of a unique entity. In other words, I claim that these three categories are three variables whose unity is complex.

38First consideration: none of these aspects, taken separately, is software. Second consideration: these levels make software happen provided that they can be taken as a whole. Software is the dynamic synthesis of problem, algorithm and code.

39Software is not the algorithm. The same algorithm can be written in many different types of languages: there is no direct relationship. Obviously, the algorithm is essential because it is what is written – it is the meaning. However, software cannot be identified with a single programming language. It is not a single high-level or a low-level language: we can in fact have different high-level or low-level languages connected to the same software, or vice versa. Likewise, we cannot state that software is just the machine code, the pure binary code, even if it represents the heart of the machine. By itself, a string of 1 and 0 has no meaning.

40Each aspect of software is interpretation: the algorithm is translated into the high-level programming language, which in turn must be translated into the machine language through low-level languages. The machine language, in turn, must be translated into a series of physical states: that is the execution. The problem is the relationship between these levels: Does the translation have a causal scope? Can an algorithm – that is, an abstract entity – give rise to a physical state? If we accept Colburn’s position, the programmer would be a sort of deus ex machina who optimizes the level’s harmony upon request. Yet, often the programmer does not know what really happens, he/she cannot follow the life of software, he/she is unaware of its developments.

41According to the perspective I propose, software is a complex artifact, consisting of a set of elements, but irreducible to only one of them. Its unity cannot be reduced to a sum of the parts.

Conclusions. Design as imaginary act

42Let us go back to our main issue: the break between software and user experience. I want to propose a solution. My hypothesis is that the mediation between software and lived experience must be sought in a more subtle and complex concept, that one lying in design. In the act of design, imagination responds to the break produced by software. In all digital experience’s levels, from the construction of software to the graphical unit interface (GUI) and to the physical machine, design guarantees the unity of digital experience – and the unity of software as well.

43From a philosophical point of view, design is a hermeneutical act which interprets and changes our experience. Vial (2013) claims that design transforms our experience in three ways: ontophanique, callimorphique and socioplastique. The “design effect” is not to be understood as a logical consequence of a syllogism, but as a phenomenological effect (Vial, 2017). A design object transforms and improves firstly our sensorial perception. Vial calls this effect effet ontophanique. “The history of technical revolutions is the history of ontophanique revolutions” because “there is no world’s manifestation outside the technical conditions in which phenomena are possible” (Ibid., p. 21, translation is mine). There is therefore a direct relationship between the technical system and the perceptive mode. Perception is always built, amplified and transformed by the overall system of devices which we cope with (telephone, camera, typewriter, laptop, etc.).

44Nevertheless, Vial mentions also another design effect: the effet callimorphique. In fact, design produces the perception of formal beauty. Design means creating harmonious forms, with a character, an expression, an empathy, and therefore responding to a primary human need.

45There is finally a third design effect: the effet socioplastique, that is, a desire for a social reform. Design invents new ways of social interactions. It always encapsulates a vision about a new possible world. The forms created by a designer, unlike those of art, have a public use, a public life. More generally, design is always driven by the ambition for a new society, a transformation of society.

46Therefore, design is a phenomenological revolution: a revolution of our way of perceiving reality and society. “A phenomenological revolution is produced when the act of perceiving is affected or modified by an artistic, scientific or technical innovation” (Ibid., p. 17). The phenomenological revolution in the digital age is a psychic and social event; it consists in “learning to perceive digital beings for what they are, without metaphysical superstructure […]” (Ibid., emphasis added).

47Design interprets and modifies our perceptive, aesthetic and social experience, in order to answer to the challenge of the break between software and lived experience. Thanks to design, the fact of not experiencing code phenomenologically becomes irrelevant to users – namely, users do not feel the need to look for software, although software is in fact the condition for their experience.