no2pho: language, voice, space
'Language' and 'speech' are the most prominent human means of expression. Remarkably, on one hand language is a means of communication, and on the other a barrier to communication between distinct actors, due to its complex structural and semantic characteristics.
Fascinated by this double role of language, by its simultaneously defining and disrupting function, especially when situated in the interaction between humans and machines, No2Pho formulates an artistic answer to this complex behaviour of language and speech, both in an analytical and in a synthesizing manner.
The synthetic speech generated in realtime is coupled to different behaviours of the system, both of which are generated and modulated by the spatial coordinates of the human visitors. The physical world, represented by the psychogeographic trajectory of the visitor, is mapped on the virtual world, which consists of 36 sound sources. The visitors wear wireless headphones, equipped with 'crickets', a wireless sensor network. The crickets network registers the movement coordinates of the visitor, using RF and ultrasound technologies. The localization data is then used to modulate the parameters of the phonemes, adapting the realtime synthesized sound to the continuous movement trajectories of the visitors. Through this psychogeographic link, the visitor can 'penetrate' and interact with the sonic world.
At a given location in the space, different sound sources converge. The visitors have access to the complete sound information only at this particular position, enabling them to extract their own semantic meaning of the whole. The sonic experience is always tied to an individual, and as such it becomes a momentary display of the interaction between human and machine within the same system. Sometime the system will 'sing', other times it will 'whisper' or 'stutter' or 'frostily recite' the information, depending on the cooperation between the physical and the virtual.
It’s a play between the interwoven parameters of the physical and the virtual network. People and voices. Wireless headphones and multiple spatialized sound sources. People walk their trajectory through physical space, synthetic voices are generated simultaneously and are modulated by the parameters of localization and orientation: different pitch sets, tonal and timbre variations, cadence and accidental moments. In this way the text can be physically crossed and experienced according to the listener's own position. Voices are layered and mixed by movements and ‘dérivés’ through space. The perception of this new-born sound poetry is externalized on three levels: the understanding of language, the musical nature of language, and the linguistic nature of music.
2. on words and letters
The direct source of inspiration for this project is the formal, while at the same time poetic approach to linguistics by collectives such as the Lettrists and Oulipo. These groups experimented with words, sounds and letters in the utmost creative and revolutionary manner. The treatment of text is entirely sonic, emphasizing the formal deconstruction of words and letters, as well as the rhythmic representation of separate graphemes within a musical structure.
"Destruction of WORDS for LETTERS
This does not mean: destroying words for other words. Nor forging notions to specify their nuances. Nor mixing terms to make them hold more meaning.
But it does mean: TAKING ALL LETTERS AS A WHOLE, UNFOLDING BEFORE DAZZLED SPECTATORS. MARVELS CREATED FROM LETTERS [DEBRIS FROM THE DESTRUCTION]; CREATING AN ARCHITECTURE OF LETTRIC RHYTHMS; ACCUMULATING FLUCTUATING LETTERS IN A PRECISE FRAME; ELABORATING SPLENDIDLY THE CUSTOMARY COOING; COAGULATING THE CRUMBS OF LETTERS FOR A REAL MEAL; RESUSCITATING THE JUMBLE IN A DENSER ORDER; MAKING UNDERSTANDABLE AND TANGIBLE THE INCOMPREHENSIBLE AND VAGUE; CONCRETIZING SILENCE; WRITING THE NOTHINGNESS.
It is : the role of the poet to advance toward subversive sources. The obligation of the poet to advance in the black and burdened depths of the unknown. The craft of the poet to open one more treasure-room door for the common man."
MANIFESTO OF LETTERIST POETRY by Isidore Isou, 1947
The accent on the sonic qualities of letters and words pushes the semantic meaning of the text to the background, while confronting the listeners with the straight aesthetic experience. Words become physical and unlash a particular sensitivity within the audience; a direct and sensual sonic occurrence. The poetry is freed of its prose, which tends to emphasize the content, disregarding the performative value of text. The kernel of poetry is probed: quasi-invisible, highly intimate elements of experience are transmitted through an appealing sonic base. A physical experience of the sender and the receiver becomes entangled; a synaesthetic experience of drifting in the imagination of 'the other'.
2. the performative voice
Since time immemorial artists of various flavours experimented with the phenomenon of 'voice'. The voice is our primary means of communication, as well as one of the most robust human technologies, expressed in speech and singing.
New techniques for human-computer interaction and communication have to be tested within new structures; similar to scientific ideas being tested within scientific experiments. Sounds should be synthesized by combining frequencies, each of them with their own intensities, elapsing within a given time frame. Sounds constructed from code.
Within No2Pho, the 2-way human interaction of the Lettrists is researched from the digital and technological perspective, the 2-way human-computer interaction. All speech is computer-generated, without application or sampling of the human voice. The behaviour and display of this speech is created by the human individuals, whose spatial trajectories modify the parameters of the phonemes, attuning the pitch, timbre and rhythm of the synthetic voices.
This technique emphasizes the performative talent of the performer, in this case the computer, while depriving the text [in the first layer] of the restrictive intellectual approach to spoken expression. Text is described in a musical manner, following the structure akin to music. The notes in a score have their contextual meaning removed, functioning only as symbols. However, between words as phonetic material, as well as between the functional phoneme-units of this material, there exists an evident relationship. In verbal language the link between the signifier and the meaning is arbitrary, which makes it possible for a word to be seen as a symbol [de Saussure, Peirce], while the musical note can be seen as free of all arbitrary meaning.
It is important to design a system fit for expressing this concept in an appropriate aesthetic manner. For this design No2Pho deploys technology developed in the speech laboratory of the Université Polytechnique Mons. With the Mbrola speech synthesizer, based on the phoneme-system within which frequency and pitch can be annotated, the sound can be constructed and 'directed' down to its finest grains.
While being immersed in the system, the visitors discover their influence on the aesthetic display of the voices. After some time, they will discover the second, semantic layer of the work. The conscious migration of the visitors through the space guides them to a convergent position, where all information streams gather. 'Space' becomes a new dimension and an essential component of the system. Extracting meaning requires collaboration with the machine.
3. link to the space
Psychogeography is a study of specific effects of the geographic environment which [with or without particular order] influences the emotions and/or the behaviour of the individual. The Situationists linked psychogeography to the term 'situation', in the philosophical, scientific and artistic fields. The term situation includes the notion of 'locality', in physics defined as 'remote or far removed objects can not have direct influence on each other'. An object can be directly influenced by another object if existing in its immediate surroundings.
'Derivé' or 'drifting' is one of the approaches which guided the Situationists, especially Guy Debord. Drift is a technique where a person rapidly evolves, passing through different emotional moods and urban ambiances, guided by the invisible streams of a city. 'Derivés' demand an open and constructive behaviour from their participants, who should be conscious of the psychogeographic effects [both on themselves and the environment]. The Situationists envisioned that approaching and exploring the space in such a way, the space will open up to the individual, unveiling previously never experienced psychogeographic characteristics. This approach is opposed to the automatized behaviour of an individual, who usually follows the same 'parcours', without exploring the space and it's hidden qualities.
4. Realtime human-machine interaction
Being a part of a living system implies that all components of the system interact with each other in realtime.
Realtime: "of or relating to systems that update information at the same rate as they receive data, enabling them to direct control of a process, or the time it takes for the process to occur”. Steven Dixon
The poetics of an artistic environment cannot be reduced to separate materials or elements that constitute it. 'Process' as a medium becomes a communicator between the elements of the artwork and the individual. The concept of the work is woven through the process and can, as such, be transmitted to the individual. The individual can then inscribe the whole in a specific communication stream.
In this process of exchange, the individual drifts in and out of generated information, which gradually forms into a coherent whole, until the moment in which the visitor begins assigning meaning to the experience. Partially due to the input of individual's own variable parameters [space, time, culture], the artistic system finds itself in a continuous process of creation.
Schematically, we can describe the flow of information in this system as follows:
sender [the work] > code [the relationship between the separate elements which constitute the work] > medium/signal [the channel in which the transmission occurs] > reconstruction/decoding [the rebuilding of the information by the individual] > receiver [interpretation by the individual].
The transmission is always disrupted by a particular type of noise or interruption of the signal. The individual will have to interpret this signal and select correct information by ordering the elements in a specific way. While decoding the signal the visitors send information back to the system. In this process, the receiver becomes the sender and the initial sender becomes the receiver, creating a feedback loop. The information that is transmitted is based on this continuous correlation and exchange.
For optimal communication with the system the environment can be experienced by only one visitor at the time, except if additional people can be considered as 'interferences' and integrated as such in the functioning of the system.
Given that every signal is continuously coded, transmitted, decoded and sent back, the artistic environment finds itself in a constant state of transformation. The sender and the receiver become two systems inherent to the work itself as they interact and unceasingly modify each other's conditions. Furthermore, the work itself changes due to the transformations of these two systems. The visitors will interpret a realtime artistic work based on the duration of their participation in the system. The work will continuously unfold and adapt its own elements, becoming a self-organizing system.
A selection of existing texts gives shape to the basis of the work. These texts are selected as a commentary on specific situations within our arts and cultural society [Richter, Flusser, Stein, Beckett...].
The analysis of the text occurs in real time, by translating graphemes to phonemes, together with their corresponding frequency and pitch values. The phonemes are resolved and every virtual voice is assigned a specific set of behaviours.
The synthesis is influenced by the visitor. All parameters for the display of the synthetic voice are linked to the 'distance factor' of the individual. Based on the formal composition, the individuals themselves determine the speed in which the semantic meaning of the text becomes clear, or becomes utterly deconstructed, completely descending into noise.
The multi-platform output requires a dual interface structure: a sensor-triggered human localization/response, and a computer-mediated rendering of the data through the Mbrola speech synthesizer, which emulates intertwined phonological, prosodic and musical properties of speech.
The MaxMbrola object processes phonetic strings of text and allows the speech processing to be integrated within the Max/MSP environment, linked to the data gathered by the sensor network and finally binaurally spatialized.
Once generated, the synthetic voices are treated through a specific DSP [digital signal processing]. The correlation between the distance factor and the phoneme-parameters [with the addition of DSP effects] generates a range of possible varieties for the creation of realtime sound and image.
The result is a combination of physical action and machine aesthetics.
It’s a human machine collaboration.
The Lettrism movement was founded in the late forties by Isidore Isou .
In the development of the art of poetry, Isou saw the Lettrist at the end of a long chiseling phase which had begun with Baudelaire reducing narrative in his poetry to anecdote, then Rimbaud disregarding anecdote for lines and words, Mallarmé reducing words to sound and spaces and finally the Dadaists destroyed the word altogether. Isou saw at the end of this phase the new beginnings of an amplic stage for culture, from which a whole host of new arts, ways of working, and social institutions would eventually spring.
Oulipo was founded in the 1960's and stands for "Ouvroir de littérature potentielle", which translates as "workshop of potential literature". It is a loose gathering of French-speaking writers and mathematicians, and seeks to create works using constrained writing techniques.
Ferdinand de Saussure [1857-1913] was a Swiss linguist, considered by many to be the father of structuralism. He laid the foundation for many developments in linguistics in the 20th century. He perceived linguistics as a branch of a general science of signs he proposed to call semiology or semiotics.
Charles Sanders Peirce [1839-1914] was an American polymath. An innovator in fields such as mathematics, research methodology, the philosophy of science, epistemology, and metaphysics, he considered himself a logician. He saw logic as a branch of semiotics, of which he is a founder. In 1886, he saw that logical operations could be carried out by electrical switching circuits, thus anticipating the digital computer.
The Situationist International is an international political and artistic movement which has parallels with Marxism, dadaism, existentialism, anti-consumerism, punk and anarchism. Formed in 1957 the SI movement was active through the 1960s and had aspirations for major social and political transformations.
Guy Debord [1931-1994) was a writer, film maker, hypergraphist and founding member of the groups Lettrist International and Situationist International.
, currently developing the No2Pho-project, is a group of artists working with image, sound and technology.
So-on/ aka unamas is working with voice, language, text and sound in a transdisciplinary and convolutional way.
Edo Paulus is using generative processes to create sound and musical textures resulting in automatical music-generating software, live music performances, sound-installations and audio for interactive imagery.
Sukandar Kartadinata is a technician who is focused on custom music&art technology.
The MBROLA project, initiated by the TCTS Lab of the Faculté Polytechnique de Mons [Belgium], is developing a set of speech synthesizers for as many languages as possible, and provides them free for non-commercial applications. The ultimate goal is to boost academic research on speech synthesis, and particularly on prosody generation.
The MaxMBROLA Project is an MBROLA-based real-time voice synthetizer for Max/MSP.