Fool s Paradise Virtual Reality Installation and Performance

Transcription

1 Contact Information Paul Hertz (home/studio) 2215 W. Fletcher St (office) Chicago, IL Project Abstract Fools Paradise is an intermedia performance created by artist Paul Hertz and composer Stephen Dembski. Dembski scored forty-eight selected proverbs from visionary poet Willam Blake s Proverbs of Hell for soprano, cello, and flute. Hertz created a virtual world where each song is associated with its own locus on an island in the shape of a book, floating in a sea of stars. The artists used a common parametric space to develop the work as an intermedia composition and are developing a new version where the parametric space governs events in an interactive VR installation. Project Description Participants in navigate through the virtual world using a VR wand and head tracking gear. As they navigate, they trigger musical and visual events. In front of the performer or participant (the VR navigator) floats a mask whose position responds to the VR wand. By entering loci marked by sculptural elements the navigator triggers the performance of a song and the appearance of a new mask. The new mask accompanies the performer to the next locus. Each sculpture that has been visited is replaced by the image of its mask, floating in the air. All masks turn their gaze towards the navigator. Multi-channel spatialized audio software developed by Paul Hertz moves the music and other sounds through space, controlled by the navigator s position and rules developed from the underyling parametric space of the composition. Granular synthesis techniques create an aura of fragmentary music around the loci. In the installation, the trail of the navigator through the space remixes the recorded performance and permits the discovery of a complete song at each locus. In performance, live musicians follow cues from the navigator s trail through the virtual world to mix live and synthesized music and perform a complete song when the navigator activates a locus. VR navigation in performance begins in a dark cave on one side of the island and ends at a cascade of words that falls over a mesa into the sea. As an installation, the work does not have a specific beginning and ending, but will return navigation to the cave any time the installation is left alone for a period of time. Wayfinding cues such as a stairway ascending to the top of the tableland from the cave and new vocal material help installation visitors decide where to go. As a composition, is grounded in abstract notational structures that help to determine its visual architecture, musical material, and event-flow. In other words, it is an intermedia artwork, where events in one sensory modality may be mapped onto events in another modality. Fools Paradise approaches technology from a critical point of view and engages culture as a historically rooted enterprise. Blake s Marriage of Heaven and Hell, from which the texts are selected, presents a critique of the tyranny of religious institutions and a plea for freedom that resonates today. The use of VR as a performance medium rather than as a passively explored architecture, the blending of live acoustic music with electronics, and the use of hand-painted and scanned texture maps in a VR landscape all point to the historical continuity of art traditions. New media do not displace old; rather, they engage each other in complex feedback loops. As artists, first of all we create experiences not precious objects or new-minted cultural values. We honor Blake as a multimedia artist avant la lettre, an artist who melds poetic language with printed and painted image in his books. Like him, we seek to meld different languages into a robust and complex whole. page 1 of 6

2 Documentation Links All documentation for this proposal can be found here: Short videos of performance, May 1, 2004: (3 minutes) (8 minutes) Documentation of the development of, including the complete score: (Song 42, audio/score) Online portfolios and web sites: Images from Performance (2004) Performance space with musicians on left, projection in center, VR navigator to the right. Aerial view of the tableland in the virtual world, with 48 event loci page 2 of 6

3 To create a little flower is the labour of the ages. The roaring of lions, the howling of wolves, the raging of the stormy sea, and the destructive sword. are portions of eternity too great for the eye of man. At each locus in a different mask accompanies the performance one of 48 different songs. A performance may consist of some or all 48 songs, in an order determined by the VR navigator. Cues from the VR world prepare the musicians for each song, and present them with the score for the current song. The masks are first executed in watercolor and then digitized and scaled for use as texture maps. They can be exhibited as part of an installation of. Following are some of the texture maps. page 3 of 6

4 A selection of texture maps for masks in. page 4 of 6

5 Intermedia Composition in Pitch class aspect of parametric space used as source material in musical composition of. Many aspects of were developed in collaboration between a digital multimedia artist, Paul Hertz, and a composer, Stephen Dembski. Hertz had developed a body of theory for intermedia composition, which proved to have useful analogues to Dembski s theories of extended tonality. Thus, the diagram above plots the relative positions of event loci in the virtual world and it also represents the available tonal material at each locus. In future developments of, Hertz and Dembski plan to use the topology of the graph of the parametric space to generate rules for remixing audio grains from a multi-channel recording of made after the performance. If in the original composition of intermedia correspondences worked at a relatively large scale, in the version in development they will work at a much finer granularity, managing real time audio synthesis under the partial control of the VR navigator. Diagram of VR world with navigator track (orange dots on black line) and penumbral regions(shaded circles) surrounding each locus (solid circles). The navigator track is an invisible record of the navigator s recent positions in the virtual space. It is designed as a series of event triggers (orange dots) distributed over an inverse kinematics system of weights and springs. If the navigator moves rapidly, the track stretches out. If she moves slowly, it contracts. If she stops, it collapses to a point. Pulses (black triangle) flowing back from the navigator s current position trigger audio events. Audio grains are selected out of the song associated with each penumbral region where a trigger is located and remixed. page 5 of 6

6 Revisions to Visual Design and Music Sketch of new sculptural elements for the 48 loci in. Along with way-finding elements and an extended interface for real time audio synthesis cued to position and wand gestures, the virtual world will use sculptural elements more in keeping with the the mythology of Blake s poetry than the icosahedral jewels of the first performance. Stephen Dembski has written new vocal material based on The Marriage of Heaven and Hell which we are currently recording. The new material helps to establish the setting of the virtual world and provide it with a sense of narrative direction. Installation Performance and installation require somewhat different configurations. For the installation, two projectors project two slightly different images one from a right eye perspective, the other from a left eye perspective through polarizing filters onto a rear projections screen. Visitors to the installation stand in front of the screen and wear polarized glasses that block one image and pass the other, so that each eye sees a the image intended for it. As a result visitors will see the virtual world in stereo, particularly if they are privileged to wear the tracking device and navigate with the wand. For the performance, one projector will provide a larger image for an audience. The VR navigator who controls the voyage through the virtual world will not need to stand in front of the screen, but can follow the navigation on a monitor. Ideally, the audience will be surrounded by eight-channel spatialized sound. page 6 of 6