Turn All Things | Overview
Turn All Things is an experiment which explores the movement of time within an image. This project utilizes various aspects of computing — image analysis, random access of media, and logical iteration — to give new rules to a flow of images, granting this flow a different evolution. One way to think of this project is as another step in the technological mediation of the world around us: the photograph and early cinema capture and hold in stasis a particular image in time; editing techniques, such as montage and mixing, bring together a confrontation of images to fold various events into a new event; and now computing techniques allow us to substitute new rules for the interaction of images, in a sense giving them a different sort of life.
Description
Turn All Things is comprised of two main efforts. The first is original video of various subjects, decomposed into short image sequences. Each image sequence is processed using image analysis to catalog various formal aspects. For example, these formal aspects may include: the reflection of light off the edge of a table, or the droop of a leaf, or the tilt of a chin. The second part of the project is custom software which reintegrates these image sequences, creating a new flow. This new flow is governed by rules based on relationships between the analyzed image sequences. The result is an evolution and movement in time: life, but not quite natural. The desire is not to approximate life, but to produce an evolution other than life.
The subjects chosen for this project come from the traditional categories of still life, portraiture, and landscape, and each subject is displayed separately in its own flat screen monitor mounted on a wall. There are two interior scenes in a Parisian apartment; one portrait of my wife in our apartment in New York City; and four outdoor scenes on the rocky coasts of Ingonish, Nova Scotia. Each case underwent its own particular analysis. For the portrait, attention was given to the positions of the eyes, nose, and mouth; for an interior, it was the intensity of the afternoon sun; for a rocky promontory, the number of seagulls. Accordingly, different rules of interaction were given to each subject, and each set of image flows revealed its own particular transformations. My wife's portrait exhibited a coyness not present in the original footage; a twilight ocean scene simulated an otherworldly visitation; and a flock of seagulls became an ironic meditation on particle physics.
Video Captures
The following are video captures of Turn All Things during playback. Nb: The following videos are simply recordings; actual playback is entirely dynamic.
Videos are offered in three formats:
- AVI
You must have the DivX codec installed. If you don't have the DivX codec, get it from www.divx.com [ win >>* | mac >>* | linux >>* ], but you may want to bookmark this page first. - QuickTime
You need QuickTime 5 or higher, available at Apple's site >>*. - MPEG-1
This format should be viewable on almost all players. Choose this format if the other formats don't work for you.
If you wish to download a video, shift-click or option-click on the links below.
If you have a low bandwidth connection and only want to see one demo, I suggest Flight \/ since it's the most dynamic example of the technique described below \/.
Corner
AVI >> (60s, 0.6MB)
QuickTime >> (60s, 1.7MB)
MPEG-1 >> (30s, 3.7MB)
Composition
AVI >> (60s, 1.0MB)
QuickTime >> (60s, 2.5MB)
MPEG-1 >> (30s, 5.5MB)
Lara
AVI >> (60s, 1.4MB)
QuickTime >> (60s, 1.8MB)
MPEG-1 >> (30s, 5.0MB)
Bobs
AVI >> (60s, 8.8MB)
QuickTime >> (60s, 9.9MB)
MPEG-1 >> (30s, 3.8MB)
Horizon
AVI >> (60s, 4.3MB)
QuickTime >> (60s, 6.5MB)
MPEG-1 >> (30s, 3.0MB)
Conservation
AVI >> (60s, 1.4MB)
QuickTime >> (60s, 3.1MB)
MPEG-1 >> (30s, 2.9MB)
Flight
AVI >> (60s, 2.7MB)
QuickTime >> (60s, 3.1MB)
MPEG-1 >> (30s, 4.1MB)
The LCD panel displayed in the videos is the Sharp LQ057Q3DC02 (5.7" screen).
Problems playing videos
- AVI
If QuickTime or Windows Media Player correctly launches, but the player complains about not being able to play the file, you may not have the DivX codec installed. Follow the www.divx.com links [ win >>* | mac >>* | linux >>* ]. You may want to bookmark this page first. If you don't want to install the DivX codec, you may want to try the other formats (QuickTime or MPEG-1). - QuickTime
Make sure your QuickTime player is version 5 or higher. If not, you may download the QuickTime player at Apple's site >>*.
Technique
The algorithm for the recombination of image flows is based on propagator matrices inspired by my studies in dynamical systems. These propagator matrices govern the probabilistic evolution of the system, and are generated by computing "energy differences" between image sequences. There are three significant aspects to this technique.
1. Image composition. Formal aspects within the image, such as position, color, and light, are used to determine the "energy" of an image. The choice of factors which contribute to an image's energy would seem arbitrary, but many choices yield nonsensical evolutions, whereas certain other choices would make "sense", that is, they would have a "direction" in time. (The correlation here between "sense" and "direction" (sens) is indebted to The Logic of Sense by Gilles Deleuze.) For example, linking the positions of the eyes, nose, and mouth in the evolution of a portrait appeals to our intuition of spatial locality. Similarly, the height of a drooping leaf yields a direction in time, since we know cut flowers tend to wither. Other energy parametrizations give more fanciful associations. Linking the positions of a boat to reflected sunlight on the horizon creates a dance between the two. Applying the atomic energy model for electrons to a flock of birds produces "quantum seagulls". Thus, the choice of energy parameters for an image sequence determines the "expression" of the resulting image flow.
2. Image trajectory. To develop a trajectory for the image flow, I use the technique of Markov chains. In this method, propagator matrices determine successive states in a system. For this project, matrix elements are computed using the energy parameters discussed in item 1 above. Moreover, I use the image sequence of the source video footage as a "model trajectory". This choice is key in creating a "sensible" evolution.
3. Beat frequency. Finally, each subject had an appropriate "beat frequency". For lobster bobs in the ocean, the beat frequency matched the rise and fall of the ocean waves. For a bird in flight, I chose twice the frequency of the beating of the wings. Other subjects also had natural frequencies, discovered through intuition and experimentation.
Verso
The title Turn All Things is taken from a passage Albrecht Dürer wrote after he had woken from an apocalyptic nightmare.
In the year 1525, between Wednesday and Thursday after Whitsunday during the night, I saw this appearance in my sleep, how many great waters fell from heaven. The first struck the earth about four miles away from me with a terrific force, with tremendous clamour and clash, drowning the whole land. ... I was so frightened when I awoke that my whole body trembled and for a long while I could not come to myself. So when I arose in the morning, I painted above here >>* as I had seen it. God turn all things to the best.
I came upon this passage after my own nightmare, so it had an unusual force. The last line struck me as a peculiar prayer, and I suppose its peculiarity may be due to its translation from German and 500 years past. In any case, I thought it would make a good title for this artwork, though for my purposes I omitted explicit reference to external agency and eschatology.
Awards
November 2002: Honorary Mention in Vida 5.0 competition >>.