e-Rite
2016
e-Rite interprets Stravinsky's Rite of Spring visually and algorithmically.
How can the underpinning structures of Stravinsky's music be perceived? How can auditory events be transformed into audiovisual experiences? e-Rite attempts to bring a new dimension to the Rite of Spring by translating its compositional style and orchestrating power to visual forms of abstraction. Geometrical algorithms and generative processes offer an immersive experience that serves the dramaturgical richness of Stravinsky’s masterpiece.
Dramaturgy and general approach
Music visualization and the hereby highlighting of the patterns behind music making is not restrained to a mathematical problem-solving situation. In this project, the ultimate goal is to provide a new interpretation of the original work, a new mise-en-scène that serves the original composition and offers a new layer of musical perception.
Based on this principle I have taken into consideration the original context of the Rite of Spring (1913), which was presented in Paris by Sergei Diaghilev's Ballets Russes in a time that Fauvism and Expressionism were the main tendencies in the visual arts and accordingly in stage design. As a result, and itself being a ballet, I am preserving the movement-oriented quality of the work and I am "celebrating" the advent of the spring with the vivid colors of Russian folklore, that the context represents.
My first approach has been to explore with Processing visual patterns that would help me evoke the above imagery, the rising of the complexity of nature and the striking "rough" sonority taking place at the second movement. Moreover, I have looked for colors that would be as close as possible to the ones found in Russian folkloric ornaments, concluding at the following samples:
#ff3333 #266c3e #003333 #ddc602 #ffcc99
An equally significant component of the project has been to find formulas that would generate visual effects close to my aesthetical vision. More on this will be explained later.
Score reading
The reading of the score is necessary in order to situate/extract the visual characteristics of the work and generate the aforementioned effects at the appropriate time. Especially in relation to the rising of nature and the emergence of its phenomena, instrumentation and the gradual introducing of soli vs. instrumental groups comprise a significant component of the work's dramaturgy.
As the following excerpt reveals, both the melodic line of the instruments as well as their relation to each other need to be first identified. Serving as an instruction sheet for turning visual patterns on and off, the musical score becomes a central component of the visualization apparatus.
As the music goes on, the orchestrational density becomes more and more complex. This gradual change is mirrored by the created visual effects.
e-Rite is based on the first movement (Introduction) and part of the second movement (Dances of the Young Girls) of the ballet.
Visual effects / objects
This section attempts a more analytical account of the visual language in use. The introduced visual objects are split in three main categories, for which representative examples are provided:
A. The rising of nature and the use of elliptical forms
Constant listenings of the work combined with the idea of natural progression and the advent of spring invoked to me a pattern of continuous movement that "had to" be visually expressed. My sketches on paper (s. above) urged me very often to draw elliptical forms (ShapeA), that would develop in time and change their angle of progress, as melody would go on.
It was this observation that motivated me to try and mirror the frequency of the played music to a parameter that would change the angle of motion. Extracting automatically frequencies of specific instruments was hard to do, however this approach worked well at the beginning of the first section, since orchestration at that point is less dense and the solo instruments can be easily deciphered.
The following coding excerpt reveals the mechanism behind the elliptical motions. The resulted effect resembles brownian motions and is activated manually, by pressing key(s) at specific cues in time:
import beads.*;
// sound variables
AudioContext ac;
Gain g;
PowerSpectrum ps;
Frequency f;
String sourceFile;
SamplePlayer player;
float inputFrequency;
float mappedFrequency;
float prevFrequency;
PeakDetector beatDetector;
float beat;
// visual variables
ShapeA shA1, shA2, shA3, shA4, shA5;
boolean shA1pressed, shA2pressed, shA3pressed, shA4pressed, shA5pressed;
ArrayList<ShapeB> roses;
boolean shBpressed;
int shBpopulation;
int alpha;
int counter;
void setup() {
/* [code omitted] */
/* prerequisites for getting main frequency */
ShortFrameSegmenter sfs = new ShortFrameSegmenter(ac);
//sfs.setChunkSize(2048); // how large is each chunk?
sfs.setHopSize(441);
sfs.addInput(ac.out);
FFT fft = new FFT();
sfs.addListener(fft);
ps = new PowerSpectrum();
fft.addListener(ps);
f = new Frequency(44100.0f);
ps.addListener(f);
ac.out.addDependent(sfs);
/* end of prerequistes */
/* [code omitted] */
smooth();
noStroke();
shA1 = new ShapeA(3); // random factor affecting angle
shA2 = new ShapeA(6);
shA3 = new ShapeA(8);
shA4 = new ShapeA(3);
shA5 = new ShapeA(10);
shBpopulation = 8;
roses = new ArrayList<ShapeB>();
for (int i = 0; i < shBpopulation; i++) {
roses.add(new ShapeB(int(random(1,3))/float(int(random(4,9)))));
}
shA1pressed = false;
shA2pressed = false;
shA3pressed = false;
shA4pressed = false;
shA5pressed = false;
shBpressed = false;
alpha = 220;
counter = 0;
ac.start();
}
void draw() {
fill(255,204,153,alpha);
rect(0-10,0-10,width+20,height+20);
if (frameCount > 30) {
Float returnedFrequency = f.getFeatures(); // This is the main frequency
if(returnedFrequency != null){
inputFrequency = returnedFrequency;
mappedFrequency = map(inputFrequency,20,2000,0,20);
}
if (shA1pressed) {
shA1.angle();
shA1.display(shA1.angle,39,59,54,4);
}
if (shA2pressed) {
shA2.angle();
shA2.display(shA2.angle,0,50,50,4);
}
if (shA3pressed) {
shA3.angle();
shA3.display(shA3.angle,0,50,50,4);
}
if (shA4pressed) {
shA4.angle();
shA4.display(shA4.angle,0,65,52,4.5);
}
if (shA5pressed) {
shA5.angle();
shA5.display(shA5.angle,39,59,54,3);
}
if (shBpressed) {
for (int i = 0; i < shBpopulation; i++) {
ShapeB rose = roses.get(i);
rose.display(0,100,100,3);
}
}
prevFrequency = inputFrequency;
/* [code omitted] */
}if (counter > 1) {
alpha = int(alpha - 0.00001/1.3107);
}
if (alpha <25) {alpha = 25;};
//saveFrame("tga/frame-######.tga");
}
void keyPressed() {
if (key == '1') {
if (shA1pressed == false) {
shA1pressed = true;
counter++;
} else {
shA1pressed = false;
shA1.newPosition();
}
}
if (key == '2') {
if (shA2pressed == false) {
shA2pressed = true;
} else {
shA2pressed = false;
shA2.newPosition();
}
}
if (key == '3') {
if (shA3pressed == false) {
shA3pressed = true;
} else {
shA3pressed = false;
shA3.newPosition();
}
}
if (key == '4') {
if (shA4pressed == false) {
shA4pressed = true;
} else {
shA4pressed = false;
shA4.newPosition();
}
}
if (key == '5') {
if (shA5pressed == false) {
shA5pressed = true;
} else {
shA5pressed = false;
shA5.newPosition();
}
}
if (key == 'r') {
if (shBpressed == false) {
shBpressed = true;
} else {
shBpressed = false; // shapes B vanish from screen..
roses.clear();
shBpopulation = int(random(8,40));
for (int i = 0; i < shBpopulation; i++) {
roses.add(new ShapeB(int(random(1,3))/float(int(random(4,9)))));
}
for (int i = 0; i < shBpopulation; i++) {
ShapeB rose = roses.get(i);
rose.newPosition();
}
}
}
}The elliptical/brownian ShapeA objects belong to the following class:
class ShapeA {
float angleCount = 7;
float stepSize = 3.3;
int i = 0;
float angle = radians(270);
float r;
float x,y;
ArrayList<PVector> points = new ArrayList<PVector>();
ShapeA(float _r) {
// set position of first point
points.add(new PVector(random(width), random(height)));
r = _r;
x = 0;
y = 0;
}
float angle() {
if (prevFrequency <= inputFrequency) {
angle = angle - radians(mappedFrequency*random(r));
} else {
angle = angle + radians(mappedFrequency*random(r));
}
return angle;
} void display(float _angle, int r_color, int g_color, int b_color, float d) {
PVector point = points.get(i);
fill(r_color,g_color,b_color);
ellipse(point.x, point.y, d, d);
PVector prevPoint = points.get(points.size()-1);
x = prevPoint.x + cos(_angle) * stepSize;
y = prevPoint.y + sin(_angle) * stepSize;
// add a new point
points.add(new PVector(x, y));
/* [code omitted] */
i++;
}
void newPosition() {
points.clear();
i = 0;
x = random(5,width-5);
y = random(5,height-5);
points.add(new PVector(x, y));
}
}B. The "algorithmic flowers"
In sections of the first movement, where orchestration becomes more dense or sound intensity increases, the introduction of new visual elements seems unavoidable. For this purpose I have used ShapeB, which initially "emerged" during analog paper sketches with shapes like flowers popping up, as spring was gaining prominence. These shapes needed a mathematical formula of their own, and therefore the simple rose curve equations of x = cos(k*t) * sin(t) and y = cos(k*t) * cos(t) were used for the desired effect:
class ShapeB {
int i = 0;
float x,y,k;
ArrayList<PVector> points = new ArrayList<PVector>();
ShapeB(float _k) {
// set position of first point
points.add(new PVector(random(20,width-20), random(20,height-20)));
x = 0;
y = 0;
k = _k;
} void display(int r_color, int g_color, int b_color,
float d) {
PVector point = points.get(i);
fill(r_color,g_color,b_color);
ellipse(point.x, point.y, d, d);
PVector prevPoint = points.get(points.size()-1);
float t = frameCount/5.0;
x = prevPoint.x + cos(k*t) * sin(t) * beat;
y = prevPoint.y + cos(k*t) * cos(t) * beat;
// add a new point
points.add(new PVector(x, y));
i++;
}
void newPosition() {
points.clear();
i = 0;
x = random(5,width-5);
y = random(5,height-5);
points.add(new PVector(x, y));
}
}In order to make these shapes understandable, some examples are provided here. The following image depicts both ShapeA on the left and various instances of ShapeB on the right, as they first appear on the screen:
These shapes increase gradually in quantity, as we get closer to the end of the first movement:
C. The sonorous power of the second movement
The second movement of the Rite of String has a prevailing tone, which constitutes the soundmark of the whole piece. With its powerful new aesthetics, its rigid temporal structure and its abrupt changes of sound colors, this section calls for a dramatic change in relation to the more calming and fluent first part of the work. This dramaturgy affords new visual effects and clear lines, tailored to emphasize the strong beats and rhythmical patterns of the music, and designed with vivid colors to catch attention. New "tree" constellations appear, as loudness reaches soundscapes beyond predefined thresholds:
The Processing code reveals the mechanisms behind this striking effect, which is complemented by flowers of the previous section (in new colors) as well as new ornaments, manually enacted, following the musical score:
//libraries
import beads.*;
//sound
AudioContext ac;
Gain g;
PowerSpectrum ps;
Frequency f;
String sourceFile;
SamplePlayer player;
PeakDetector beatDetector;
float beat;
// visuals
PGraphics layer0; // layer for tree and roses
// tree
float dotSize = 9;
float angleOffsetA;
float angleOffsetB;
int counter;
// roses
ArrayList<ShapeB> roses;
boolean shBpressed;
int shBpopulation;
int c;
// ornaments
boolean pattern;
ArrayList<ShapeC> ornaments;
int y;
void setup() {
size(1280,720);
frameRate(30);
/* [code omitted] */
/* prerequisites for beat detection */
SpectralDifference sd = new SpectralDifference(ac.getSampleRate());
ps.addListener(sd);
beatDetector = new PeakDetector();
sd.addListener(beatDetector);
beatDetector.setThreshold(0.1); // 0.006
beatDetector.setAlpha(.9f);
beatDetector.addMessageListener
(
new Bead()
{
protected void messageReceived(Bead b)
{
beat = 10.0;
}
}
);
ac.out.addDependent(sfs);
/* end of prerequisites */
/* visuals */
// instantiate layer0
layer0 = createGraphics(width,height);
// tree
angleOffsetA = 0;
angleOffsetB = radians(50);
counter = 1;
// roses
shBpressed = false;
shBpopulation = 8;
roses = new ArrayList<ShapeB>();
for (int i = 0; i < shBpopulation; i++) {
roses.add(new ShapeB(int(random(1,3))/float(int(random(4,9)))));
}
c = 0;
// ornaments
pattern = false;
ornaments = new ArrayList<ShapeC>();
// initiate sound
ac.start();
}void draw() {
/* background layer */
layer0.beginDraw();
layer0.noStroke();
layer0.smooth();
// tree
if (frameCount % 4 == 0) {
layer0.fill(0,51,51,10);
layer0.rect(0-10,0-10,width+20,height+20);
layer0.pushMatrix();
layer0.translate(width/2, height);
angleOffsetA = radians(random(-1.5,1.5));
if (beat == 10) {
layer0.fill(255,51,51);
layer0.beginShape();
seed1(dotSize, radians(270), 0, 0);
layer0.endShape();
}
layer0.popMatrix();
beat -= 0.1;
if (beat < 1.0) beat = 2.0;
}
// roses
if (shBpressed) {
for (int i = 0; i < shBpopulation; i++) {
ShapeB rose = roses.get(i);
c = int(random(3));
if (c == 0) rose.display(221,198,2,3); // near yellow
if (c == 1) rose.display(255,204,153,3); // beige
if (c == 2) rose.display(255,255,255,3); // white
}
}
layer0.endDraw();
background(255);
image(layer0,0,0);
/* foreground layer = moving ornaments */
if (pattern) {
for (int i=0; i<ornaments.size(); i++) {
ShapeC ornament = ornaments.get(i);
ornament.display();
}
}
}
void seed1(float dotSize, float angle, float x, float y) {
if (dotSize > 1.0) {
if (counter == 1) {
layer0.curveVertex(x, y); // 1st control point
counter++; // only here counter is used
seed1(dotSize, angle, x, y);
} else {
float r = random(0, 1.0);
if (r > 0.02) {
layer0.curveVertex(x, y);
float newx = x + cos(angle) * dotSize;
float newy = y + sin(angle) * dotSize;
seed1(dotSize * 0.99, angle - angleOffsetA, newx, newy);
} else {
layer0.curveVertex(x, y);
float newx = x + cos(angle);
float newy = y + sin(angle);
seed2(dotSize * 0.99, angle + angleOffsetA, newx, newy);
seed1(dotSize * 0.60, angle + angleOffsetB, newx, newy);
seed2(dotSize * 0.50, angle - angleOffsetB, newx, newy);
}
}
} else {
layer0.curveVertex(x, y); // 2nd (last) control point
}
}
void seed2(float dotSize, float angle, float x, float y) {
if (dotSize > 1.0) {
// Create a random numbers between 0 and 1
float r = random(0, 1.0);
if (r > 0.05) {
layer0.curveVertex(x, y);
float newx = x + cos(angle) * dotSize;
float newy = y + sin(angle) * dotSize;
seed2(dotSize * 0.99, angle + angleOffsetA, newx, newy);
}
else {
layer0.curveVertex(x, y);
float newx = x + cos(angle);
float newy = y + sin(angle);
seed1(dotSize * 0.99, angle + angleOffsetA, newx, newy);
seed2(dotSize * 0.60, angle + angleOffsetB, newx, newy);
seed1(dotSize * 0.50, angle - angleOffsetB, newx, newy);
}
}
}
void keyPressed() {
/* [code omitted] */
if (key == 'p') {
pattern = true;
int chance = int(random(2));
if (chance == 0) y = height;
if (chance == 1) y = -343;
ornaments.add(new ShapeC(int(random(width-35)),y));
}
}The following screenshots complement the presentation of the second movement and reveal the richness and theatricality of the visual representation: