Sound visualisation

Objective of the tutorial

In this tutorial we will see how to employ new StarField class to create simple sound visualisation.

Ok, so you are fairly good AS3 coder, you have read numerous computeSpectrum () tutorials (such as this one), and you followed three other tutorials about StarField class. This means, you are ready to make some cool stuff :) To get straight to business, we will start from following code stub:

package {
	import flash.display.*
	import flash.events.*
	import flash.filters.*
	import flash.media.*
	import flash.net.*

	import sandy.core.*
	import sandy.core.data.*
	import sandy.core.scenegraph.*
	import sandy.primitive.*
	import sandy.materials.*

	public class Spectrum extends Sprite {
		// screen size
		private var w = 400;
		private var h = 300;
		// sandy scene
		private var scene:Scene3D;
		// starfield
		private var sf:StarField;
		// sound
		private var s:Sound;


		public function Spectrum () {
			// set up scene
			scene = new Scene3D ("", this, new Camera3D (w, h), new Group (""))
			// set up sky
			sf = new StarField (); scene.root.addChild (sf);
			// subscribe to Event.ENTER_FRAME
			addEventListener (Event.ENTER_FRAME, enterFrameHandler);
			// set up some sound
			s = new Sound (); s.addEventListener (Event.COMPLETE, completeHandler);
			s.load (new URLRequest ("http://www.helpexamples.com/flash/sound/song3.mp3"),
				new SoundLoaderContext (1000, true));
		}

		private function completeHandler (event:Event):void {
			// start & loop playback
			var song:SoundChannel = s.play (); song.addEventListener (Event.SOUND_COMPLETE, completeHandler);
		}


		private function enterFrameHandler (event:Event):void {
			// render the scene
			scene.render ();
		}
	}
}

This is document class for empty 400×300 FLA that creates empty StarField object (named “sf”), adds it to the scene and then renders it every frame. Additionally, some mp3 is loaded and played in endless loop.

The concept

Here is a rough plan of what we will do:

We will take each band of sound spectrum and display it as a circle of “stars” with radius that depends on band values. This is simpest 3D thing I came up with, feel free to use your imagination and alter it :)

The implementation

Ok, so 1st thing to do is make a symbol for “star”, as described in 3rd tutorial. I am not really creative, so I will just use a bunch of meaningless symbols following each other on timeline and name the class “Symbol1”.

2nd thing to do is to add them to our StarField object in document class constructor:

// create 8 bands with 10 "stars" in each
for (var i:int = 0; i < 80; i++) {
	var s1:Symbol1 = new Symbol1;
	s1.blendMode = BlendMode.ADD;
	s1.gotoAndPlay (1 + Math.round (Math.random () * (s1.totalFrames - 1)));
	sf.starSprites [i] = s1; sf.stars [i] = new Vertex ();
}

3rd thing to do is to grab sound spectrum in enterFrameHandler:

// buffer
private var ba:ByteArray = new ByteArray ();
// (in enterFrameHandler)
SoundMixer.computeSpectrum (ba, true);

Finally, we have to actually arrange “stars” according to a plan above. This is that part again where you play with numbers until you like it:

// (in enterFrameHandler)
for (var i:int = 0; i < 8; i++)
{
	var j:int, r:Number = 0, v:Vertex;
	for (j = 0; j < 512 / 8; j++)
	{
		// aggregate spectrum values
		r += Math.abs (ba.readFloat ()) * 3;
	}
 
	for (j = 0; j < 10; j++)
	{
		v = sf.stars [i * 10 + j];
		v.x = 100 * (i - 4) / 4;
		v.y = (r+ 30 + 4e-3 * v.x * v.x) * Math.sin (j * 0.2 * Math.PI);
		v.z = (r+ 30 + 4e-3 * v.x * v.x) * Math.cos (j * 0.2 * Math.PI);
	}
}
// rotate StarField around
sf.rotateY ++; sf.rotateZ ++;

Basically, this already works as we planned.

Final touches

Let's put whole thing on fire, shall we? StarField class provides support for rendering events that might be handy for this kind of job. Using these events, you can modify underlying BitmapData object and/or prevent/force its clearing after event processing. What we will do here is 1) prevent BitmapData object clearing, 2) blur it instead, and 3) transform pixel colors towards red using ColorTransform:

// point, filter and color transformator
private var p:Point = new Point ();
private var f:BlurFilter = new BlurFilter (); 
private var ct:ColorTransform = new ColorTransform (0.9, 0.7, 0.5);
 
private function beforeRenderHandler (event:StarFieldRenderEvent):void {
	// do not clear bitmap data from last frame
	event.clear = false;
	// blur and fade instead
	event.bitmapData.applyFilter (event.bitmapData, event.bitmapData.rect, p, f);
	event.bitmapData.colorTransform (event.bitmapData.rect, ct);
}
 
// (in constructor)
sf.addEventListener (StarFieldRenderEvent.BEFORE_RENDER, beforeRenderHandler);

The result

If you actually followed the tutorial, you should have ended up with the code like this:

package {
	import flash.display.*
	import flash.events.*
	import flash.filters.*
	import flash.geom.*
	import flash.media.*
	import flash.net.*
	import flash.utils.*

	import sandy.core.*
	import sandy.core.data.*
	import sandy.core.scenegraph.*
	import sandy.events.*
	import sandy.primitive.*
	import sandy.materials.*

	public class Spectrum extends Sprite {
		// screen size
		private var w = 400;
		private var h = 300;
		// sandy scene
		private var scene:Scene3D;
		// starfield
		private var sf:StarField;
		// sound
		private var s:Sound;
		// buffer
		private var ba:ByteArray = new ByteArray ();
		// point, filter and color transformator
		private var p:Point = new Point ();
		private var f:BlurFilter = new BlurFilter (); 
		private var ct:ColorTransform = new ColorTransform (0.9, 0.7, 0.5);


		public function Spectrum () {
			// set up scene
			scene = new Scene3D ("", this, new Camera3D (w, h), new Group (""))
			// set up sky
			sf = new StarField (); scene.root.addChild (sf);
			// subscribe to Event.ENTER_FRAME
			addEventListener (Event.ENTER_FRAME, enterFrameHandler);
			// set up some sound
			s = new Sound (); s.addEventListener (Event.COMPLETE, completeHandler);
			s.load (new URLRequest ("http://www.helpexamples.com/flash/sound/song3.mp3"),
				new SoundLoaderContext (1000, true));
			// create 8 bands with 10 "stars" in each
			for (var i:int = 0; i < 80; i++) {
				var s1:Symbol1 = new Symbol1;
				s1.blendMode = BlendMode.ADD;
				s1.gotoAndPlay (1 + Math.round (Math.random () * (s1.totalFrames - 1)));
				sf.starSprites [i] = s1; sf.stars [i] = new Vertex ();
			}
			// subscribe to StarField rendering event
			sf.addEventListener (StarFieldRenderEvent.BEFORE_RENDER, beforeRenderHandler);
		}

		private function completeHandler (event:Event):void {
			// start & loop playback
			var song:SoundChannel = s.play (); song.addEventListener (Event.SOUND_COMPLETE, completeHandler);
		}


		private function enterFrameHandler (event:Event):void {
			// get sound data
			SoundMixer.computeSpectrum (ba, true);
			// arrange "stars" in 8 circular bands
			for (var i:int = 0; i < 8; i++)
			{
				var j:int, r:Number = 0, v:Vertex;
				for (j = 0; j < 512 / 8; j++)
				{
					// aggregate spectrum values
					r += Math.abs (ba.readFloat ()) * 3;
				}

				for (j = 0; j < 10; j++)
				{
					v = sf.stars [i * 10 + j];
					v.x = 100 * (i - 4) / 4;
					v.y = (r+ 30 + 4e-3 * v.x * v.x) * Math.sin (j * 0.2 * Math.PI);
					v.z = (r+ 30 + 4e-3 * v.x * v.x) * Math.cos (j * 0.2 * Math.PI);
				}
			}
			// rotate StarField around
			sf.rotateY ++; sf.rotateZ ++;
			// render the scene
			scene.render ();
		}

		private function beforeRenderHandler (event:StarFieldRenderEvent):void {
			// do not clear bitmap data from last frame
			event.clear = false;
			// blur and fade instead
			event.bitmapData.applyFilter (event.bitmapData, event.bitmapData.rect, p, f);
			event.bitmapData.colorTransform (event.bitmapData.rect, ct);
		}
	}
}