• Back to all articles

Video and Audio Streaming with Flash and Open Source Tools

Flash has always been developed and used for multimedia purposes, but until version 6, the possibilities for audio streaming were limited, and there was no video support. With versions 6 and 7, Macromedia introduced video support and a new file format to support various ways of streaming. This article covers only a streaming variant called "progressive download", which does not need server support.

FLV Streams

Streaming is built upon a new file format called FLV, which separates the streamable content and the Flash movie. The results are a very compact Flash movie acting as a multimedia player and a container for streamable content, from which the Flash movie loads a stream on demand.

A single FLV stream contains one audio stream and one video stream. Flash supports uncompressed sound and various compressed formats like MP3 and ADPCM, as well as the proprietary Nellymoser audio codec. With Flash version 6, Macromedia also introduced video support. In version 6, only the Sorenson H.263 video codec was supported. This is a slightly modified version of the open H.263 standard. The latest Flash version, 7, introduced a second video format called "Screen Video", which is a simple, lossless video format, especially developed for screen capturing.

Converting and Creating Content

One method for creating FLV streams is converting existing audio and video content with FFmpeg. FFmpeg is mature and excellent software for converting audio and video to and from various formats.

Converting a video can be done simply with:

 
$ ffmpeg -i infile.[avi|mpeg] stream.flv

FFmpeg uses the Sorenson H.263 video format for encoding video data. There is no support for the Screen Video format at this time. While the Screen Video format is mainly useful for screen capturing applications, Sorenson's H.263 is a multipurpose video codec with good compression rates, suitable especially for encoding motion pictures.

Another project dealing with FLV streams is called libflv. While FFmpeg is a general audio and video converting suite, libflv is focused on working with FLV streams. The project is still in a very early stage, but is able to encode videos in the Screen Video format and allows simple FLV stream manipulations like (de-)multiplexing of audio and video streams. A simple GTK-based screen capturing application can be found in the example directory.

Building a Simple Multimedia Player

After we've created some streamable content, a Flash multimedia player is needed. One huge advantage of a Flash-based player over other plugin-based multimedia players is that there are no constraints about its look and how it is integrated in your site's design.

MING is an Open Source library which is able to create Flash files with almost all recent Flash features, including Action Script and sound and video support. It also has language bindings for a bunch of script and programming languages. The examples presented in this article are written in PHP4. Porting them to other supported languages like C/C++, Java, Python, or Perl should be trivial.

To run the following example, a current CVS snapshot of MING is needed. It is available either via SourceForge's anonymous CVS service or pre-packaged at http://klaus.geekserver.net/ming/.

First, we create a new movie instance and set the dimensions and background color:

	ming_useswfversion(7);
	$movie=new SWFMovie(7);
	$movie->setDimension($width, $height);
	$movie->Background($r,$g,$b);

The new Flash movie can now be filled with Flash objects called characters. For the multimedia player example, we create a video canvas object and add it to the movie. The add() method takes a character, inserts it to the current frame, and returns a handle to the object. This can be used to move, rotate, resize, or remove an object. If the object is going to be used with ActionScript, a name can be assigned to it.

	$stream = new SWFVideoStream();
	$stream->setDimension($width, $height);
	$item = $movie->add($stream);
	$item->moveTo($x, $y);
	$item->setname("video");

The SWFVideoStream() constructor can also take an FLV file as an argument. In this case, the video stream will be embedded to the Flash file. However, this approach has some drawbacks. First, the resulting Flash movie will become at least as big as the stream. Also, the stream's frame rate must not exceed the Flash movie's frame rate, and each Flash file is limited to 16,000 frames, which means that the embedded stream can contain at most 16,000 frames.

A multimedia player application should be able to load and play streams dynamically. Therefore, the SWFVideoStream() constructor is called with no arguments. Thus, only an empty video canvas will be created, which will be controlled by the following ActionScript code:

	connection = new NetConnection(); 
	connection.connect(null);
	stream = new NetStream(connection);
	video.attachVideo(stream); 	
	stream.setBufferTime(10); 
	stream.play('http://localhost/mystream.flv');

The ActionScript first creates a pseudo connection by passing null to the connect() method of the NetConnection object. In contrast, a real connection to a Macromedia streaming server can be made by passing a valid URL to the method. Having a NetConnection instance, a new NetStream object can be created and attached to the empty video canvas. This object handles streaming and provides methods for controlling the stream. The above example loads an FLV stream from the local Web server with a downloadbuffer of 10 seconds. The ActionScript code can be compiled and added to the movie with:

	$action = new SWFAction($action_string);
	$movie->add($action);

Until now, the Flash movie just loads and plays a certain FLV stream. To control its behavior, a simple user interface consisting of some buttons and a seek-slider is needed. Flash has its own compressed lossless bitmap format called DBL. MING provides a small utility called png2dbl to convert PNG images to DBL. Such images are used for the player's control buttons:

	$button = new SWFButton();
	$flags = (SWFBUTTON_UP | SWFBUTTON_HIT | SWFBUTTON_OVER | SWFBUTTON_DOWN);
  	$button->addShape(ImageShape("images/pause.dbl"), $flags);
        
	$action = new SWFAction("stream.pause();");
 	$button->addAction($action, SWFBUTTON_MOUSEDOWN);
	
	$button_ref = $movie->add($button);
	$button_ref->moveTo($x, $y);

The above example creates a pause button for the multimedia player. An interactive button is created in two steps. First, its look has to be defined by adding shapes for certain mouse events. In Flash, a shape is the basic representation for graphic objects. For each mouse event, a differently-shaped object can be assigned to the button. In the above example, the button always looks the same. In the second step, the button's action can be defined by assigning ActionScript to a special event.

One drawback of using progressive download streaming without server support is that there is no possibility of getting the stream's total length. Therefore, the seek-slider's functionality is limited to seeking within the already-loaded parts of the stream.

The dragable part of the seek-slider is realized as a movie-clip object. A movie-clip runs as an independent movie in the Flash movie. It has an independent timeline, can handle scripts, and handles external events itself.

	$mc = new SWFSprite();
	$shape = new SWFShape();
        $shape->setLine(4,25,0,0,128);
        $shape->movePenTo(0, 5);
        $shape->drawLineTo(0, 10);
        $mc->add($shape);
	$mc->nextFrame();
	
	$slider = $movie->add($mc);
	$slider->moveTo($xMin, $y);

A movie clip (SWFSprite) has methods similar to those of a movie object. The add() method inserts a Flash object to the current frame, and nextFrame() finishes the current frame and creates a new one. The movie clip is also a normal Flash object which can be added to a movie and placed on the stage. The functionality of the seek-slider is defined by three small scripts. The first two actions make the movie-clip dragable:

	$a = new SWFAction("startDrag(this, $xMin, $y, $xMax, $y, 1); drag = true;");
        $slider->addAction($a, SWFACTION_MOUSEDOWN);
        
        $a = new SWFAction("stopDrag(); drag=flase;");
        $slider->addAction($a, SWFACTION_MOUSEUP);

The third, lengthier script sets the stream position, depending on the slider's x-position if the slider is actually moved by the user, or sets the slider's x-position, depending on the stream's current time:

	// width in px
	width = xMax - xMin;
	
	paused = false;
        if(drag) { 
		// pause stream while seeking
        	_global.stream.pause(true);
                paused = true;
			
                x = _root._xmouse - xMin;
                seekTo = (_global.streamLen / width) * x;
                _global.stream.seek(seekTo);
       	}
       	else {
		pos = (_global.stream.time * (width / _global.streamLen)) + xMin; 
                this._x = pos; 
                this._y = y; 
      	}
      
      	// restart paused stream
   	if(paused)
	{
               _global.stream.pause(false); 
        }

This script is assigned to the $slider handle with the SWFACTION_ENTERFRAME event.

After we've added all the elements to the Flash movie, the first frame has to be closed with the nextFrame() call. Since we don't need another frame, the movie can also be finished:

	$movie->nextFrame();
        $movie->save("FLVPlayer.swf");

The resulting multimedia player (requires Flash 7)

Example Video: Copyright by Thilo Weigel, University of Freiburg

Conclusion

With Flash, it is easy to create a lightweight, fully customized, embedded video and audio player. There are powerful Open Source tools available for creating content and creating Flash movies. This article introduced the basic concepts of Flash streaming and working with MING. The mediaplayer presented here provides only the most basic features and was only intended as a simple example. It can be extended in many ways, which is left to the reader.