Flash Media Server Uptime Monitoring

I recently needed to implement a way to monitor Flash Media Server for Load Balancing purposes. FMS traffic can be load balanced easily but in this case we wanted to have it stop sending traffic to an FMS server that was down. We've had a couple of times where FMS servers have run out of memory and stopped responding.

The load balancer we are was set up to look for a static file to know if the server is running but since FMS isn't a web server that wasn't going to be very useful. Fortunately FMS comes with a command line tool called fmscheck that can test a stream so it was fairly easy to expand that into an easy way to keep track of the FMS going down.

First I setup a simple shell script with just one line.

/opt/adobe/fms/tools/fmscheck --host localhost --app appname --play testvideo 0 10 --logfile /var/www/html/fmtest.txt

This will basically attempt to play the first 10 seconds of a video called testvideo.flv from the appname application. Notice that the .flv extension is not in the command. The output of the command will be saved in fmstest.txt and if successful will look like this.

#2008-08-23 16:42:20 
#Software: Adobe Flash Media Server Check 1.0.0 r15  
#2008-08-23 
#Fields: date    time        x-event          x-sname
2008-08-23     16:42:20     Connect         rtmp://localhost:1935/appname
2008-08-23     16:42:20     NetConnection.Connect.Success   Connection succeeded.
2008-08-23     16:42:20     Success         StreamCreated
2008-08-23     16:42:20     Success         StreamCreated
2008-08-23     16:42:20     NetStream.Play.Reset     Playing and resetting testvideo.
2008-08-23     16:42:20     NetStream.Play.Start     Started playing testvideo.
2008-08-23     16:42:20     NetStream.Data.Start     -
2008-08-23     16:42:26     NetStream.Play.Stop     Stopped playing testvideo.
2008-08-23     16:42:26     Program complete successful 

If it fails to stream, presumably because FMS is down or a problem with the app then the output looks like this.

#Version 1.0.0 r15  
#2008-08-23 16:30:06 
#Software: Adobe Flash Media Server Check 1.0.0 r15  
#2008-08-23 
#Fields: date    time        x-event          x-sname
2008-08-23     16:30:06     Connect         rtmp://localhost:1935/appname
2008-08-23     16:30:06     NetConnection.Connect.Success   Connection succeeded.
2008-08-23     16:30:06     Success         StreamCreated
2008-08-23     16:30:06     Success         StreamCreated
2008-08-23     16:30:06     NetStream.Play.StreamNotFound   Failed to play 0; stream not found.
2008-08-23     16:30:06     Play fail 

The load balancer wants a simple file that it can check for any change so I added a trivial php file that reads in the file and looks for "Program complete successful" and outputs "FMS Success" if it finds it.

<?php

$file = 'fmtest.txt'; 
$handle = fopen($file, "r"); 
$input = fread($handle, filesize($file)); 
fclose($handle); 

if ( strstr( $input, "Program complete successful" ) )
   print "FMS Success";
else
   print "FMS Failure";

?>

Cron runs the shell script every 2 minutes and the load balancer is checking regularly for the "FMS Success" output. If it sees anything else or the web server doesn't respond then it will stop using this FMS for traffic. This is also useful for watch programs to notify you as soon as an FMS server stops responding.

I've started a new job at Ackerman McQueen as Director of Interactive Technology. While Ackerman McQueen is primarily an ad agency, they also build a lot of Rich Media Sites and that will be my focus. Going forward expect to a lot more articles on Flex and Flash and a shift away from the content management space. CMSs will still be a big part of what I do but I think there will more interesting things to say on the video and RIA side.

Last year Amazon shook up the web development world with EC2, the first Elastic Computing Cloud. Later the addition of S3 for storage and SimpleDB for data fleshed out their platform as a real alternative for web application development. Now Google has launched AppEngine, a virtualized application hosting system that combines some of the features of all the Amazon services for a new take on the hosted application development.

Still an invite-only beta, AppEngine is sure to get a lot of attention just from the Google name alone. I haven't had the opportunity to play with it but just looking through the docs reveals many interesting ideas. The first shock for many developers is that the service is Python only. There will be other languages added soon but I'm already seeing grumbling in the PHP community. Many assume that PHP will be added but I'm not so sure. Python is likely the first choice since it is widely used inside Google but I'd expect the next language to be Ruby because of its similar behind-the-scenes approach to Python. After that, while PHP is possible, I wouldn't be surprised for them to go another direction like Java, a Java scripting language subset or even server side JavaScript.

The second surprise will be the use of CGI as the interface. While most devs associate CGI with old-fashioned, slow apps; it really makes sense in this kind of environment where everything can change between every request.

The biggest shock will be the "No SQL" approach. Data in AppEngine is done through their own datastore, which acts more like an object database. App Engine's infrastructure takes care of all of the distribution, replication and load balancing of data and you use a Django style modeling API. There is also GQL which fulfills the same purpose as SQL for managing data. This is going to be the biggest stumbling block for many developers since 90% of the average developer's work is moving data in and out of SQL Server or MySQL and most people aren't used to being abstracted very far away from that.

It will be interstesting to see how many people jump into Django, the leading Python framework and end up staying there for even non-AppEngine work. This could be the turning point that where Django becomes the next Rails if not even bigger.

Over the last month I've been looking very closely at competing open source content management systems. Typo3 is my goto CMS due to its power and flexibility but no tool can be perfect for every task and Typo3 does have issues with end-user friendliness.

The main ones I've checked out so far are Drupal and Joomla. Both have large communities, local user groups and development books. This post will cover my first impressions of Joomla.

Joomla is most like Typo3 of the two but there are some strong differences too. Where the core organizational principle of Typo3 is the site tree, Joomla has a two layer heirarchy of Categories and Sections. Content is categorized that way and then pages and menus are built based on that. This is an interesting approach and would work well for certain kinds of content especially a news or magazine style site but I do feel the rigid two level approach is a weakness. If this were a more open category tree like Typo3's News category approach it would be a lot more interesting. I would see the category approach as weaker than the Typo3 sitetree approach for high pagecount sites.

One thing I did like about Joola is the interface look and feel. While the category approach isn't completely intuitive, the overall interface is bright and modern looking. Typo3's interface is fairly busy and old-fashioned looking in comparison. While the upcoming Typo3 4.2 interface makes some progress, this is still a weak spot.

As I expected after my experience with commercial CMSs, one area where Typo3 is way ahead is with advanced graphics support. Typo3 is the only CMS I've seen with support for complicated graphic based menus and headers as well as auto sizing images to fit layouts.

There is a large Joomla commmunity, both local and internet-wide. The local community is mostly entreprenuers who are using Joomla as part of another goal. My personal interest would rather see a developer centric community so this is a weak spot for me. The larger community surprised me in that many of the extensions are commercial. This makes sense in terms of a userbase who are using Joomla as a product but isn't what I want after being involved with Typo3's very open-source community.

The templating is pretty flexible and I didn't have a lot of issues with mimicing one of my typo3 sites layout. This wasn't a site using any advanced Typo3 features like Fexible Content Elements but most of my Typo3 sites haven't needed those either. I didn't try to write any extensions for it but from what I saw so far I don't see following up further with Joomla. While it appears to be a capable product, I'm looking for something more different than Typo3 so Joomla isn't offering enough advantages to switch or enough differences to use where Typo3 isn't a good fit.

When the Eclipse PDT project went 1.0 I started experimenting with using it as a PHP development environment and I think I'm ready to commit to it as my standard development environment.

The other piece of the equation is moving to a local php/mysql setup. I'm used to having a development linux box and stuck with that for a while to avoid subtle issues moving work from my local mac to a live linux server. I always custom compile my php on our live servers mostly to optimize them for our projects, especially to fully support Typo3's imaging abilities but I've found it awkward to compile php on Mac OS X. There are several all-in-one Apache/PHP/MySql installers for Mac but I never really liked how they were set up.

Recently I had been looking into the Zend Core package for installing PHP behind IIS with FastCGI and noticed they have a Mac version too. Unlike MAMP and XAMPP, Zend Core is really designed for production use and the MySql install is optional. I tried it out and found it much more like how I like to set up servers and worked fine with my standard MySql install. Even better it comes setup with Zend debugger and was easy to integrate into PDT's debugging mode.

There were two things I needed to do to get it fully operational. First was to run the update script right after install. This is a command line script at /usr/local/Zend/Core/setup/setup. Before I did that I had a problem with the mail function throwing an undefined function error. The second change I did was change it to run apache under my default name and group on OS X so it would be easy to use the htdocs folder as my PDT projects folder.

I've had no trouble developing within complicated apps like Typo3 and Drupal in this setup and am really happy with it. I see people recommend MAMP and XAMPP all the time for Mac PHP developers and I'm surprised I don't see Zend Core come up more.

Yahoo and the Front End Developer.

I've been a big fan of Yahoo's developer site (http://developer.yahoo.com/) Beyond their excellent javascript library, they also have lots of good video content ranging from deep javascript tips to advice on developing for screen readers.

It's really interesting to see how important front end technologies have become again. I started out as a front end developer in the 90s. I was coming from a print background and my skills with Photoshop were more important than coding in a time when sites tended to be static html pages. As the technology improved and site became more sophisticated I began doing more and more backend coding, first with php then later with java and .NET.

In the last couple of years, Ajax and Flash have brought the frontend back into the spotlight. Yahoo dev posted a blog entry about how frontend development has become complicated enough and important enough that they have developed a whole large-scale training process to make sure they have the talent they need. I find it really interesting both because I really like front end development and because it reflects the growing importance of user experience and interface design.

Went to a Joomla User Group yesterday to check it out. I've been aware of Joomla for a while as a leading open source PHP CMS but haven't had a chance to see how it compares with Typo3 or other CMSs I'm familiar with. I thought the user group would be a good chance to get a quick overview.

My first impression was that the interface was friendlier and more modern looking than Typo3's but as I saw more of the system I feel like its approach to managing content is less intuitive. I really like Typo3's strong sense of a page hierarchy. Joomla's user management was less flexible as well.

Most of the people at the user group weren't php developers and I can see the appeal of Joomla for someone who wants to avoid code. In comparison, Typo3 is much more of a developer toolkit. I think this gives it more power and flexibility at the high end though obviously at the cost of complexity and learning curve up front.

Since none of the people were deep php developers I didn't get to see the extension development side. I never build a typo3 site without custom extension work so that will be an important part of any CMS that I add to our toolkit. I was surprised to see that many extensions were commercial, I assumed that most would be GPL like typo3 extensions are.

I'll look at Joomla some more but I think if I want a system with less of a page approach than Typo3 that Drupal might be a better fit for the kind of sites I build.