Network Outage Monitoring
via Christmas Lights

 

 

  Inspired by an old slashdot article and my obsessive need for shorter network outage response times, I decided this year to create a 'christmas lights' frontend to our Network Management System. The results have been very encouraging.
 
 
Background:

I recently moved from NOC operator to engineer and can no longer spend all my time staring at and refreshing network status webpages :(. As such I wanted something that could alert me to outages with little or no effort on my part. I already had a slew of christmas decorations and I remembered this guy writing a lava lamp interface for CVS build monitoring.
  Materials used:

X10 firecracker starter kit (via ebay)
OLD thinkpad running debian sarge
Bottlerocket GPL'ed firecracker control software
9 feet garland
6 extension cords
4 mini christmas trees
3 strings of white lights
3 strings of red lights
1 partridge in a pear tree*
*=lie
Setup:

 - Connect the firecracker wireless interface to the laptop
 - Install bottlerocket (deb: apt-get install bottlerocket)
 - Configure bottlerocket to use the correct serial port
           ln -s /dev/ttyS1 /dev/firecracker
 - Wire all white lights back to X10 transceiver/lamp module
           always A1 - I called about this
 - Wire all red lights back to the other lamp module module
           selectable, in my case A2
 - write bash script to check for alarms & change lights
 - install into cron

WMV     AVI    REAL

(real and AVI quality suck, sorry)
  Operation:

In a nutshell, the thinkpad checks a remote cron-generated file every five minutes for the existence of any alarms. Not wanting to reinvent the wheel for this, I use data from our pre-exisitng and very capable network monitoring system to populate that file. The thinkpad then turns on the corresponding circuit of christmas lights based on what it found in that file - white for all clear and red for trouble.

 
  Pictures: