2003 Christmas Lights and Webcam Stats & Updates

The Xmas/2002 lights had over 100,000 hits (my neighbors did express some concern last year about the flickering lights ;-) but then again, it got Slashdot'ed which resulted in a TON of traffic. The 2003 numbers were up quite a bit ... and then I got Slashdot'ed AGAIN ... but I was a LOT better prepared this year and things never crashed ... although the neighbors were kinda wondering what the heck was going on!

For 2003, the Web Server was a 2.4 GHz Xeon with a GByte of RAM and ISP connectivity was (basically) a T-3. It is running RedHat Linux with the Apache Web Server, and I used Analog to analyze the web logs. In addition to the raw Apache Web Logs, the christmas webcam CGI program has extensive logging that correlated well with the Apache logs. Except in a few cases where noted, all data pertains ONLY to Christmas related stuff rather than from the entire www.komar.org site.

There is a LOTTA info presented here - for those who likes statistics, check out   the webcam generated stats,   and the Slashdot Effect Analysis.   Please email me if you have any suggestions/ideas/thoughts on how this data could be better analyzed/presented.

There is a lot of info from the Analog Report from the Apache Web Server logs - here is some summary info:
    388,851  Total Christmas related Web hits
    135,943  Total Christmas related Page Views
  6,680,000  KBytes of Christmas related text and images transferred
     67,871  Hits on the Christmas Webcam 
     16,069  Hits on the main xmas & christmas lights page
     20,864  Referrals from Slashdot
      1,461  Referrals from Google
      1,006  Referrals from www.uglychristmaslights.com - who said my lights are ugly?!?   ;-)
	865  Referrals from www.weirdlinks.com - and they are not ... too ... weird either!

The Christmas Webcam has extensive logging - here is a summmary (webcam controllable from 1700-2400 MST):
     67,579  Total christmas webcam accesses (darn good correlation with Apache log data)
     38,929  Most webcam accesses in a single day - December 20th (After Slashdot)
      1,425  Most webcam accesses in a single day - December 03rd (Before Slashdot)
      7,567  Most webcam accesses in a single hour - 1700-1800 on December 20th (After Slashdot)
	274  Most webcam accesses in a single hour - 2300-2400 on December 13th (Before Slashdot)
	238  Most webcam accesses in a single minute - 1702 on December 20th (After Slashdot)
	 14  Most webcam accesses in a single minute - 2242 on December 13th (Before Slashdot)
	 11  Most webcam accesses in a single SECOND - 1700:08 on December 20th (Slashdot'ers were waiting!  ;-)
     17,662  Number of time webcam was accessed within the 5 second throttle by more than one web surfer
     21,801  Different/Unique IP addresses that accessed the webcam (2,068 were unresolveable)
	184  Non-proxy hostname that hit it the most - m084-217.nv.iinet.net.au - this guy must like Christmas Lights!  ;-)

	 83  Total different countries that accessed webcam - had very cool geolocation stuff IP address stuff
     48,214  Total hits from United States
      5,373  2nd most popular country - Canada
      2,328  3rd most popular country - Australia

      7,919  Total "ACTION" requests to turn some/all lights ON/OFF
        533  Turn ALL lights OFF (most popular change)
        398  Turn ALL lights ON
      8,720  Total "ACTION" requests to pan/zoom webcam
        362  Pan Right (most popular change)
	229  Pan Left
      1,134  Move webcam to one of the "targets"
	380  Most popular target - "Burnt Out!"
     14,025  Different/unique images served up by xmas_webcam (some images are cached/re-used)

NOTE: Slashdot bills itself as "News for Geeks" and is VERY heavily-read. While the signal-noise ratio of the article comments are sometimes questionable, it can cause a very dramatic jump in web traffic as thousands of people read posted stories and then follow the links to your web site ... RIGHT NOW ... i.e. basically a DDOS - distributed denial of service! ;-) Here is the actual Slashdot article and comments. And what was kinda interesting about this dramatic jump in traffic was that it was not only a test of "digital" stuff like the web server, ISP bandwidth, Perl/CGI code, but also "analog" stuff as there are interfaces to various sensors and the webcam itself, plus you are turning a lotta lights ON & OFF - Slashdot certainly provided one heck of a light show for the neighbors! ;-)

For those interested, here is a more extensive analysis/discussion of the Slashdot Effect on the Christmas Lights/Webcam which includes some raw Analog reports. BTW, another "big blog/news" site is Fark ... story never made the front page of that, but it might have been interesting to see what this coulda done ... i.e. who is more "powerful" - Slashdot or Fark and could either/both be the Grinch that stole Christmas? Maybe that question will be answered at Xmas/2004!

A number of other sites linked/referred to the Christmas Webcam.
Some of these have misc. discussions about it that are kinda fun to read:
   www.majorgeeks.com/vb/showthread.php?s=&threadid=24229 - if the shoes fits, I should wear it, eh?!? ;-)
   pub158.ezboard.com/fphoenixposterszonefrm1.showMessage?topicID=1122.topic - they talk about blowing my fuses!
   www.7thgencivic.com/forums/showthread.php?s=&threadid=129685 - a Honda car discussion forum - go figure?
   www.pdacorps.com/forum/forum_posts.asp?TID=669&PN=1 - control the christmas lights with your PDA!

And here are some other links with minimal discussion:
   www.weirdlinks.com/main.php?id=C14_42_1 - name says it all
   www.xmas4all.net/ - I could use some help translating the guestbook comments
   www.stormwerks.com/linked/ - #!/usr/bin/girl - must be a popular Blog
   www.fridayfishwrap.com/mt/archives/2003_12.html - another popular blog based on the incoming traffic

And various Christmas related sites such as:

The list above is NOT complete - lotta other stuff ... and even some Adult Sites - go figure! 

Here are the Update Messages from the christmas webcam as it played out

2003_11_25 UPDATE: I'm starting to deploy the christmas lights, so the Webcam isn't operational yet (plus the panning motor on the webcam is misbehaving a bit) ... but I've enabled the various X10 sensors as can be seen below - check back around the December 1st to see and control the christmas lights.

2003_12_01 UPDATE: Webcam is up and operational ... still have problems with the panning motor - it won't go the entire range of motion - check back in a few days and hopefully I'll have this fixed.

2003_12_04 UPDATE: With help from my friend Jon, the webcam has the entire range of motion working again - 30 pan and 20 elevation.

2003_12_14 UPDATE: Web Stats for the first two week of December now available and first set of cool webcam pictures from the archives - Email Santa if you see any more.

2003_12_20 1400 UPDATE: Slashdot crowd shows up and POUNDS the site - webserver hanging in there - should be interesting after 1700 when the lights become active! ;-)

2003_12_20 1800 UPDATE: Webcam CGI itself saw 7,567 page views between 1700 and 1800 with a peak of 238 at 1702. Needless to say, the 5-second throttle engaged a LOT ... but there were still 567 unique images generated. Few misc. errors in the web server logs, but it is holding strong. The X10 modules are clicking like mad and got a bit warm; but no blown fuses yet. Not sure what the neighbors are thinking about the light show! Overnight web stats should be interesting (no calls from the ISP ... yet) - check back in a day or so and I'll have some data on the Slashdot Effect ... ;-)

2003_12_20 2400 UPDATE: A quick look at the Analog web stats show the Webcam Perl/CGI itself got 38,929 page views today and a grand total of 210,910 hits on Xmas related stuff for a total of 3.4 GBytes of Xmas traffic today - the vast majority was due to the 16,540 referrals from Slashdot. Total unique images was 3,694 comprising 70,800 KBytes - needless to say, the 5-second throttle engaged a LOT - 15,918 times to be exact ... so I feel sorry for those web surfers who "lost the battle" with other folks and weren't able to "use" the webcam/webcontrol ... but things should calm down a bit, so try again tomorrow. The entire www.komar.org site traffic was over double that, but since I have a Hulk'in Server/ISP, there was no meltdown of the site. I had some neighbors express concern ("what the heck is going on?!?") but no calls from the Homeowners Association, Fire Department, or Utility company ... yet! ;-) i.e. Slashdot was NOT able to stop Christmas! ;-)

2003_12_27 UPDATE: Webcam/Webcontrol will be operational until Midnight MST Dec 31st - instead of college football bowl games, I'll be taking down christmas lights on the 1st! ;-)

2003_12_31_2358: Last Web Surfer (user-0cdv0su.cable.mindspring.com) uses the christmas webcam - deactivated the following morning.

2004_01_02 UPDATE: All the lights down (wasn't that many this year) and packed away for next year.