Misc. Stats from AWDK's 2002 22,000+ Xmas Lights!

There was over 100,000 "web hits" over the month of December on the xmas lights stuff. Does this mean that 100,000+ different people looked at the lights ... or 100,000+ people turned the lights ON and OFF? NOPE! As with any statistics, there are various ways to interpret it, so here's the actual data I have and you can draw your own conclusions - mine was the Xmas lights, and especially the xmas_webcam were pretty popular! ;-)

In addition to the raw Apache Web Logs, the xmas_webcam program had extensive logging that correllated exactly with the Apache logs; except for the evening of December 24th when the site was Slashdotted which means it got POUNDED - see more notes/details below.

Here is some summary information:
   ~130,000  Total xmas lights related Web hits from Apache Logs
     40,928  Hits on the xmas_webcam 
     26,134  Hits on the "main" christmas web page
      1,896  Hits on the Wind Power web page
      7,439  Referrals from Slashdot
        597  Referrals from Der Standard (daily Austrain paper)

     26,846  xmas_webcam "direct" logs (Lost a bunch - see notes)

     13,706  Most logged hits in a single day (December 25th - note: ISP power outage caused machines to be down from 1030-1327)
      1,274  Most logged hits in a single hour (2100-2159 on December 25th - but no data after 1853 on December 24th ...)
         21  Most logged hits in a single SECOND (1849:25 on December 24th)
    1847:03  Slashdot affect starts (20 hits in previous 47 minutes)
                First hit comes from: pcp683341pcs.olathe01.ks.comcast.net 
    1853:09  xmas_webcam logging stopped working (252 hits in previous 6 minutes)
                NOTE: Apache logs indicate much higher rate of "pounding"   ;-)
     13,232  Different IP addresses hit the xmas_webcam
      2,645  Of those, 2,645 were unresolvable
     11,839  Different domains (Difference due to some IP's resolving differently)
      2,097  Unique second-level domains

      2,087  Hits from *.attbi.com (most popular second-level domain)
        497  Hits from *.aol.com (slackers!   ;-)
        458  Hits from *.co.uk (most popular non-USA country code)
	402  Hits from *.com.au & *.net.au (Aussies come in a close second)
	124  Hits from *.colorado.edu (Most popular .edu domain)
        314  Domains with proxy or cache in the name (probably represents more than one person)
	 89  Person who used the xmas_webcam the most - cs6669166-48.houston.rr.com

      5,117  Total requests to turn some/all lights ON/OFF
        768  Requests rejected because lights were being updated too fast - rate greater than one per second!
        661  Turn all lights ON (most popular change)
        535  Turn all lights OFF (2nd most popular change)
	 24  Turn ON Zone-2, turn OFF Zone-4 (least popular)
      7,590  Different/unique Images served up by xmas_webcam

The Apache Web Server keeps logs of every file requested. If someone requests a file/image, it is logged here first, so this is probably the best "real" count of how many folks actually requested files which was 100,327. However, I had the static xmas light images on another server (to reduce the load) and while I don't have that direct data, that was probably another 30,000 or so hits. The actual Apache log data is a bit raw, but a weblog analyzer (Analog in this case) was used to crunch it. In addition to number of hits on certain files, it provides some summary information about where people came from, what sites referred them to the xmas lights (interesting to see who linked to it), and various search words used to find it among other things.

In addition to the web logs, the xmas_webcam program itself has extensive internal logging that kept track of who changed the lights, how many times they did, what lights they toggled, etc. This seemed to work pretty well ... except unforunately, it stopped logging the night I got Slashdotted. Perhaps not too surprising - the Internet connection was a T-1, which despite me putting everying possible on another network/server, is woefully inadaquate for serving up the required dynamic content when you are being pounded. The math is fairly easy: the webcam image is ~30 Kbytes, add in some HTML text, and you'll saturate a 1.544 MegaBITS/second T-1 line with about 4 queries/second. So I lost quite a bit of data for that night when the activity was the busiest ... and the lights were going CRAZY!!!)

Here is the some data from the internal xmas-webcam logging:
    List of IP & Domains sorted by name
    List of IP & Domains sorted by number
    Breakdown by DAY/Hour of the 26,846 xmas_webcam log entries - see note #1
    Breakdown by DAY/Hour of the 7,590 different/unique xmas_webcam images - see note #2

Note #1: This shows activity during the "non-active" hours of 1800-2200; xmas_webcam would simply show a static picture and suggest you come back later.

Note #2: A related data source was the number of different/unique images requested by folks. Since these were "real-time" images, they were only available when xmas_webcam was "active" and would be generated when someone toggled a zone(s) of lights ON/OFF, when someone said "update webcam", or when they first hit the website and there wasn't a "recent" image available.

If you have any questions/comments/suggestions about this data, pls send me an Email.