Thursday, November 17, 2005

Netcraft Reports on Cogent Fiber Cuts

Two separate cuts. [here]


Not again.

FiOS Install: The Bricklin Experience

Dan Bricklin writes up a great description of what I am getting next week: Verizon FiOS. [here]

So, now you know. It's the PPPoE that's going to kill my static connection to the Internet.

Any of you folks who volunteered to host GrabPERF, now would be a great time to step forward.

Call Out To Matt Mullenweg


This morning, I checked my e-mail, and had a note from Matt...

...via PayPal.


Matt, you are one of the true gentlemen out there. Thank you for dropping by and restoring my faith in the community.


Now, about the lack of template control in ;)

Technorati: ,

IceRocket: ,

Wednesday, November 16, 2005

New Home!

The folks at WordPress.Com scraped up one of their tin tickets for me, and I will be migrating appropriate content over to the new site:

See you there!



GrabPERF: Technorati gets a thumbs up as the lights go dim here

Scoble posted that Technorati just released a major update. [here]

Sifry and Kennedy have pulled one more rabbit out of the hat, just before GrabPERF goes off the air. Congrats guys.


Technorati: ,

IceRocket: ,

b2evolution Hosting Recommendations

Anyone have good recommendations for a b2evolution hosting service that will allow me to import my existing database?

Technorati: ,

IceRocket: ,


In order to reduce personal costs for both connectivity and electricity, this Web server and all associated applications will no longer be available as of 14:00 EST (19:00 GMT) November 22, 2005.

I am deeply saddened to have to take this step, but the cost of operating them is becoming a burden to my family.

Thank you all for your support over the years.

I hope to have this blog back up at another location sometime in early December.

GrabPERF and The Newest Industry: LAST CALL

UPDATE: We have a new home:
Stop by and check us out.

UPDATE: The shutdown date has been set.

NOVEMBER 22, 2005

Ok folks, this is it.

Last Call.

We have reached a point where the cost of the cable modem that connects my servers to the Internet has to be slashed.

Without the semi-static IP that I have had for the last 18 months, all of my Web services and applications will have to be terminated.

What does this mean to you?

GrabPERF will be shutting down. will be shutting down. will be shutting down.

And this blog will be shutting down.

I hate to do this, because it has been a great ride, but until we can get back on our feet financially, it's a step that has to be taken.

I will be keeping this site up until November 30, 2005 November 22, 2005. After that, I will be shutting down all access.

This is not a plea for donations. This is a bare cold fact.

Heat the house or host my Web servers.

It's a pretty easy call.

GrabPERF: Blogwise Improves!

Blogwise had a slight server issue earlier this month. Sven, the great guy who runs the service, posted a very open statement of what happened.

I'm very sorry if you tried to access Blogwise over the last four days and got either no service, or an appallingly poor connection. It looks like there was a multiple server failure which, considering I only have a few, was quite damaging.

Events were made worse by my being out of the country at the time (ironically, my first break since Blogwise was started over three years ago). Although I had somebody monitoring the servers, and I knew fairly early that something was up, my lack of access to a computer and awkward time zones made fixing things extremely difficult.

The site should now be running again. If you had assumed Blogwise had closed suddenly, it hasn't. I'm still here and as committed as ever - we just had the most unfortunate event at the worst possible time. This event has highlighted all sorts of vulnerabilities in the service, and has acted as a grim reminder of the need for redundancy and proper support.

It's going to be tough to build a more reliable infrastructure, and to continue developing Blogwise. I don't have the luxury of investors, nor a great deal of revenue from advertising, but the need to resolve this has been made painfully obvious and time and money will be found.

Finally, a thankyou to you all for your continued support, your patience and your understanding. With the site back up, I'm going to enjoy my last three days in Chicago and look forward to getting back into the flow of things this weekend.

- Sven, 12 Nov 2005

On the upside, performance has improved dramatically!

Blogwise performance

Great work Sven...and I feel your pain.

Technorati: , ,

IceRocket: , ,

Tuesday, November 15, 2005

Tom Bihn Bags: The Slacker Manager is Evil

The Slacker Manager just introduced me to Tom Bihn bags. [here]

For any of your louts who forgot my birthday, or if you like GrabPERF and want to show your appreciation: I want what SM got.


Google Analytics — Professional Bloggers Slamming the Service

Looks like Jeremy Wright and Darren Rowse have their own beefs with Google Blahnalytics (Term Courtesy of Jeremy Wright).

Gee Google, these two guys are opinion-shapers. I think maybe you should come out and say that there are issues.

Or say something.


Or are your Overlords at the NSA still pawing over the data and won't let you release it?

Technorati: , , ,

IcreRocket: , , ,

Google Analytics — Help Center Lies!

Help Center: Supporting Google Analytics.

How long does it take to see report data after adding the tracking code?

After you first install the tracking code, it may take several hours for report data to appear in your account. Google Analytics generally updates your reports every hour, but data can take up to 6 hours to appear in your account.


It's now over 24 hours, and no data.

For a company that prides itself on its massive data infrastructure and processing capabilities, the influx of millions of users trying a new service should not be an issue...right?

Maybe they need to follow up with some of their colleagues at the NSA and borrow one their supercomputers. The data is eventually going to end up there anyway.

Technorati: , ,

IcreRocket: , ,

Monday, November 14, 2005

Google Analytics — Apparently, I have no visitors

So, you think that Google Analytics is hot and cool?

So, have you had any visitors yet?

I haven't.

But hey, what about those silly Web server logs that say that I have had many, many visitors?

We are the Google; your data cannot be tracked; therefore, it is irrelevant.

So far, I am not impressed.

And I am not alone. [here]

Technorati: , ,

IcreRocket: , ,

More Photos from the weekend

Just uploaded a bunch more pics to Flickr from this weekend.

Needed a break from writing a report and got these up.


Technorati: ,

IceRocket: ,

Google Analytics — HUMILIATION!

Google, if you are going to launch a Web service, better make sure it's up and available for your customers so that they can evangelize it.

It may be good...when I can get to it.

You all forgot…again!

Wish me a happy birthday, you forgetful louts! ;)

It's 37 this year.

And while you're at it, wish Charles Windsor a happy 57th.

Saturday, November 12, 2005

Weekend at Sweetgrass Winery

The Farmhouse At Sweetgrass Winery

I have posted some pictures of our trip to Sweetgrass Winery on Flickr. I will add more as I process them.

Keith and Constance are great hosts, and we are really looking forward to watching their progress on the winery/distillery up here. Plus, Maine is a gorgeous place.

Technorati: ,

IceRocket: ,

Friday, November 11, 2005

All of your requests will be stored in /dev/null for archive purposes

MySQL 5.0 has a Blackhole Storage Engine. [here]

Cool. And if you make me grumpy, all of your performance data will be stored there! :)

Via: Jeremy Zawodny

Thursday, November 10, 2005

“The legend lives on…”

If you are Canadian, you know what follows after those words.

My friend Kennedy writes to inform me that today is the thirtieth anniversary of the sinking of the S. S. Edmund Fitzgerald. [here]

30 years. Wow.

Out of Touch This Weekend

Ok, if there is an issue with GrabPERF or any of the other services on this server this weekend, it is unlikely that I will be able to fix it.

We are going to see friends in Maine, and we aren't even sure if our mobile phones will work.

I will try and check mail once a day, but don't expect long essays or grand thoughts this weekend.


I want one.


I hate Windows.

Linux Desktops make me nuts.

And Dell...don't take me there.

Ubuntu: Now Uninstalled

Ubuntu, oh how I wish I understood you!

My understanding of why people prefer Windows and OSX is cemented. Evolution keeps crashing, and the wireless card, which worked perfectly at the office, can't see my WRT54G.

Sorry Linux, until you can make it JUST FREAKING WORK, you are sent back to /dev/null.

I don't have time to play; I have work to do.

Coming to you from Ubuntu

Ok, this is the first post from the new Ubuntu-powered work laptop. I partitioned the drive and installed the OS and the only hiccups have been with Evolution, which I think I have beaten into the ground.

Even the ndiswrapper hack worked perfectly for my wireless card. Nice! This has come a long way since the last time I had an installed linux as my desktop.

I like the Gnome interface (I know that will get me flamed), but it is good to be back on the old standby. I am still in dual-boot land, as there is at least one app that I absolutely cannot use from Linux.

Now, if this wasn't a work laptop, I would be really happy...but no one has volunteered a GrabPERF development laptop for me to use, so Dell Hell it is.

I will continue to post my experiences as I use this more.

Wednesday, November 9, 2005

GrabPERF: Update to the Line Graph

Last night I made some code modifications to the Line Graph ("Basic Chart") function in GrabPERF. Previously, the performance graph had been built using data gathered from the rollup table, hourly_site, except for the current hour, which was built from the raw data table, data.

Now, both the Availability and Performance graphs are built from the raw data table.

Also, I began using the Math/Stats PEAR package to calculate the statistical data from the results.

However, the most significant change is that the graph now displays Geometric Mean and 85th Percentile values for each hour. This will give a better sense of the "average" performance, as well as the degree to which there are outliers in the data.

Feedback is appreciated.

Sudden Changes In Global Internet Performance Statistics

Ok, this is a weird set of data, gathered by our friends over at the Internet Traffic Report

Traffic Volume -- 30 Days
Packet Loss -- 30 Days

Any of you black-shirted, BGP-lovin', AS-hoarding packet geeks out there want to comment on what happened on or about Oct 27-29, 2005? I am sure the rest of us would LOVE to know.

Technorati: , , ,

IceRocket: , , ,

Why do I have to dial '1' for long distance?

The question is a simple one: why do I have to dial "1" for long distance from my home phone?

This likely doesn't apply to VoIP users, but I still use POTS at home.

I understand that it is an artifact of the way that the long distance system was originally built. However, my home line is the only phone I use that requires me to dial "1" first.

On both my mobile (T-Mobile) and office phones (MCI VoIP), I dial 10 digits and I am done.

So, why can't the local phone systems figure this one out?

Technorati: , , , ,

IceRocket: , , , ,

Tuesday, November 8, 2005

GrabPERF: Second Agent Upgrade Complete

The second measurement agent is now up and running on Ubuntu!

And the happy news is that it is reporting CONNECTION ERRORS!

Well, ok, not so good news for those of you who are getting them. On the other hand, a gap in the measurement data should indicate a serious problem now.

GrabPERF: Updating Second Agent

In order to avoid doing some work today, I will be upgrading the second GrabPERF Agent from Mandrake 9 (10?) to Ubuntu Breezy 5.10.

Hopefully this will improve the stability of this machine and allow connection error signals from the kernel to be captured correctly by cURL.

Geographic IP database using PERL, PHP and MySQL -- UPDATE: September 16 2008

Updated September 16 2008 to reflect the numerous changes that have resulted since the original article was posted in 2005 - smp

Targeting Web site content to the specific visitors who view the site is a very important marketing advantage. Being able to track incoming visitors by the country that they originate from is an additional item that can assist companies in ensuring that visitors are presented with relevant content. This may seem like a daunting task, but it can be achieved with a high degree of accuracy using publicly accessible data, and Open Source software.

IP to Country Mapping

The idea for IP to Country mapping is one that has started to appear more frequently on the Internet in recent months. All GeoIP systems do warn users that they are not 100% accurate. The accuracy of GeoIP mapping can be affected by things such as large corporate or ISP networks where traffic is routed out a small number of public access points, regardless of the traffic's point of origin.

Making IP Addresses Searchable

The first issue that needs to be addressed is how to determine if an IP Address is in one of the ranges that is defined as originating from a distinct country. The simplest way to range-match IP Addresses is to abandon the dotted-quad notation we are all familiar with, and convert the IP Address to an IP Number.

All IP Addresses can be converted into decimal numbers that fall into a known range between 0 and (2^32)-1 (4294967295). In reality, the range is even smaller than that, as public IP Addresses fall between (IP Number: 0) and (IP Number: 3758096383).

A quick search of the Web showed me that there is a way to create functions to convert IP Addresses to their numerical (IP Number) equivalent and reverse the process.
sub ip2long {
return sprintf("%u",unpack("l*", pack("l*", unpack("N*", inet_aton(shift)))));

sub long2ip {
return inet_ntoa(pack("N*", shift));


if ($start_ip =~ /\d+\.\d+\.\d+\.\d+/) {
## my $ip_address = shift($start_ip);
$long_start = ip2long($start_ip);
## print "$ip_address converts to $ip_number\n";

If you are using PHP in your applications, this conversion process is made even easier by native function calls.
Convert IP Address to IP Number
$ip_number = sprintf("%u",ip2long($ip_address));

IP Address Location Data

Now that we have settled on a format for the IP data to be used in the database, we now have to find IP data that allows us to map IP Addresses to countries. This is easier than it sounds, as this data is centrally held by the 5 regional IP Registries -- ARIN, RIPE, APNIC, LACNIC, and AFRINIC. After poking around in the depths of their Web sites, I found that they actually provide text formatted versions of the allocated and assigned IP ranges that they are responsible for. All of the registries use the same format, which makes parsing the these files a simple process.

As of March, 2005, the distribution of IPV4 networks in the database by registry is:
| registry | num_networks |
| arin | 36073 |
| ripencc | 14813 |
| apnic | 10474 |
| lacnic | 1460 |
| afrinic | 443 |

5 rows in set (0.52 sec)

I chose to use the PERL module WWW::CURL to retrieve the files. You could re-write the application to use LWP or some other method on systems where cURL is not supported, as it is a simple file download over FTP. I update the data once a day, which may at first appear excessive. However, I have seen upwards of 40-50 new rows added to the database in a single day.

Some may ask why I chose to write the downloaded files to a file rather than immediately inserting them into the database. Using this two-step process gives me the ability to manually rollback to an older database if there is a problem retrieving one of the registry files. I have set an arbitrary limit of 75,000 lines for the entire aggregated file; if the file is less than that, the remainder of the process is aborted.

The data retrieved from the registries is in the following format.

Registry Raw Data Format

The fields are all "|" (pipe-character) separated, and are described below.
REGISTRY: apnic,arin,ripencc,lacnic,iana
COUNTRY_CODE: One of 240 unique 2-character country codes or "*"
ADDRESS_TYPE: asn,ipv4,ipv6
ADDRESS: Either the starting IP Address or AS Number or "*"
NUMBER: Number of IPs in range or "1" if ADDRESS_TYPE is "asn"
DATE: Date IP range or AS Number was added to database or "*"
RANGE_TYPE: "allocated" -> borrowed; "assigned" -> owned

Storing the Data in MySQL

To store the data, I created a two-table MySQL database named "ip_registry", using the script below.
Database creation statement for 'ip_registry'
CREATE DATABASE ip_registry;

Table structure for table 'country_code'
CREATE TABLE ip_registry.country_code (
code char(2) default NULL,
country varchar(50) default NULL,
UNIQUE KEY code (code)

Table structure for table 'ip_map'
CREATE TABLE ip_registry.ip_map (
code char(2) default NULL,
registry char(10) default NULL,
ip_from double default NULL,
ip_to double default NULL,
UNIQUE KEY registry (registry,ip_from,ip_to)

As of March 2005 September 2008, the "ip_map" data table for my system runs to 63,263 88,000+ rows. This value will change daily, and may decrease suddenly at times. The registries make an effort to aggregate as many IP networks as possible into the largest possible contiguous block, and this aggregation process will reduce the number of individual entries by 2,000 - 3,000 rows in a single day.

The recognized standard for country codes is ISO 3166. In this standard, each nation is assigned a unique, two-character code. The ONLY exception I found to this rule is that, for historical reasons, the IP registries have entries for the United Kingdom listed with two country codes (GB and UK). I could have corrected this in the Perl script by standardizing on a single country code, but I preferred the solution of adding another row to the "country_code" table.

From the raw Registry data, I determined that only four of the fields useful for the project that I was working on: REGISTRY, COUNTRY_CODE, ADDRESS, and NUMBER. I then wrote PERL code to read the raw IP Registry data from the data file I created previously, convert the starting IP address to a number, use this starting IP Number that to generate the end IP Number, and then insert the rows into a database.

PERL: IP Number conversion and database insert

if ($line_count >= 115000) {

## Establish Database Connection

print "\n\nOpening database connection";

my $dbh = DBI->connect("DBI:mysql:host=[host];database=logger","[username]","[password]",{PrintError=>0});

## Remove existing values
my $sth = $dbh->do("TRUNCATE TABLE ip_map");
print " --> Data from ip_map table dropped";

$sth = $dbh->prepare("INSERT into ip_map values (?,?,?,?,?,?)");

$count = 0;

print " --> Completed\n";

## Insert Data Into Database

print "Inserting data into the database";

open (PROCESS, "<$file");
while ($line =
) {
chomp ($line);
if (($line =~ m/\|ipv4\|/) and ($line !~ m/\|\*\|/)) {

($registrar,$country_code,$item_type,$start_ip,$num_ip,$entry_date,$registry_type) = split(/\|/, $line);

$long_start = 0;

if ($start_ip =~ /\d+\.\d+\.\d+\.\d+/) {
## my $ip_address = shift($start_ip);
$long_start = ip2long($start_ip);
## print "$ip_address converts to $ip_number\n";
$long_end = $long_start + ($num_ip-1);
$count += $sth->execute($country_code,$registrar,$long_start,$long_end,$num_ip,$start_ip);



The "TRUNCATE" statement in the script has the affect of dropping the table and re-creating it using the column names and types defined in the initial create statement. It is easier to rebuild the data table each time new data is inserted to ensure that duplicates and overlaps do not enter into the database.

Why is the value of "$long_end" defined by "$long_start + ($num_ip-1)"? The IP address ranges delivered by the registries count the starting value as one of the items in the set -- i.e. counting using ordinal numbers.

If cardinal numbering is used to calculate the address range, incorrect IP addresses will be generated.
IP Number Calculations
WRONG! 216853760 = 216853504 + 256 -> END_IP =
RIGHT! 216853759 = 216853504 + (256-1) -> END_IP =

I have also added a sanity-check that stops the insertion process if the number of lines in the data file is less than 75,0000 115,000. This would prevents the creation of a truncated database if one of the registries does not update their data files or the script is unable to retrieve the data files. The value of 75,000 115,000 appears high, but the data files that the data is extracted from also contain autonomous system and IPV6 data, as well as the IPV4 data that is inserted into the database. Currently, the data file runs to over 90,000 130,000 lines, so the 75,000 115,000 line barrier seems very reasonable to prevent inserting a broken dataset.

Querying the Database

Now that the database is constructed, we can start to run queries against it.
mysql> select ip.code,ip.registry, ip.ip_from, ip.ip_to,
-> from ip_map ip, country_code co
-> where (ip.code = 'IS') and (ip.code = co.code);

| code | registry | ip_from | ip_to | country |
| IS | ripencc | 1049722880 | 1049731071 | ICELAND |
| IS | ripencc | 1359937536 | 1359970303 | ICELAND |
| IS | ripencc | 1383088128 | 1383096319 | ICELAND |
| IS | ripencc | 1385447424 | 1385455615 | ICELAND |
| IS | ripencc | 1390215168 | 1390280703 | ICELAND |
| IS | ripencc | 1403846656 | 1403863039 | ICELAND |
| IS | ripencc | 1433681920 | 1433690111 | ICELAND |
| IS | ripencc | 1439023104 | 1439039487 | ICELAND |
| IS | ripencc | 1440481280 | 1440514047 | ICELAND |
| IS | ripencc | 2644312064 | 2644377599 | ICELAND |
| IS | ripencc | 3238264832 | 3238330367 | ICELAND |
| IS | ripencc | 3245150208 | 3245154303 | ICELAND |
| IS | ripencc | 3261718528 | 3261726719 | ICELAND |
| IS | ripencc | 3264217088 | 3264282623 | ICELAND |
| IS | ripencc | 3556884480 | 3556892671 | ICELAND |
| IS | ripencc | 3558785024 | 3558793215 | ICELAND |
| IS | ripencc | 3565084672 | 3565092863 | ICELAND |
| IS | ripencc | 3584524288 | 3584532479 | ICELAND |
| IS | ripencc | 3585114112 | 3585122303 | ICELAND |
| IS | ripencc | 3585433600 | 3585441791 | ICELAND |
| IS | ripencc | 3586023424 | 3586031615 | ICELAND |
| IS | ripencc | 3587538944 | 3587547135 | ICELAND |
| IS | ripencc | 3587981312 | 3587997695 | ICELAND |
| IS | ripencc | 3641278464 | 3641282559 | ICELAND |
| IS | ripencc | 3642535936 | 3642540031 | ICELAND |
| IS | ripencc | 3650592768 | 3650596863 | ICELAND |
| IS | ripencc | 3650596864 | 3650600959 | ICELAND |
| IS | ripencc | 3651915776 | 3651919871 | ICELAND |

28 rows in set (0.09 sec)

So the database structure is sound. It is important to build the file using all four five registries; even though Iceland is now covered by RIPE, older IP allocations and assignments were been handled by both RIPE and ARIN.

Having the registry information helps build in the flexibility to add a WHOIS functionality using this database, something that I have done on for my own IP Tracking and logging tool. This allows for further drilldowns on the data, beyond the scope of this article.

The main item that will be of interest to most Web site administrators is that they can now build dynamic pages using a data source which tracks their visitors' announced IP address to the country of origin with a high degree of accuracy. This is particulary useful if you are attempting to distribute users to geographically diverse mirror sites. You can also do fun things, such as displaying the flag of the country that the visitor is coming from.

A generic copy of the registry retrieval and database insertion script can be yours just by contacting me.

Monday, November 7, 2005

Kathy Sierra: Balance in Life

Kathy Sierra once again reminds us that only the truly intelligent employers realize that the Work/Life balance is more important than ANYTHING else. [here]

And her reason for this: clients who abuse startups and small companies who then abuse their employees to work miracles.

The takeaway:

And as the tech employment market starts to tick up ever so slightly, it's becoming less and less of an "employer's market" again. I don't care about the Aeron chair, but I do care about having a life beyond work. If you can't make your business model work without promising your clients a miracle (which we're expected to pull off), change your business model! And when you DO ask us to go our ass off again, a little worshipping goes a long way ; )

TypePad: Recovery is a painful and rewarding thing

Mena Trott posted a note on her blog that details all of the performance enhancements and upgrades they have been through recently. [here]

And it has improved.

TypePad performance

I know it's easier to malign an organization than to give it credit where it is due; and I am giving the SixApart team a huge amount of credit for bringing this back from the edge.

Sunday, November 6, 2005

GrabPERF: Second Agent Down…AGAIN

I lost the second measurement agent at 2005-11-05 15:34:35 yesterday. And as it's remote and threw some kind of fatal error, I have no way to re-start it until tomorrow.


Volunteers to host another measurement location?

FEMA: Yes, we are really stupid

FEMA sends a bill for $3.7 billion to the state of Louisiana

Hmm...I would refuse to pay on the grounds that no usable services were provided by FEMA. In fact, I think that Louisiana should send FEMA a bill.


Thursday, November 3, 2005

GrabPERF: A little enhancement

One thing that I have been frustrated with in cURL is the lack of OS level network error reporting.

Guess what? I should read the docs. It's been in the code for a few revisions now.

The problem: It appears to behave erratically. I have it working successfully on one measurement machine (local network agent), but when I blindly rolled it out to the remote Gomez Agent, all the measurements melted down and started throwing this mysterious Error Code 43.

The difference is that the local machine is running linux 2.4.20-31.9 (most likely Redhat 9) and the Gomez Agent is running 2.4.22-26mdk (Mandrake 9? 10?).

I tried to test the code on one of my linux 2.6 (Fedora Core 3) servers running cURL 7.12.3, which should have this code in place, but it returns nothing when the connection fails due to a reset or timeout.

I now understand the need for network homgeneity. ARRRGH!

I just don't get why the cURL RPMs for Fedora on the cURL site throw module dependency errors; and I am not getting involved in an RPM upgrade "falling dominoes" party at 01:00 EST.

I hope the Connection Timeout errors help some of you folks out who where just getting gaps when weird network errors were occurring.

Technorati: Tag Search Update — WOW!

A while back, I posted a note that the Tag search app at Technorati was behaving in a manner that did not match the performance of the rest of their products. [here]

The Technorati team followed up with me today and asked if I had seen any improvement.


Ummm...yeah. Just a little.

UPDATE: And Technorati just told me that they started using HTTP Compression.


Technorati: ,

IceRocket: ,

Wednesday, November 2, 2005

Going to DC

I will be in DC/Baltimore on Thursday and Friday on business. No time for a meetup, but if anything weird happens with GrabPERF or the server, I will get to it as soon as I can.

Tuesday, November 1, 2005

BlogPulse: WOW!

From a Web performance


From 5 seconds to 1.5 seconds.

Technorati: ,

IceRocket: ,

Gomery blames Chr

This is shocking!

I am appalled! I don't believe that Jean Chretien could have committed such an act! The man had an unblemished....

errr, nearly spotless...

ok, a little tarnish here and there...

...ummm, where did put my handcuffs again?

Canada’s Shame: Water, Water Everywhere, Except for the First Peoples

Keeping tabs on the news from home, it appears that our nation has condemned many communities of the First Peoples to communities with water not fit for any living thing. [here and here]

Canada is consistently rated one of the best countries in the world to live in. But how can this tragedy be allowed to continue? How can a developed nation, holding huge reserves of the world's water supply, poison it's own First Peoples with the basis of life?

Canada likes to see itself as morally superior to the United States. But allowing the systemic racism embedded in the culture of leadership to doom a generation of the most valuable (and undervalued) people in our country to disease and poverty is a shame that needs to be exposed.

The cover needs to be taken off this pot; and like Katrina, the boiling mess underneath will show the true face of the nation.

Technorati: , , ,

IceRocket: , , ,

Sunday, October 30, 2005

The World Is Clustered

A very insightful article on the clustering of wealth and innovation around the world, riffing on a theme by Friedman. [here]

I firmly believe this concept. I have lived three cluster areas -- Vancouver-Victoria, BC; San Francisco - San Jose Metro; Boston Metro -- and the level of innovation and original thinking here is unparalleled. When I travel to non-cluster cities, I am always stunned by how far out on the edge I feel; and I am boring and a follower.

It is also interesting, as a side-note, to mention the political leanings of the cluster cities. If you are talking geography, do these clusters tend to be more open to change, more welcoming of difference, and (at least in the Excited States) seen as more "liberal"?

Lisa, does Bill have a blog yet?

IceRocket doesn’t like GrabPERF

Guess Blake and the team at IceRocket don't want to be measured anymore.

Guess I will disable those measurements -- they will remain off until further notice.

Blake got back to me -- issue fixed! Yay!



PubSub: Anyone have issues getting to PubSub Today?

Just noticed this.


Anyone have troubles getting PubSub sites today?



Want to buy North America’s Oldest Company?

This makes me sad.

A South Caroline businessman is making an attempt to purchase the Hudson's Bay Company. [here]

The company, lovingly referred to as the Bay, was incorporated in 1670 and once owned more real estate in North America, in fact "all the land in the Hudson Bay watershed – a mass that amounts to about a third of present-day Canada." [here]

The HBC is the early history of Canada. Without their greed and ingenuity, Canada would never have grown beyond the Valley of the St. Lawrence.

Perhaps this is the revenge for War of 1812.

Technorati: ,

IceRocket: ,

Saturday, October 29, 2005

GrabPERF: End of Daylight Savings Time in the US

At 2AM EDT Oct 30, 2005, it will become 1AM EST Oct 30, 2005. This shift will take us from GMT -4 to GMT -5.

As always, hang onto your hats, as I have no idea how the time change will affect the graphing in GrabPERF. My theory is that there should be double measurements for the 1 AM hour, and the hourly rollup will fail, as there will already be an entry for the time period.

I could write an exception for this twice yearly foobar, but let's just say this update is scraping the bottom of the GrabPERF punch-list.

At least I don't have this problem.

Feedster: Odd errors from the local GrabPERF Agent

I have been getting Partial File errors out of cURL on the Agent that is running on my home network. These started at around 16:15 EDT October, 28, 2005. This is why the availability for this measurement is running at around 50%.

Now, did the folks at Feedster make a networking/server change around that time?



Snowing, but not sticking…yet.

Need I say anything more?

Thus it began.

Friday, October 28, 2005

Jeff Nolan and TypePad

Jeff Nolan adds his voice to the TypePad rumblings. [here]

Take away quote:

Some very experienced IT professionals are chiming in that they understand how difficult it is to run a datacenter (something I myself have not done I would add) and I am sure the empathy is well meaning. However, there are ample examples of large datacenter operations that operate with a very high degree of uptime and performance so while running a high capacity and complex datacenter may be difficult, it is being done successfully elseware.

Mena and Ben (further putting the human face on the company, a good PR tactic) may be plenty sorry, but I'm still stuck paying for shitty Typepad performance. In fact, I'd like to see this subscription software model go even further to align with customers by putting a service level agreement (SLA) in place that lays out a series of financial penalties the provider incurs when performance and availability fall below predefined thresholds. I bet they would not have to be sorry then, because companies that have these kinds of service level agreements are always highly motivated to get it right the first time.

Blog Herald Slams TypePad

Wow. It's not often you read something like this.

Oh yeah, it's the blogosphere.

Take away quote:

The question then is: if you ran out of space and were having problems, why did you continue to take on new customers during this period? Surely a responsible business with serious capacity issues would have closed their doors to new business to assure that its current clients were taken care of.

Its called greed.

Thursday, October 27, 2005

Switched to Pingoat

I've been using Ping-O-Matic for a long time, but in the last 2-3 days, it has simply not been responding.

Switched over to Pingoat -- hope that works better.

Rojo: Here Rojo! C’mon boy! Don’t run away from me!

Rojo has started showing unusual performance values over the last 24 hours.


Most of the gain has been in First Byte times -- the time between the Client issuing the GET, and the Server's response arriving back at the Client.

Since I use Rojo, this is a concern for me.

Technorati: ,

IceRocket: ,

LifeHacker Nails It: Rules of Engagement


  1. Engage. Participate. Be fully present. No auto-pilot.

  2. Meetings and multiple appointments are a fact of work-life; the least we can do is be on time so they can start on time and our peers are not kept waiting.

  3. Respect the attention of your peers. Come prepared means come prepared.

  4. Always have a pen and paper for note-taking. First, you respect others who are giving you information by acknowledging it, and secondly youÂ’re expected to capture it, and follow-up; forgetting is not an option.

  5. Whatever your role is, youÂ’re expected to be the expert in that role. Own it, and donÂ’t be shy about it. Stake your claim proudly. (This was part of the no bench-warmers philosophy.)

  6. When you say youÂ’ll follow-up on something, do. If itÂ’s not going to happen, say so. People trip when you sweep stuff under the rug.

  7. Own up to your mistakes and be okay with them. Making mistakes is perfectly fine for we all make them. However huffing and puffing about them with excuses and justifications is not fine. Get over it (we already did) and just correct it.

  8. Communicate. We have found that relying on mind-reading doesnÂ’t work that well for us.

  9. Trust and be trust-worthy. Much easier when Rules 1 - 8 are honored and we all keep it real.
ng this out now to be posted prominently.

Wednesday, October 26, 2005

GrabPERF: Outage — 20:39-21:09 EDT October 26, 2005

For reasons that I have yet to discover, my outbound connection to the Internet appears to have been down between the times listed in the title.

As I have said before, my connection is less than "professional grade", i.e. "don't do this at home".

If I find a cause, I will update.

I apologize for the incovenience.


Found a MONDO huge problem with mysqldump in MySQL 5.0.15 -- it locked the tables. Totally.

I backup the database by taking a dump 4 times a day, 12 hours apart on two machines.

The Web server machine, where I was testing MySQL 5.0.15 prior to the production upgrade, is using the version mysqldump that shipped with that.

I need to read the docs and disable the table locks.

Ok. Does Gutter Helmet just not get it?

Ok, finally got some action from the people at the Customer Service Centre for Gutter Helmet towards helping us fix the final drip between the house and the gutter. This drip is, of course, right over the back door and targets anyone leaving the house when its raining.

The Customer Service Centre was having some trouble reaching their technician. Lucky for them, the technician was across the street, working on our neighbour's house. Maybe we should have told them...?

So, I call Samantha, and Samantha goes across the street to ask the gentleman to call in and speak to the rep handling our case.

Samantha called me after she did this and said that the technician did everything short of growl and bare his teeth at her.

Look, Gutter Helmet: your on-the-ground "technicians" are the face of your company (except for the fast-talking sales dudes). This is the third -- yes, THIRD -- technician we have encountered who has had the customer service skills of a molotov cocktail.

We don't ask for much; a little civility is all that is required.

"Yes ma'am. The Customer Service Rep got a hold of me and I will come over as soon as I'm done here. Thank you."

That's all it takes.


UPDATE (to the update): When Tracy from the Customer Service Centre called me back to follow-up, I told her that the technician had been and this had been confirmed by Samantha.

I then told her about the technician's attitude. In fact, it was the same technician who had come out previously, and given us some attitude. And the bad news was, he had also given Tracy some attitude.

Maybe this will be the final post on this topic.

Tuesday, October 25, 2005

Upgrade to MySQL 5.0.15 Complete

Wait! Didn't he say he had done this already?

Yup. But forgot one stupid little step. You have to re-compile PHP to work with the new MySQL 5.0 libraries. I tried to restart Apache and it barfed all over the PHP module with MySQL library module mismatch errors...or something like that.

Done and the system is now using the right MySQL libraries.

Usual caveats about system weirdness apply. Operator weirdness is your own issue.

Gada.Be — Web Performance Art

I have never seen Web performance graphs as art, but this one is unique.

Gada.Be -- October 25, 2005

You could almost compose music around this pattern.

I have heard from Chris Pirillo about some of the work he has been doing to try and improve the performance of this site. It's hard to get something right when no one else has done it before.

But I still hear the music of this graph in my head.

Technorati: ,

IceRocket: ,


In case some of you want to host a GrabPERF Measurement Agent, the current prerequisites are available here.

I hope to get some folks signing up for this during this round.

Really. A river runs under my servers

Think I make this up?

GrabPERF/Newest Industry Server Room

These are the servers that power GrabPERF and this blog.

Don't worry. The UPS on the floor is dead.

Typepad: Ouch. Again.

The TypePad application is currently experiencing performance degradation.

You could say that.


UPDATE: Looks like TypePad has returned to something resembling normal. Slow, but normal.


UPDATE: Geek News Central commiserates with TypePad. [here]

Technorati: , ,

IceRocket: , ,

Using MySQL 5.0

Ok, so I lied; sue me.

The main database is now running MySQL 5.0.15.

Please report any weirdness you see with any of my applications.

Testing of MySQL 5.0

Ok Everyone, if you start seeing some odd behaviour from my sites (GrabPERF, Newest Industry,, it's because I am doing some testing to see if the scripts, processes, and applications I have built on top of MySQL 4.1 can stand the upgrade.

I will not be upgrading the primary database system until all of this testing is complete.

Venetian Server Room

Have I ever mentioned that when it rains, water comes into our basement?

Have I ever mentioned that said water flows less than an inch directly below the only place in the house my servers can go?


Monday, October 24, 2005

GrabPERF: CIA Web Site Run By Very Tired Hamsters

I have said it before, and I say it again: the performance profile of the CIA Web Site SUCKS.


How can you stand this if you are a major government agency? Guess no one at the CIA cares.

Technorati: IceRocket:

GrabPERF: Comparing Technorati Blog and Tag Search

Normally when I discuss the performance of a page I am measuring using GrabPERF, it's either good news ("you just got 5 times faster!") or bad news ("your page hasn't loaded in 6 months; you still there?").

Today, something a little different: a question. What's the question?

Why is the performance of a Technorati Blog (aka Traditional) Search so different from a Technorati Tag Search?

For those of you who have been around for a while, you know that Technorati allows you to search for results based on a Traditional search engine methodology, which is date-ranked, most recent first. It also provides a way to search through the user-defined tags that are appended to posts, or listed as category titles.

The issue that I have been seeing from my measurements is that Tag Searching is performance substantially worse than Traditional Search.



What I need to understand from the Technorati team is the particular technical challenges that differentiate Traditional v. Tag Searching, because the difference in performance is astonishing.

And then there is the success rate of the Tag Search.


When I examine the data, almost all of the errors on the Tag Search measurement are Operation Timeouts. I have set the GrabPERF Agent to time out when no response has come back for the server in 60 seconds. So, effectively 15% of the Tag Searches do not return data to the client in 60 seconds.

So, while the Traditional Search has been tuned and optimized, there appears to be much work left to make the Tag Search an effective and useful tool.

Technorati: , , ,

IceRocket: , , ,

Sunday, October 23, 2005

Never Work Alone: Integrating your IT Team…or vice versa

The gang at the Never Work Alone blog have a fantastic post describing some of the solutions to the Introverted IT / Extroverted Sales-Marketing integration issue.[here]

The best points:

  • When hiring, place a premium on being able to explain technical issues to users and determine whether they've mastered the material. Expect this to cost more.

  • Offer raises for taking training in oral technical communication

  • Offer "days off" learning the essential business function of the department. You don't understand what they do, they often don't really GET what you do either, nor why its important - gieve them a chance to understand each other

  • Train non-IT staff to repeat back in their own words what the IT person explained to them and confirm that they got it right (a good idea for any complex communication)

My eternal salvation comes from falling into the first category listed above. I can tear apart a packet trace and spot issues at the TCP layer, and then turn around and explain this issue to the VP of Marketing in terms that she can understand, and are relevant to her.

That is not dumbing it down, as many IT people feel. This strategy (or survival mechanism) allows a technical person to appeal to a wider audience. Being recognized across your organization, not just in your team, leads to greater rewards in the long run.

Saturday, October 22, 2005

Search Engine Referral Statistics

Jeremy Zawodny asked the community for stats on where their search engine referrals were coming from.[here]


Google is crushing all other engines. However, I have noticed that my traffic has dropped substantially since the new Page Rank indexing began this week. Wonder what's up there...

Happy Birthday Mighty Girl

Mighty Girl got the hippest 30th Birthday Present anyone could ask for. [here and here]

Bryan, make mine an Orange VW "Thing".

VW Orange Thing ... I think I love you!

Step Right Up for the new…It!

This needs this theme song.

Via Business 2.0

Ok, this falls under freaky spam

This was in my mailbox just now. Huh?!?!?


Thinking about having a baby?

We are looking for couples to take part in a new BBC series exploring the science behind getting pregnant and pregnancy. If you are thinking about trying for a baby or have already started trying and would like to find out more, please call us on 0141 204 6620 or e-mail:


We'd like to advise you that we got your email address from a mailing list company. Their lists are compiled from those who have agreed, when visiting relevant websites, to receive contact from third parties

If you would like to be removed from our list please reply to

I am trying to figure out what list I said yes to to get this email.

GrabPERF: Yahoo BlogSearch Tuning

The Yahoo! BlogSearch started off showing less than remarkable results in terms of performance (I leave the qalitative judgement to other critics). Over the last 11 days, the team at Yahoo! have realized that there may be an issue, and they have been working on it.

On Thursday afternoon (Oct 20, 2005), they obviously implemented a major change that caused performance to improve dramatically.

Yahoo! BlogSearch -- Oct 22 2005

This improvement was due to some back-end changes in the search itself. How do I know this? All the improvement came in first-byte (server response time).

------------------- -------------------

10/20/2005 00:00:00 1.2159322
10/20/2005 01:00:00 1.2658667
10/20/2005 02:00:00 1.3596000
10/20/2005 03:00:00 1.1870328
10/20/2005 04:00:00 1.1672373
10/20/2005 05:00:00 1.2970500
10/20/2005 06:00:00 1.2220333
10/20/2005 07:00:00 1.3705500
10/20/2005 08:00:00 1.4188667
10/20/2005 09:00:00 1.4439000
10/20/2005 10:00:00 1.5772000
10/20/2005 11:00:00 1.4943559
10/20/2005 12:00:00 1.4794426
10/20/2005 13:00:00 1.4017333
10/20/2005 14:00:00 1.6012500
10/20/2005 15:00:00 1.4380333
10/20/2005 16:00:00 1.1326441
10/20/2005 17:00:00 0.5613000
10/20/2005 18:00:00 0.5656833
10/20/2005 19:00:00 0.5766833
10/20/2005 20:00:00 0.5219831
10/20/2005 21:00:00 0.4722131
10/20/2005 22:00:00 0.5022333
10/20/2005 23:00:00 0.4569138

Would love to hear from the Yahoo team, and learn exactly what they did to bring about such a massive improvement.

Friday, October 21, 2005

Is Web 2.0 Suffocating the Internet?

At my job, I get involved in trying to solve a lot of hairball problems that seem obscure and bizarre. It's the nature of what I do.

Over the last 3 weeks, some issues that we have been investigating as independent performance-related trends merged into a single meta-issue. I can't go into the details right now, but what is clear to me (and some of the folks I work with are slowly starting to ascribe to this view) is that the background noise of Web 2.0 services and traffic have started to drown out, and, in some cases, overwhelm the traditional Internet traffic.

Most of the time, you can discount my hare-brained theories. But this one is backed by some really unusual trends that we found yesterday in the publicly available statistics from the Public Exchange points.

I am no network expert, but I am noticing a VERY large upward trend in the volume of traffic going into and out of these locations around the world. And these are simply the public peering exchanges; it would be interesting to see what the traffic statistics at some of the Tier 1 and Tier 2 private peering locations, and at some of the larger co-location facilities looks like.

Now to my theory.

The background noise generated by the explosion of Web 2.0 (i.e. "Always Online") applications (RSS aggregators, Update pings, email checkers, weather updates, Adsense stats, etc., etc.) are starting to really cause a significant impact on the overall performance of the Internet as a whole.

Some of the coal-mine canaries, organizations that have extreme sensitivity to changes in overall Internet performance, are starting to notice this. Are there other anecdotal/quantitative results that people can point to? Have people trended their performance/traffic data over the last 1 to 2 years?

I may be blowing smoke, but I think that we may be quietly approaching an inflection point in the Internet's capacity, one that sheer bandwidth itself cannot overcome. In many respects, this is a result of the commercial aspects of the Internet being attached to a notoriously inefficient application-level protocol, built on top of a best-effort delivery mechanism.

The problems with HTTP are coming back to haunt us, especially in the area of optimization. About two years ago, I attended a dinner run by an analyst firm where this subject was discussed. I wasn't as sensitive to strategic topics as I am now, but I can see now that the topics being raised have now come to pass.

How are we going to deal with this? We can start with the easy stuff.

  • Persistent Connections

  • HTTP Compression

  • Explicit Caching

  • Minimize Bytes

The hard stuff comes after: how to we fix the underlying network? What application is going to relace HTTP?

Comments? Questions?

Technorati: , , , , ,

IceRocket: , , , , ,

Thursday, October 20, 2005

GrabPERF: Silly Bug Caught and Squashed

Was looking at some data in the interface tonight and I realized something. In Scatter Charts where there is an error, the average times where far higher than they should be.

Guess what?

I was trying to be cute by combining the average time calculation and the availability calculation into a single query. But by doing this, I was including error measurements in the calculation of the average component and total times.

Not good.

Fixed now.


Colour Printers — “They know when you’ve been bad or good…”

The EFF is letting us know that the US Government (your government, not mine) has secretly collaborated with printer manufacturers to embed a unique tracking code in every document coming from your colour printer. [here]

Via: Travis Smith

GrabPERF: MSN Search gets snarky

Looks like the folks at MSN Search have decided to block the GrabPERF measurements of the standard search term.

Oh well, doesn't bother me. I never really considered them a viable contender for any search title. "Too little, too late" is my personal opinion on the MSN Search experience.

Also, it was the slowest of the Big Three.

Average Performance -- Oct 1-19, 2005

--------------- --------- ------
Google - Search 0.3524125 18891
Yahoo - Search 0.6395253 18849
MSN - Search 0.7182230 18876

Looking forward to see if anyone from MSN notices...or cares.

UPDATE: MSN Search appears to have fixed their problem. the measurement is now re-instated.

Would sure like to know what happened here.

Technorati: , ,

IceRocket: , ,

Wednesday, October 19, 2005

Comcast and Hotmail: 1d10ts Untie!

Seems there is an unannounced little issue sending mail from Comcast to Hotmail. [here]

I use a Web mail program; guess the twits at comcast think that everyone should be.

Technorati: , ,

IceRocket: , ,

Gutter Helmet: Not alone with installation problems

Hey everybody! Looks like someone else had a bad experience with a Gutter Helmet installer. [here]
Berry was dismayed to discover that his $3,500 gutter-protection system appeared to be inviting one of the very maintenance nightmares it was supposed to protect against: roof rot. Two leaks sprang up after the pricey helmets had been installed, he claimed, which caused rainwater to build up underneath the custom-fitted panels and drip down behind the gutter "for days" after a rainfall.

Hmmm...I sure hope that my local rep manages to find some time in his busy schedule to call me back. Soon.

Technorati: , , ,

IceRocket: , , ,

Gutter Helmet: An Update

The update on my Gutter Helmet posts (1 and 2) is that there is no update.

We received a phone call from a local Gutter Helmet rep about 2 weeks ago. He spoke to Samantha who, outlined some of our concerns and issues. I then called and left him a message last week, re-iterating these same concerns, and noting that there was still a leak between the new roof and the new gutters which needed to be fixed by flashing.

Nothing. No response. Silence.

I know it's Gutter Helmet's busy season. I know this because the number of hits to my previous posts are increasing.

Maybe some customers are experiencing better installations; I hope so. All I can do is continue to recount my experience to you.

Technorati: , , ,

IceRocket: , , ,

GrabPERF: Feedster Makes A “Huge” Improvement

Ok, for the average viewer a 0.25 second improvement doesn't look like much.


However, this is extrememly deceptive. The searches were using the the bare minimum terms necessary.[SEARCH TERM]

This stopped working correctly last night. Effectively, it looks like Feedster's Date Search is broken. How did I figure this out? Well, the size of the results page went from 30,000+ bytes to 15,000+ bytes. This is always a clue that either:

  1. The server has started using a compression technology

  2. The page being returned is not the one you expect

I have switched the measurements back to Relevance Searches; these are less important to blog searches (more like old-school search engines). This will keep Feedster on an even par with everyone else, it will have to stay this way until Feedster fixes their Date Search.

The new search is:[SEARCH TERM]&sort=relevance

I would love to hear from the Feedster team when they have fixed this issue.

UPDATE: Scott Johnson and Jeff Kletsky from Feedster dropped by. The relevance search is back up and running, and the measurement has been updated accordingly.

Tuesday, October 18, 2005

More Random Tom Waits

For lack of anything intelligent to say right now...

Step Right Up
Tom Waits
from Small Change

Step right up
step right up
step right up
Everyone's a winner, bargains galore
That's right, you too can be the proud owner
Of the quality goes in before the name goes on
One-tenth of a dollar
one-tenth of a dollar
we got service after sales
You need perfume? we got perfume
how 'bout an engagement ring?
Something for the little lady
something for the little lady
Something for the little lady, hmm
Three for a dollar
We got a year-end clearance, we got a white sale
And a smoke-damaged furniture
you can drive it away today
Act now, act now
and receive as our gift, our gift to you
They come in all colors, one size fits all
No muss, no fuss, no spills
you're tired of kitchen drudgery
Everything must go
going out of business
going out of business
Going out of business sale
Fifty percent off original retail price
skip the middle man
Don't settle for less
How do we do it?
how do we do it?
volume, volume, turn up the volume
Now you've heard it advertised, don't hesitate
Don't be caught with your drawers down
Don't be caught with your drawers down
You can step right up, step right up

That's right, it filets, it chops
It dices, slices, never stops
lasts a lifetime, mows your lawn
And it mows your lawn
and it picks up the kids from school
It gets rid of unwanted facial hair
it gets rid of embarrassing age spots
It delivers a pizza
and it lengthens, and it strengthens
And it finds that slipper that's been at large
under the chaise longe for several weeks
And it plays a mean Rhythm Master
It makes excuses for unwanted lipstick on your collar
And it's only a dollar, step right up
it's only a dollar, step right up

'Cause it forges your signature.
If not completely satisfied
mail back unused portion of product
For complete refund of price of purchase
Step right up
Please allow thirty days for delivery
don't be fooled by cheap imitations
You can live in it, live in it
laugh in it, love in it
Swim in it, sleep in it
Live in it, swim in it
laugh in it, love in it
Removes embarrassing stains from contour sheets
that's right
And it entertains visiting relatives
it turns a sandwich into a banquet
Tired of being the life of the party?
Change your shorts
change your life
change your life
Change into a nine-year-old Hindu boy
get rid of your wife
And it walks your dog, and it doubles on sax
Doubles on sax, you can jump back Jack
see you later alligator
See you later alligator
And it steals your car
It gets rid of your gambling debts, it quits smoking
It's a friend, and it's a companion
And it's the only product you will ever need
Follow these easy assembly instructions
it never needs ironing
Well it takes weights off hips, bust
thighs, chin, midriff
Gives you dandruff, and it finds you a job
it is a job
And it strips the phone company free
take ten for five exchange
And it gives you denture breath
And you know it's a friend, and it's a companion
And it gets rid of your traveler's checks
It's new, it's improved, it's old-fashioned
Well it takes care of business
never needs winding
Never needs winding
never needs winding
Gets rid of blackheads, the heartbreak of psoriasis
Christ, you don't know the meaning of heartbreak, buddy
C'mon, c'mon, c'mon, c'mon
'Cause it's effective, it's defective
it creates household odors
It disinfects, it sanitizes for your protection
It gives you an erection
it wins the election
Why put up with painful corns any longer?
It's a redeemable coupon, no obligation
no salesman will visit your home
We got a jackpot, jackpot, jackpot
prizes, prizes, prizes, all work guaranteed
How do we do it
how do we do it
how do we do it
how do we do it
We need your business
we're going out of business
We'll give you the business
Get on the business
end of our going-out-of-business sale
Receive our free brochure, free brochure
Read the easy-to-follow assembly instructions
batteries not included
Send before midnight tomorrow, terms available
Step right up
step right up
step right up
You got it buddy: the large print giveth
and the small print taketh away
Step right up
you can step right up
you can step right up
C'mon step right up
(Get away from me kid, you bother me...)
Step right up, step right up, step right up
c'mon, c'mon, c'mon, c'mon, c'mon
Step right up
you can step right up
c'mon and step right up
C'mon and step right up

Monday, October 17, 2005


I have permanently blacklisted all of people making comments from -- it appears that Google's little nightmare is getting worse.

Thankfully b2evolution makes that easy. I hate to think about what the the other blogging platforms will have to go through to make this happen.

PS: This is in response to Chris Pirillo's well-timed rant on this topic. [here]

Sunday, October 16, 2005

GrabPERF: Agent Upgrade (PERL Module)

Last night, I upgraded the Local Network GrabPERF measurement Agent to the WWW-Curl-3.02 PERL module. I will be doing the same thing on the Gomez Agent tomorrow morning, once I bring it back up from it's power-outage induced slumber.

GrabPERF: Interesting Blog Search Data

Ok, this is a little weird.

First, go look at this page.

What's weird in this data is the difference that you can see between the one-word and the two-word searches.

I was asked to look into this by someone at one of the Blog Search Engines (who shall remain anonymous), under the suspicion that there would be some difference in the performance of these these two-searches.

I want to watch this for a while before I publish the next Search Index results. What I am seeing here is that there could be a methodological issue in the search. Be interesting to see how these react to three word searches.

If any of the folks from the search engines want to chat about this, drop a comment in the usual place.

Saturday, October 15, 2005

GrabPERF: Lost an Agent for the Weekend.

The Gomez Network Agent is offline due to a power failure at the hosting facility. I will bring this Agent back online on Monday morning.

Sorry for the inconvenience.

GrabPERF: Water in the basement

Woke up to 3 inches of water in my basement. Where the GrabPERF Web and database servers are hosted.

Oh, and this blog and my personal Web site.

Given the amount of rain we have had lately, this is the only water we have had.


Friday, October 14, 2005

MySQL Replication

I am thinking about setting up MySQL replication for the GrabPERF database so that the massive number of inserts done does not interfere with the performance of the Web interface.

Does anyone out there have any advice/comments/concerns/experiences on doing this for an active database with large amounts of data in it?

Apache 2.0.55

The Web server has been upgraded to Apache 2.0.55.

Let me know if you see any unusual weirdness.

Tuesday, October 11, 2005

TypePad: Server failure leads to outage


'nuff said.

UPDATE: From Technorati

TypePad Service

The TypePad application is currently experiencing performance degradation. You may be experiencing application errors due to the issue with published blog content noted below. We are aware of the issue and are working hard to correct it as soon as possible; we apologize for the inconvenience.

UPDATE (1:10 pm) We have been working on correcting this and you should see the improvement currently.

At present, the stats page continues to be disabled. We expect that stats will be restored today.

Posted by Six Apart Ops at 11:15 AM

TypePad Blogs

UPDATE (1:10 PM). Service has been restored to published weblogs shortly after the last update and is continuing solid.


UPDATE (12:31 PM). We are currently restoring service to published weblogs. While we restore service, you may experience degraded performance. We appreciate and thank you for your patience.


UPDATE (12:02 PM). One of the main servers has gone off line. We are in the process of restarting it.

We apologize for the inconvenience, and we will post an update here as soon as possible.

Posted by Six Apart Ops at 11:16 AM

Technorati: ,

IceRocket: ,

Rojo Rojo Double Double

I am seeing an awful lot of doubled posts on Rojo today. Anyone know what's up?

Yahoo Blog Search: Compression?

I guess bandwidth doesn't matter to Yahoo. They don't use gzip/HTTP compression any more.

Ok, not completely true. It's there on the Portal Page. But nowhere else.


I give up.

Oklahoma University Bombing Coverup

Ok, I love inflammatory headlines. But they go with this story: In Bombing At OU Stadium There Are More Questions Than Answers

Travis Smith was the first to clue me in that this happened. [here]

I wonder this one is being swept under the rug? If it was a suicide, it's sad; but is there more to this?

GrabPERF: Yahoo Blogsearch added to Search Index

Ok, the GrabPERF Search Index has a new member: Yahoo Blogsearch.

We'll see how it works out in term of performance; Scoble has some initial comments on quality of results. [here]

Monday, October 10, 2005

GrabPERF: Search Index Weekly Results (Oct 04-10, 2005)

The weekly GrabPERF Search Index Results are in.


  • Level3/Cogent Peering Dispute affected all results this week

Week of October 04-10, 2005

-------------------------------- --------- ------- --------
PubSub - Search 0.4032768 99.86 6644
Google - Search 0.5196460 100.00 6735
Yahoo - Search 0.6926705 100.00 6694
MSN - Search 0.7543688 100.00 6722
Google Blogsearch (ATOM)- Search 0.8480318 99.82 6725
Google Blogsearch - Search 0.8880010 99.84 6731
eBay - Search 0.9723281 100.00 6736
Newsgator - Search 0.9935382 99.61 6685 - Search 1.2988528 99.94 6721
Findory - Search 1.3819290 99.93 6693
Technorati - Search 1.4609834 99.88 6671
Amazon - Search 1.6579611 98.94 1799
BENCHMARK RESULTS 1.7649221 99.56 108774
Feedster - Search 2.4226014 99.73 6671
IceRocket - Search 2.6644861 99.51 6698
Blogdigger - Search 2.7916678 99.14 6378
BlogLines - Search 3.5648172 96.84 6735
Blogpulse - Search 6.6344053 98.96 6736

These results are based on data gathered from two remote measurement locations in North America. Each location takes a measurement approximately once every five minutes.

The measurements are for the base HTML document only. No images or referenced files are included.

Technorati: , , , , ,

IceRocket: , , , , ,

Gada.Be — Gotta Be Up To Be Useful

I love Chris Pirillo's new search tool, Gada.Be.

I fired up the standard measurement on GrabPERF: "new york"


I just de-activated the measurement, it was hanging the whole measurement system. I'll fire it up again tomorrow and see if there is some improvement.

UPDATE: has become usable, but performance does vary substantially.


My Hidden Talent

For all the people I know: Please get off the floor and stop laughing.

***Your Hidden Talent***

You're super sensitive and easily able to understand situations.
You tend to solve complex problems in a flash, without needing a lot of facts.
Decision making is easy for you. You have killer intuition.
The right path is always clear, and you're a bit of a visionary.

What's Your Hidden Talent?

Via: Jason Womack

Sunday, October 9, 2005

Web 2.0: It is as bad as I feared

Renee Blodgett confirms my fears about the Web 2.0 bubble starting. [here]

Hope you have a parachute or a soft spot to land...

GrabPERF: Come kick the tires before you buy

GrabPERF is now officially back in the open: All data; All the time.

I hope you like some of the changes I have made in the site navigation pages. I am slowly adding new features when I can, so watch for a trickle of new items appearing.

GrabPERF is still up for sale. I can't dedicate the time necessary to making this system world-class given my current responsibilities, unless I can find some serious sponsors to help out.

Enjoy, and let me know if you want your site added!

Saturday, October 8, 2005

Friday, October 7, 2005

Google Feed Reader: YUCK!

Ok, I thought I was missing something about the Google Feed Reader. I thought it was clumsy and flashy and mostly usefless for the way I read blogs.

Chris Selland makes me feel less alone. [here]

Paul Kedrosky is not amused. [here]

Brad Hill says interesting, but call me when you're ready. [here]

Bud at The Community Engine says that it is not user-friendly. [here]

Sean Coon: flashy AJAX + proprietary sandbox == Google wants to own your experience. [here]

GoogleÂ’s UX staff take a vacation, return to horror and destruction

I'm sticking with Rojo.