mPulse

Monday, August 27, 2007

GrabPERF: Met with the Technorati Team

Had an on-site meeting with Dorion Carroll and the Technorati team. We discussed some ways to extend and expand the GrabPERF infrastructure, and improve the measurement gathering.

Look for changes in the next 6 months that could radically improve the system you know and love!

Wednesday, August 22, 2007

IP Registry Statistics - August 2007

My system has a daily job to collect and aggregate the IP Blocks distributed by the five registrars into a single database, and then provide high-level WHOIS information for this data. If you want to try this yourself, the interface here.

On an extremely irregular basis, I aggregate the statistics from this data, and present it to the masses for the examination. I might actually automate this data someday!

So, for August 2007 (as of August 21, 2007), here are the aggregated IP distribution statistics broken down by registrar and country.

Sunday, August 12, 2007

State of the GrabPERF Update -- August 2007

It has been at least a year since I last updated everyone on the state of GrabPERF. That's because for most of the last year, the system has been rolling along without a hitch or a major systemic change. The last major change to the agent code was alomost exactly a year ago, when I added the ability to capture text matches.

It wasn't until last week, however, that I allowed folks to be able to see the results of these text match failures. Let's just say that motivation has been low and my real job has been keeping me busy.

I did want to share the growth, and mellowing of the system as it progresses into year three.

grabperf-measurement-count-aug122007

Total Measurements Per Day

grabperf-uniqe-test-count-aug122007

Unique Tests Measured Per Day

Back in July 2005 when I started this grand experiment, I was gathering 10,000 measurements a day from less than forty tests. The system spiked at 390,000 measurements per day (April 2006) and 147 tests (June 2006).

Starting just after that, I started reducing the number of tests to improve system efficiency, and began developing the text match capability.

There have been some changes to the number of measurements, but on the whole, the system has been completely stable for the last 12 months.

As some of you may have noted, I have add some new features in the last 10 days, and re-organized the structure of the system to allow for better tracking of usage. Over the next few months, I will be attacking the code to make it process things more efficiently, but not substantially change the appearance or functionality of GrabPERF.

It has been noted by some commentators that the design doesn't pop and sizzle. No AJAX, DHTML, or other flashy gizmos. Guess what? The system is designed to deliver data efficiently and effectively. And as someone who has seen the performance fall-out from badly delivered Web 2.0 implementations, I will stick with clunky and effective, as, in the end, you gotta put all those bytes on the wire.

For those that have stuck with the system over the last year, thanks. I enjoy delivering the best measurement data money can't buy, and hope you stick around for the ride.

Thursday, August 9, 2007

GrabPERF: Content! Watch your content!

Last night, I got motivated.

Ok, I got manic. Goes with my life.

As a part of that mania, I had a breakthrough on how to present GrabPERF data that I've actually been collecting for nearly a year: text match failures.

GrabPERF has the ability to match text on page results using a standard PERL regex. By putting a regex into the measurement configuration, I can confirm that the data downloaded matches what should be there.

If there is not a match, the headers and page text are captured and inserted into a table. Up until yesterday, I was the only one who could view the data. Now, if you go to the graph configuration page (http://grabperf.org/measure_page.php?test=[insert test id here]), and see the following type of result, then click through the links.

content_error-1

If your graph configuration page says no text match configured, and you want one, let me know!

Tuesday, August 7, 2007

GrabPERF: Ads Gone

As a part of the reworking of the GrabPERF code, I removed the Google ads from all pages. They were an annoyance, and displayed items were incredibly irrelevant for the site.

Saturday, August 4, 2007

GrabPERF: Substantial Navigation Changes

If you use GrabPERF on a regular basis, the somewhat flaky navigation method has become second nature to you. In fact, to circumvent some of the idiosyncrasies, you have probably bookmarked your favourite pages.

Yesterday, I broke your links.

When I redesigned GrabPERF in February 2006, I had just discover the require function in PHP, and decided to build the entire the structure using a single container page as the framework, and individual functions called using URL parameters.

As time went on, my own "brilliance" started to get in the way of maintaining and updating the code. It took me 10-15 minutes to figure out how I constructed pages, and then find the right code to fix or update.

Yesterday, I got completely fed up with this structure.

Now, all functions have their own unique pages, making maintenance a snap. And as an added benefit, I can now effectively track the usage of individual pages, so I know where to through development efforts.

Some of the changes.

http://grabperf.org/homepage.php?page=compare&test=2&tests%5B%5D=276&tests%5B%5D=277&tests%5B%5D=279&tests%5B%5D=280

becomes

http://grabperf.org/compare.php?test=2&tests%5B%5D=276&tests%5B%5D=277&tests%5B%5D=279&tests%5B%5D=280


http://grabperf.org/homepage.php?page=scatter&test=277&hours=2

becomes

http://grabperf.org/scatter.php?test=277&hours=2

 

I apologize for the confusion that this may cause, but in the long run, this will help me make the code better, and more robust.

Friday, August 3, 2007

YSlow: Cool idea, performance inhibitor

So, a colleague of mine told me about the YSlow plugin for Firebug yesterday. I had a look at it and it does do an amazing job of analyzing Web pages according to Steve Souders performance rules.

The only hitch in the get-along: a number of pages I visited when the plugin was activated were effectively unusable for up to a minute, presumably as YSlow analyzed them.

Great idea, all Web developers whould use it to put me out of a job. But the overall performance of the plugin will need to be improved before I re-install it.
Technorati Tags: , , ,