the onda

After yesterday's poor performance live blogging the Steve keynote, I spent some time thinking about how frustrating it is to have a fundamentally communications-related service go down the way Twitter did for all of us who were trying to let the world know about what went on during the keynote.

I can't really blame Twitter. The pile-on that they are suffering from as everyone tells them they can't scale is something that any other startup would kill for, and the fact that they are taking egg on their face is to be completely expected. I don't think I know of a single small startup these days that wouldn't be similarly crushed by the load they experience during events like Macworld— in fact, I can almost guarantee that we at Tabblo would have been.

However, the big bummer about the way we use the Internet today is that we are breaking its fundamental architectural principle of loosely couple services that you don't count on for 100% reliability. This was the genius behind SMTP: if the receiving mail server was down for whatever reason, the sending one had a protocol for either finding a relaying service or for backing off and re-delivering later. Unfortunately web services aren't built like this (some of the bigger ones like Amazon and Google are built like this on the backend though which is why they scale, but it takes big bucks to get there).

And you know what? For most web services, this single point of failure design is ok. Just not messaging-based ones. When we use messaging-based services, we expect uptime (witness how annoying Gmail's recent glitches have been), and at the very least, reliability on the message delivery front.

We'd do well to think of this as we shift any time an attention to web services that have grown their own internal messaging systems, or even those that aim to replace them.

I am sure that with microformats, syndication and personal publishing platforms that we own ourselves and host on elastic computing clouds like Amazon's EC2, we can rebuild most of the messaging/publishing-based services that are currently appealing (Twitter, Jaiku, Flickr) but flakey. However, it is going to take time, standards, and the realization that we can in be in control of more reliable online experiences.

In the meantime, just be glad those evil phone companies spent billions building "carrier-grade systems" (well except for AT&T EDGE network which sucks).

Postscript: Dave Winer sort of nails it.