Transparent Commodity Infrastructure and Web 2.0

Posted by Antonio 2 years, 5 months ago (Feb. 12, 2006)

Marc Hedlund over at O'Reilly Radar had an interesting piece about how smart kids, when left to their own devices in a new environment, will independently come up with a set of good development practices for Web 2.0. Adam Green followed up with a post about how some of these practices aren't so new after all since he was using them almost a decade ago and Andover.net (both pre and post the Slashdot acquisition).

While I think Adam is right that the best of the Web 1 companies were implementing some of these practices back in the 1990s, it is much more the norm now than it used to be. Part of this comes from the maturing of the problem domain-- we are all that much more educated now than we were back then about how to think through building web applications. But the biggest contributor to this brave new way of doing things in my mind is transparency of commodity infrastructure.

Let me use an example: back in 1998 if you were building a web-based startup, you were probably running on Solaris/SPARC and using an Oracle database. You were also likely to be running on some sort of a Java servlet engine (though there were exceptions, this was again the leading edge). This huge apparatus usually required at least 1 of the following: DBA, sys-admin, release manager, and build manager-- nevermind all of the consultants and vendor people that it took to solve problems that arose from trying to get everything working together.

Fast forward to 2005. Anyone still using Solaris/SPARC for web apps is either a moron or a depressed Sun shareholder. MySQL and Postgres are now considered "enterprise-grade," and if you should be so masochistic as to still want to do Java development on the app-tier, you've got Tomcat, Jetty, and even JBOSS available to you on your platform of choice.

Now here is the key: every piece of infrastructure is free, has a thriving online community that can help you with issues better than the vendors ever did, and perhaps most importantly, can scale down to run on almost any type of laptop.

This last piece is what I found was missing from Marc's post: the fact that in the Brave New World, every developer can get to have the entire stack on his own machine. The value of this should not be underestimated. It allows for features to be prototyped and tested much more quickly. If you are running across different platforms on the dev machines (Linux, OSX, and Windows) it enforces good discipline in the build system and in the modularity of the code. The platform differences will also surface bugs which would not immediately show up in a more homogenous environment (for example, concurrency is a great thing to suss out by deploying your app across platforms).

When we started building Tabblo, I insisted that we be able to build the full stack across different flavors of Linux (our deploy platform), OS X, and Windows. Given that we have a bunch of C code in the application, this proved particularly painful at the outset in the Makefiles and Linux/OSX differences. Moving it to Windows meant dealing with a bit of a Rube Goldberg that leveraged VMware for piece of the app and the portability of tools like MySQL and Python for another piece. Having taken that pain up front however, I do not think that there is anyone at work today who regrets the investment. We can all work completely independently even while completely offline despite the fact that our deploy infrastructure requires a dozen machines running together in a data center.

This was just not possible last decade. We owe it to the transparency of the open source infrastructure that we can do this now across a whole set of components that used to be the domain of experts and alchemists. Anyone who doesn't take advantage of it now is doomed to slow release cycles, the need for a full QA team on hand, and a staff-imposed level of overhead that is tough to build a business out from under.

What a great Brave New World.

Tags:

Comments

Post a comment

(Please use only plain text. Though I will escape all of your HTML, URLs will be clickable)

Your name:

Comment: