Lessons from the cookie laboratory

I did a double-take when I finally caught up with the Sept 5 issue of the New Yorker last night. The theme of the issue is food, and Malcolm Gladwell's contribution details a research and development project to develop a new kind of cookie. The subject of the piece is Steve Gundrum, who runs a food R & D company called Mattson and is also -- it turns out -- an armchair software theorist. Gundrum decides to organize the project into three teams:

The open source team is favored to win the bakeoff, but creative friction and communication overhead take their toll, and instead the conventional team wins, followed by the open source team, with the XP team far behind. Gladwell turns to Joel Spolsky who offers this comment:

One of the reasons open source works well for Linux is that there isn't any real design work to be undertaken. They were doing what we could call chasing taillights.

Them's fightin' words! But while Andrew Morton, co-maintainer of the Linux kernel, puts a different spin on things, I don't think he'd wholly disagree. In his fascinating talk at SDForum he notes that the modularity of the Linux kernel makes it unusually amenable to decentralized development. And he sees Linux's innovation more in terms of economics than technology:

All these IT companies are congealing around this stack of free software which nobody owns -- or if you like, which everybody owns. This allows lots of different competing industry players to use the same basic set of system software without relinquishing control to the provider of that software. These are, frankly, old technologies -- ten, twenty years old. There's no real intellectual value left in these things.

What about the higher levels of the stack? This comment surprised me:

When you have end-user customer-facing applications, my own personal belief is that's just not an area in which open source is going to play. If these applications are sufficiently useful and interesting that people are willing to pay for them, it doesn't make sense to me that you should develop a high-value contemporary modern application using open source.

Although Steve Gundrum's experiment was hardly rigorous or conclusive, it does touch on a real issue with open source. Striking a balance between an architecture of participation and an architecture of control is a central concern for all kinds of product development. In the case of most successful software applications that ride above the commodity stacks, the balance we've struck so far typically locates participation outside the API boundary. Whether that can change, and whether it should, are two very interesting questions.

Former URL: http://weblog.infoworld.com/udell/2005/09/06.html#a1295