Hives, swarming, scale, and connectedness

When a beehive gets to critical mass, it makes a new queen, and then a bunch of the bees swarm off to create their own hive. (Although irrelevant to my point here, it is a strange fact, according to Richard Dawkins' remarkable 1996 book, River Out of Eden: A Darwinian View of Life, that the old queen, and not the new one, travels with the swarm.)

Parallels exist in human societies. There is, for example, a notion that the maximum size of a productive working community of humans is in the low hundreds. When a company grows larger than that, swarming often occurs -- and can be beneficial even if the new hive is just another building in the same office park.

In a discussion of reputation management, Bill Seitz notes:

Scaling such a system is the big challenge. CPAN is small enough to gather ratings info or comments about. But to try and do so for UseNet postings becomes really really hard.

He adds:

The big meta-problem I see with reputation management is aggregating karma across sites/markets.

Bill's observations nicely capture the next two challenges for weblogging communities. First: how to swarm? Here on Radio's community server, a variety of interest groups are forming. The shared infrastructure (directory, rankings) will, over time, become less useful to the community as a whole. Emerging interest groups will need and want their own infrastructure.

So how do hives of bloggers swarm? It's not enough to run individual instances of, say, Moveable Type. A new hive needs the kind of community infrastructure provided by Radio or Blogger. To satisfy this need, these services will likely become partitionable, or deployable, or both.

Once swarming can occur, the publish/subscribe RSS machinery can really start to shine. Things are, honestly, a bit incestuous in the blogrolling world, at least so far. "Tell me who you read, and I'll tell you who you are," I saw somewhere recently. Problem is, the blogrolls all look a lot alike. Being a member of the Radio community, or the Blogger community, shouldn't ultimately be a primary affiliation. People are software developers, or musicians, or knitters, or ... well, lots of things, often more than one.

For a while, USENET mapped out the diversity of human interest. Is weblogging headed in the same direction? Should it be? And if so, how and why will it scale differently from USENET? To come back to Bill's question, I think one key difference will be the ability to measure and monitor karma both within and across hives. A meta-Blogdex will see more continuity and interconnectedness than USENET crawlers do. It will be in a position to do some amazing collaborative filtering. Where that will go I'm not sure, but I'm curious to find out.


Former URL: http://weblog.infoworld.com/udell/2002/02/07.html#a52