I visited my local den of piracy in order to read Linked sooner than Amazon could ship it to me. Now I know more precisely what Clay Shirky means when he talks about "power law distributions" and "scale-free networks." An example of a power law distribution, as Clay noted in his ETCON talk on LiveJournal, is the way in which such networks display a "rich-get-richer" pattern of clustering. This is something Tim O'Reilly has often called attention to, as well. There is a kind of naive egalitarian notion that because links are free and everything is connected, all nodes are created equal. In fact, although it apparently never occurred to early network theorists, nothing could be further from the truth. As networks grow, new nodes don't attach themselves randomly. They prefer to connect to the nodes that are already the best-connected.
"Power law distribution" captures this idea nicely for me. The term "scale-free networks" is less intuitive. In this context, "scale" refers to the connectivity "embodied by the average node and fixed by the peak of the degree distribution." Thus, a bell curve has intrinsic scale. In contrast, a "scale-free" network -- whose graph asymptotically approaches many weakly-connected nodes at one end, and a few highly-connected nodes at the other -- has no intrinsic scale.
Terminology aside, what's most compelling here is the notion that these are universal properties of networks of all kinds: molecular, hypertextual, social, transportation. And further, that hubs -- whether they are the Chicagos and Atlantas of air travel, or the Yahoos and Googles of the Web, or the Mavens and Connectors of Malcolm Gladwell's social universe -- arise inevitably, and are sources of both immense power and frightening vulnerability.
Extra credit to the author, Albert-Laszlo Barabosi, for this unexpected reference to Bose-Einstein condensation:
Just as in a Bose-Einstein condensate all particles crowd into the lowest energy level, leaving the rest of the energy levels unpopulated, in some networks the fittest node could theoretically grab all the links...It destroys the hierarchy of hubs characterizing the scale-free topology...And there is a network in which we cannot fail to notice one node that carries the signature of a Bose-Einstein condensate. The node is called Microsoft.
OK, that's a little too cute, I admit. But this is an important book. We are only beginning to uncover the practical applications of this kind of network thinking. One example is blogging as a form of high-tech marketing. Another is Valdis Krebs' use of network visualization as a management tool. As always, the flip side of powerful is dangerous. When marketers really wrap their heads around blogs, the results won't always be pleasant. And, as Ray Ozzie noted recently, people won't enjoy being laid off when social network analysis proves they are redundant nodes. But we are embedded in these networks, we can visualize them, we will manipulate them to gain individual and collective advantage. The sooner we acknowledge this power and learn to use it responsibly, the better.
On the same shelf as Linked, in the pirate's den, I found a copy of David Gelernter's book, Mirror Worlds. Written in 1991, it resonates powerfully a dozen years later. I couldn't stop grabbing quotes. Here's a keeper:
At the same time as we develop vast complex software worlds, the simple machines of information structure are also just being invented. The wheel, the ramp, the wedge, the screw, the lever.
From Gelernter's 1991 perspective, it wasn't clear what those simple machines would be. It's a bit clearer now: HTTP, HTML, XML, RSS. We'll use these wheels and levers to make structures that rival the most magnificent monuments:
New software Saint Marks' [cathedral] will rise. They will monopolize the energy and attention of thousands in the building, will broadcast an aesthetic and a world-view to millions, will mold behavior and epitomize the age.
In 1991, Yahoo!, Napster, and Google were right around the corner. What's right around the corner now? Predictions are always dicey. For example, here's what Gelernter imagined:
To use a Mirror World program, you sit down at your computer, which has a large color screen and a connection to the local fiberoptic utility cable...Or -- if you're willing to put up with a smaller picture and it's a nice day -- you pick up your laptop, tune in Data Radio, and head for the hammock.
It's always fascinating to compare prediction to reality. In 1991 I'd have been more surprised by the Data Radio in my hammock than by the fiberoptic utility cable. Well, the fiber isn't here yet, but if it were a nice day, I'd be typing these words from my hammock.
Gelernter also foresaw the struggle between the private and the public that will dominate the rest of this decade, at least:
The Mirror World isn't snoopware. Its goal is merely to convert the theoretically public into the actually public. What was always available in principle merely becomes available in fact. Organized, archived, spiffily presented, up to the minute and integrated into a whole. That's all. But that's a lot.
And then some. When the artificial barriers of practical obscurity fall away, we are likely to want to rethink how we define public versus private records.
So, what is a Mirror World? A vast, detailed representation of a company or a city, or of parts of these structures, or even of the larger economic and political structures to which they belong. What's it good for? To monitor and debug the things that are represented:
Consider the modern, state-of-the-art fighter aircraft. It's so fantastically advanced that you can't fly it. It is aerodynamically unstable. It needs to have its "flight surfaces" adjusted by computer control every few thousandths of a second...Modern organizations are in many cases close to the same level of attainment--except that, when they're out of control, they don't crash in flames; they shamble on blindly forever.
Well, not quite forever. Today's Wall Street Journal featured this grimly comic fantasy about the financial debacle we have lately endured:
In a recent letter to shareholders, Ralph Wanger, the iconoclastic chief investment officer at Liberty Wanger Asset Management in Chicago, laments that his firm lost $956 million through the end of the third quarter. To put that figure in perspective, he tried to figure out how hard it would be to lose that much money on purpose. His explanation follows:
"One way to do it would have been to convert $956 million into $100 bills on Jan 1, 2002 and order our 20 investment professionals to spend all their time burning it. It sounds sort of festive really -- drink some beer, make S'mores and enjoy the glow, warmth and fellowship around the bonfire (singing Kumbaya optional). How hard would we have had to work to do this? Well, if one person diligently burned one $100 bill at the rate of one bill every 10 seconds and worked seven hours a day, five days a week, 50 weeks a year, that one person could burn up $63 million in a year. It would take all 20 of us working full time at this repetitive task to get rid of $956 million in just nine months."
Fidelity Investments' $129.7 billion would, the WSJ added drily, have been "some bonfire." How do you gain control of this unpilotable airplane? Pervasive instrumentation, transparency, distributed computing, loose coupling. Nowadays we call this stuff Web services and grid computing. What we haven't figured out, yet, is how to use a coordination language like Gelernter's Linda, with an associative memory like Linda's (or JavaSpaces', or TSpace's, or Ruby's) tuplespaces, to weave it all together. My guess is that we will. And the sooner, the better.
Former URL: http://weblog.infoworld.com/udell/2002/11/30.html#a523