Tangled in the ThreadsJon Udell, November 26, 2001
The Event-Driven Internet
What worked for the GUI should work for the Internet tooIn the web services paradigm, software speaks when spoken to. With pub/sub, software speaks when it has something to say.
When it comes to defining the emerging next-generation Internet, we are all blind men groping an elephant. Back in July, in a column called Kenamea's Application Network, I highlighted some of the agreed-upon features of the beast. These included peer autonomy, reliable and transacted communication layered over HTTP, network-global (versus per-application) access control, and a publish-and-subscribe event notification scheme.
Recently, I talked with the folks at KnowNow, whose technology touches many of these same bases. Event notification has been a longstanding research interest of KnowNow's founders, Rohit Khare and Adam Rifkin, so it's not surprising that pub/sub is a central feature of their system.
The KnowNow event router (downloadable from KnowNow) is running on a machine, and listening on port 8000.
The router's kn_apps folder has an application subfolder -- let's say it's called Dommy -- with an index.html page. On that page, there can be one or more event-enabled DOM elements.
Given this setup, here are the effects:
The content of every event-enabled element is associated with a router-managed pub/sub topic, and stored persistently in the router's cache.
When the page loads, it updates the content of each event-enabled element in response to a message received from the router.
If an element's topic does not already exist, the router creates it.
When the topic's data is modified, the page receives an event and updates itself in realtime.
The page can itself modify the topic's data, by way of an HTML form that's wired to the topic.
Here is an example of an event-enabled page. It has two event-enabled elements: a paragraph, and a form.<html> <script src="/kn/?do_method=lib"></script> <script src="../jscomponents/form.js"></script> <script src="dommy.js"></script> <body onLoad="init()"> <p>This element is subscribed to /what/dommy/test:</p> <p id="kn:/what/dommy/test">test</p> <p>This form publishes to /what/dommy/test:</p> <form action="kn:/what/dommy/test" method="POST"> <input type=text name="kn_payload"> </form> </body> </html>
When first loaded, this page creates the topic /what/dommy/test. By convention, KnowNow topic names are hierarchical. The first level describes the general type of a topic (e.g., who, what, where), the second names an application, and the third names a specific element.
Suppose the URL of this page is http://SomeServer.com:8000/kn_apps/Dommy/. I load that page into my browser, which happens to run on the same box as the router, and you also load the page into your browser which is running somewhere else. Now, I type "it changed!" into the form and hit Enter. The paragraph element, which was formerly "test," becomes "it changed!" -- in both our browser instances.
This is pretty interesting stuff! The microserver was sourced into the index.html page by this line:<script src="/kn/?do_method=lib"></script>
If you and I happen to be monitoring that topic in our browsers, the paragraph element will change interactively. But nothing says that the parties interested in that topic are always, or only, people running browsers. Of more general interest is the notion of a computing fabric in which processes subscribe to events, are notified of changes, and then take appropriate actions. Here's the skeleton of such a process, based on the Python microserver:from pyKnowNow import kn def callback(event): for item in event.rawEvent.items(): print item, ':', item, '\n' if __name__ == '__main__': k=kn(host='SomeServer.com',port=8000,user=None,password=None) k.start() k.subscribe(kn_from='/what/dommy/test',cb=callback) while k.connected(): pass k.stop()
This process will be notified whenever an event is posted to /what/dommy/test, and will echo the event. The notifier might be a DOM-wired form on a web page, but could as easily be some other process which (as in the first Python example) is talking to the event router.
Variations on the pub/sub theme
Khare and Rifkin believe passionately that Internet-scale networks of event-aware processes will be one of the key architectural innovations of the next decade. I think they probably right. If you think about it, there is nothing very new about the current web services craze. It feels more like consolidation than innovation. The web has always been about HTTP endpoints exchanging packages of formatted data. You could brutally but perhaps not unfairly say that web services only extends this model to communication among processes as well as among people.
There is nothing in the architecture of web services, per se, which breaks free of the conventional client/server, or call/response, model of data exchange. Publish/subscribe, as an architectural principle, promises something quite different. It leads to classes of applications that do not simply speak when spoken to. Rather, they speak -- to other applications and to people -- when they have something to say.
If this event-driven style is going to be an important feature of the Internet landscape, we should expect to see other examples of its use. And that's exactly what I am seeing. Over at UserLand Software, for example, Dave Winer's crew have added pub/sub capability to the xmlStorageSystem, and also to Radio UserLand. In the latter case, a publisher of an RSS channel can specify, using a cloud element, a registration procedure that a reader of the channel can use to sign up for change notifications. Here is Dave's example:<cloud domain="data.ourfavoritesongs.com" port="80" path="/RPC2" registerProcedure="ourFavoriteSongs.rssPleaseNotify" protocol="xml-rpc"/>
If I want to find out about changes immediately, rather than on the usual hourly schedule, I'll call ourFavoriteSongs.rssPleaseNotify, and ask it to alert me -- by means of XML-RPC, SOAP, or HTTP POST.
The pub/sub style is also woven deeply into Hailstorm. All of its services are kinds of data storage, and many of their XML-based APIs support registration for change alerts. Here too, web services protocols are the building blocks, but the real action takes place at a higher level of abstraction. To register for change alerts in Hailstorm, you use its data manipulation language (HSDL) to update a service document -- for example, the one that supports the .NET Inbox service. You'd add a /myInbox/subscription node, with a /myInbox/subscription/trigger that specifies an XPATH expression defining the set of nodes in the service document that are sensitive to change.
Internet software will, undoubtedly, make growing use of events to minimize polling, and ensure faster and more complete sharing of information among cooperating processes. Will these event mechanisms converge on a standard? Perhaps, though I'm not sure that moving this stuff up a level in the protocol stack is a huge priority, at least not yet. With Hailstorm, in particular, issues of identity and ownership of personal data loom larger. If you want to offer a service that reacts to changes in my .NET Inbox, we'll both need to live in the realm of Passport authentication, and I'll need to grant you permission to install the subscription trigger that watches for changes. If we can arrive at an open standard for doing these things, with multiple implementations from the commercial and open source worlds, then we can have a meaningful discussion about standardizing an event mechanism.
Jon Udell (http://udell.roninhouse.com/) was BYTE Magazine's executive editor for new media, the architect of the original www.byte.com, and author of BYTE's Web Project column. He is the author of Practical Internet Groupware, from O'Reilly and Associates. Jon now works as an independent Web/Internet consultant. His recent BYTE.com columns are archived at http://www.byte.com/tangled/
This work is licensed under a Creative Commons License.