This work is licensed under a Creative Commons License.
August 1999
The Web makes distributed computing easy. XML is making it even easier.At UserLand Software, Dave Winer recently deployed the kind of simple, elegant, and useful application of Internet technology that always puts a smile on my face. At his site, you can run a Mail To The Future application that enables you to send mail to yourself (or, actually, anyone) at some future date.
This is, all by itself, a wonderful idea. And since the service is intrinsically programmable, like every CGI-based Web service, it's both an interactive application that you can drive with a browser, and a component that you can drive with a script. I could, for example, use it as an HTTP-based reminder component built into a groupware or workflow system of my own devising. To do that, I'd follow the procedure outlined in a previous article, (Measuring Web Mindshare). There, I showed how it's possible to combine Yahoo and Alta Vista to measure the inbound links to a list of sites in a given category. And I pointed out that every Web site that offers a CGI-based service is, potentially, a programmable component that can be woven into another Web application.
What's this got to do with XML? In addition to revolutionizing content management, XML is going to change how we build and use Web-based software components. In this article, I'll demonstrate one way that XML can do that. But first, let's review the basics of HTTP-style component programming.
Web sites that support CGI-based services implicitly define what I call Web APIs. For example, part of Alta Vista's Web API is a method that might be formalized as:
getReferenceCount ( sitename )
My mindshare-measuring script invokes what might be called a Web remote procedure call (RPC) on the Alta Vista site in order to count references to a target site. You don't have to write scripts to use Web RPCs; we do this trivially when we save bookmarks to CGI-based services (e.g., a bookmark to an Alta Vista query, not its result). But when you automate the use of these RPCs, you can create new and interesting applications.
To do that involves a process that I call reverse-engineering the Web API. In the case of Yahoo, the relevant API is just its tree-structured URL namespace (e.g. /Computers_and_Internet/Software/Operating_Systems/Windows/). In the case of Alta Vista, it's a CGI-style URL built on this pattern:
http://www.altavista.com/cgi-bin/query?pg=q&kl=XX&q=link%3A{SITENAME}+-url%3A{DOMAIN}
Here {SITENAME} and {DOMAIN} are, effectively, function-call arguments; the script plugs in real values for these arguments and then "calls" the Alta Vista "component." Alta Vista's Web API is easy to discover because it uses the HTTP GET method; that means the URL you need to parameterize is left sitting in the browser's Location (Netscape) or Address (Microsoft) field. When a site uses the HTTP POST method instead, discovering its API is a little harder. Usually, though, you can view the source of a Web form, discover the names and values that will be sent to the CGI engine, and formulate the URL your script will need to use to manipulate that site's CGI services.
Dave Winer is taking this game to a new level. His product, Frontier, has evolved from a popular scripting language for the Mac into a Web content-management toolkit for Mac and Windows. Frontier includes excellent XML support, and Dave's been a pioneer in understanding and applying XML. (If you're new to XML, or Extensible Markup Language, see http://www.xml.com/ for a good orientation.) Along the way, he cooked up a marvelously simple and elegant idea that he calls XML-RPC. In a nutshell, it's a way of formalizing the CGI interface to a Web-based service using XML to define both the methods supported by the service, and the data marshalling used when communicating with such services.
The architecture of XML-RPC is gloriously simple. A service that supports XML-RPC, such as Mail To The Future, responds to method calls that are nothing more than HTTP POST requests with XML-formatted payloads. For example, to add a new message to your queue at Mail To The Future, you invoke the method:
mailToTheFuture.addMessage(username, password, msgstruct)
To use this method, you post an XML-RPC request that expresses that method name, and its three parameters. Listing 1 is a Perl script that illustrates one of the (infinitely many) easy ways to form and transmit such a request. The XML-RPC request appears in literal form following the script's __END__ directive.
An XML-RPC request can be as easily read and written by a human as by a program. That's one of the great features of all Internet software: the protocols (HTTP, SMTP, NNTP) are text-based and therefore easy to understand, and to manipulate in any programming language.
Of course that's also true for ordinary scripted CGI work, such as our mindshare-measuring script. What's special about XML-RPC? Well, consider the response that comes back when you send the wrong password to the MailToTheFuture service using the normal CGI interface:
"Sorry! There was an error: Can't send you the cookie for 'yourname@yourmailhost.com' because the password is incorrect."
You can certainly teach your script to recognize and deal with these kinds of responses, but it's going to be a fragile mechanism. You're relying on free-form text patterns and, when they change, your script will break. In the case of our mindshare-measuring script, for example, a page redesign at either Yahoo or Alta Vista will very likely force the script to have to recognize new patterns in the output from these components.
Now consider the response that comes back from MailToTheFuture's XML-RPC interface when you send the wrong password:
<?xml version="1.0"?>
<methodResponse>
<fault>
<value>
<struct>
<member>
<name>faultCode</name>
<value>
<int>4</int>
</value>
</member>
<member>
<name>faultString</name>
<value>
<string>The password is incorrect.</string>
</value>
</member>
</struct>
</value>
</fault>
</methodResponse>
Like the request, this response is well-formed -- and thus automatically parseable -- XML. Every XML-RPC service will use this same pattern. It's true that, for each application, you'll need to decide how to handle faultCode 4. But you won't need to guess that the output is an example of a methodResponse, or that its value is a fault object containing a struct made up of a faultCode and a faultString.
This notion of a standard, human-readable-and-writable, script-parseable lingo for requests and responses is a lot more in tune with the ecology of the Web than DCOM, CORBA, or RMI. It's all HTTP-based so you get firewall penetration, SSL encryption if you need it, and a wealth of tools that know how to work both the client and server ends of the HTTP protocol.
As you might expect, XML-RPC isn't the only implementation of this idea. There's also Allaire's Web Distributed Data Exchange (WDDX) and webMethod's Web Interface Definition Language (WIDL).
Like XML-RPC, WDDX defines an XML-formatted request/response protocol. The WDDX SDK includes libraries that enable COM, Java, JavaScript, VBScript, Perl, and Cold Fusion applications to serialize complex data structures as WDDX packets, and exchange such packets. This means, for example, that you can take a server-based ColdFusion array, serialize it as XML, send it to a browser, and unpack it as a VBScript array object used in a browser-based script.
Using webMethod's WIDL, you can encapsulate existing CGI-style services, such as Alta Vista's, in an XML interface. The company's B2B Integration Server uses this technique to create XML-based proxies that wrap clean, consistent, and stable XML packaging around CGI services that are messy, inconsistent, and ever-changing.
The next step is API discovery. That is, instead of reading the docs at mailtothefuture.com to learn how to drive its XML interface, you'll just hit the site with a standard interface query, and it will tell you what methods it supports.
Depending on whom you talk to, XML is either SGML-lite or HTML on steroids. Either way, most discussions of XML focus on issues of document management. Free-form HTML pages are becoming rule-governed XML pages that can be reliably searched, updated, and transformed by parser-driven applications. But even as XML takes Web content to a whole new level of structure and sophistication, there's another side to the story. It's also taking distributed Web computing to a whole new level.
Does distributed computing have to be any harder than this? I don't think so.
Jon Udell, an independent author and Web/Internet consultant, was BYTE Magazine's executive editor for new media, the architect of the original byte.com, and author of BYTE's Web Project column. His new book, Practical Internet Groupware, is forthcoming from O'Reilly and Associates.
Listing 1: Invoking the mailToTheFuture.addMessage method from Perl
#! /usr/bin/perl -w
#
# This script implements the addMessage method at Dave Winer's
# www.MailToTheFuture.com site, using XML-RPC.
#
# Jon Udell, udell@monad.net, <http://udell.roninhouse.com/>
my $req_data = join('', <DATA>); # slurp the XML from __END__ zone
use strict;
use LWP;
my $ua = new LWP::UserAgent; # make a user agent
my $req = new HTTP::Request 'POST'; # make a request
$req->url('http://www.mailtothefuture.com/RPC2'); # set url
$req->header(Host => "yourhost"); # Host: yourhost
$req->user_agent($ua->agent); # User-Agent: libwww-perl/5.36
$req->content_type('text/xml'); # Content-Type: text/xml
$req->content_length(length($req_data)); # Content-Length: 762
$req->content($req_data); # append request
my $res = $ua->request($req); # send the request
if ($res->is_success)
{ print $res->content; } # echo response
else
{ print "Could not transmit request\n"; }
__END__
<?xml version="1.0"?>
<methodCall>
<methodName>mailToTheFuture.addMessage</methodName>
<params>
<param>
<value>yourname@yourmailhost.com</value>
</param>
<param>
<value>yourpassword</value>
</param>
<param>
<value>
<struct>
<member>
<name>dateTime</name>
<value>5/23/99; 11:15:00 AM</value>
</member>
<member>
<name>messageBody</name>
<value>test message body</value>
</member>
<member>
<name>receiverMailAddress</name>
<value>yourname@yourmailhost.com</value>
</member>
<member>
<name>subject</name>
<value>test message subject</value>
</member>
</struct>
</value>
</param>
</params>
</methodCall>
This work is licensed under a
Creative Commons License.