Tangled in the Threads

Jon Udell, October 20, 1999

XML-RPC Programming with Zope

Zope integrates XML in deep and far-sighted ways

In the programming newsgroup, Digital Creations' Michel Pelletier showed us how Zope can automate the delivery of an RSS channel based on a query:

Did you know that Zope can return Catalog queries in RSS? If you have, for example, a Catalog full of News items, those items can be turned into a 'channel' quite easily. This example is straight off of the shiny new Zope site (http://www.zope.org). The URL directly to the RSS is (http://www.zope.org/SiteIndex/news_rss).

[click here for full example]

The important bits are right at the end there. The <dtml-in> tag iterates over a sequence of news items returned by the Catalog query. This useful to turn just about any kind of Zope objects into a stream of RSS, or any other kind of XMLish representation of data (like RDF).

That's just plain neat! It doesn't involve any special XML smarts, really. It's just an illustration of how Zope nicely abstracts data sources and makes it easy to flow them out as HTML or, in this case, XML.

But Zope is deeply XML-aware. For example, you can call this Zope DTML method using an http: URL as shown above. You can also, as Michel pointed out, call it using XML-RPC:

Py$ import xmlrpclib
Py$ x = xmlrpclib.Server("http://www.zope.org/")
Py$ x.SiteIndex.news_rss()
<?xml version="1.0"?>
<!DOCTYPE rss PUBLIC "-//Netscape Communications//DTD RSS 0.91//EN"
  "http://my.netscape.com/publish/formats/rss-0.91.dtd">
<rss version="0.91">
<channel>
...etc...

What's going on here? Zope's entire web API -- everything that you can call using http: URL syntax -- is alternately available as an XML-RPC API. In this example, we see Python sending the XML-RPC request that corresponds to the URL http://www.zope.org/SiteIndex/news_rss.

There are, of course, infinitely many ways to construct such a request. In an earlier article about XML-RPC, I showed how to use Perl to format and transmit a raw XML-RPC request. To fetch Zope's RSS file, you could modify that script to open an HTTP connection to www.zope.org/SiteIndex and send this request:

<?xml version="1.0"?>
<methodCall>
<methodName>rss_news</methodName>
<params>
<param>
</param>
</params>
</methodCall>

Serving XML-RPC in Zope

It's evident that Zope is, automatically, an XML-RPC server. The foundation of that support is Fredrik Lundh's XML-RPC for Python. This package handles the marshalling of data between Python and XML-RPC data structures, and implements both the client and server sides of XML-RPC. Here's a side-by-side illustration, running on my NT box:

server	client
(1) python lib\python\xmlrpcserver.py serving on port 9090 (4) CALL demoMethod ({'age':42,'name':'jon'},)	(2) python >>> import sys >>> sys.path.insert(0,'c:\zope2\lib\python') >>> import xmlrpclib >>> x = xmlrpclib.Server('http://localhost:9090') (3) >>> x.demoMethod({'name':'jon','age':42}) (5) {'age': 42, 'name': 'jon'}

Here's a rundown of what's happening:

1. Server: Load the XML-RPC listener. By default it listens on port 9090.
2. Client: Load the XML-RPC client library. Open a connection to the server.
3. Client: Call a method, named 'demoMethod,' on the server. Pass it a Python dictionary (like a Perl or Java hashtable) with a string-valued member, 'name', and an integer-valued member, 42.
4. Server: Echo the name of the called method, and return its arguments to the caller.
5. Client: Receive returned arguments.

To actually use this XML-RPC server, you'd implement a Python function called demoMethod which expected, and did something useful with, that Python dictionary.

So where are the angle brackets? Where's the XML? It's all happening, but under the covers. To make things clearer, we can send this raw XML-RPC request from Perl:

<?xml version="1.0"?>
<methodCall>
<methodName>demoMethod</methodName>
  <params>
    <param>
      <value>
        <struct>
          <member>
            <name>name</name>
            <value>jon</value>
          </member>
          <member>
            <name>age</name>
            <value><i4>42</i4></value>
          </member>
        </struct>
      </value>
    </param>
  </params>
</methodCall>

Here is the response:

<?xml version='1.0'?>
<methodResponse>
<params>
<param>
<value><struct>
<member>
<name>age</name>
<value><int>42</int></value>
</member>
<member>
<name>name</name>
<value><string>jon</string></value>
</member>
</struct></value>
</param>
</params>
</methodResponse>

Now it's clearer what's going on. Lundh's XML-RPC Python kit, which is available standalone and was also integrated into Zope by Eric Kidd (who was working for UserLand Software at the time), turns incoming XML-RPC requests into native Python data structures. There is no need, in Lundh's standalone XML-RPC server, to parse and transform that incoming XML data stream. You just write a Python method that expects a Python data structure; it makes no difference whether the underlying XML-RPC request came from Perl, Python, Java, or any other implementation.

The Zope perspective

The same situation applies in Zope. One way to extend Zope is to write an "external method" in Python. Here's the same example in Zope terms:

def demoMethod(self,params):
    return params

To hook this code into Zope, put it into a file, put the file in Zope's Extensions subdirectory, and register the method as an External Method using Zope's management interface. The self parameter binds the method to a specific Zope folder. Let's say that folder is called 'demo.' We can call that method two ways: as a normal http: URL, and using XML-RPC. If we do the former, we'll need to come with some way to represent the inbound data structure. In CGI terms, we do this kind of thing all the time:

http://localhost:8080/demoMethod?params=name:jon,age:42

But suppose we need to pass an array of these structures? And suppose the structures themselves are more complex? That's the beauty of XML-RPC -- it's a standard way to represent, and pass around, complex data. When you call demoMethod using the XML-RPC protocol, the Python method in Zope that handles the call receives the complex data as a Python data structure. Again, the point is that any XML-RPC implementation can generate the call. Java or Perl or Tcl data structures on one site can get turned into XML-RPC, sent over the wire, and turned into Python data structures. Results expressed as Python data structures can get turned into XML-RPC, sent over the wire, and turned into Java or Perl or Tcl data structures.

So what's all this good for? Well, I'm working on a project to receive and store RSS newsfeeds aggregated by Dave Winer's Scripting News site. The data structures on his end originate in Frontier, Dave's commercial scripting engine. I'm receiving them in Zope, via the Internet's standard HTTP protocol.

Thanks to XML-RPC, the plumbing just isn't an issue. Dave's going to send an array of structs, I'll receive them -- in Python -- as a list of dictionaries. What then? Well, in principle dumping that data into a SQL store is child's play. In practice, I've had to do a lot of digging around in Zope and Python to figure out just how to do it. That's a tale for another time. It's clear to me, though, that this web-oriented approach to distributed computing, using XML-RPC (or SOAP, or anything roughly equivalent to these protocols, is the Right Way to realize the dream of "the network is the computer."

---

Jon Udell (http://udell.roninhouse.com/) was BYTE Magazine's executive editor for new media, the architect of the original www.byte.com, and author of BYTE's Web Project column. He's now an independent Web/Internet consultant, and is the author of Practical Internet Groupware, from from O'Reilly and Associates.

This work is licensed under a Creative Commons License.