Tangled in the Threads

Jon Udell, November 3, 1999

Why Isn't ODBC a Standard Feature of Linux?

Microsoft has understood for years the importance of generalized, abstract data access. Linux still doesn't get it.

Years ago, Microsoft announced a strategy called Windows Open Services Architecture (WOSA). The crowning jewel of WOSA was and is ODBC (Open Database Connectivity), a model in which the Windows platform provides a general set of database-access services, and database vendors provide adaptors for their products. As a result, Windows applications can get out of the business of writing adaptors for datasources, and can instead focus on doing something useful with the data that's accessible to them by way of the standard ODBC infrastructure.

The same idea plays out elsewhere in Windows. There's unified access to printers, for example, and to modems. And Microsoft did a wonderful job of neutralizing the differences between the various network transport protocols. Windows networking services run identically over TCP/IP, IPX/SPX, and NetBEUI, because the platform abstracts the differences among these protocols in a neutral layer.

"Microsoft is just a marketing company," the Linux community likes to gloat. There's plenty of truth in that statement, but the real story is a lot more complex. When it comes to WOSA-style abstraction of core services, it's Microsoft that emerges as the innovator, and Unix/Linux as the foot-dragging Neanderthal.

Why do I raise this issue now? Well, last week I was working on a Zope/Python application that needs to talk to a SQL database. I started out with Gadfly, an all-Python lightweight SQL engine that's bundled with Zope. Gadfly is really slick, and it's a treat that Zope comes with a built-in SQL engine that you can use immediately for prototyping. But an in-memory, script-language-based SQL engine wasn't going to handle my project's needs forever. And, as nifty as Gadfly is, it's just a subset of SQL -- no ALTER TABLE command, for example. So it was time to hook Zope up to a conventional SQL engine.

Working on the NT box in my home lab, that was a no-brainer thanks to ODBC. Zope provides an adaptor product called ZODBCDA. It installs in seconds, and immediately gives your Zope environment access to all the ODBC datasources your system is configured to talk to. These might be local .MDB files accessed via the Jet engine, or local or remote instances of Oracle, SQL Server, or any other database.

After prototyping my application, it was time to deploy it. As it happens, the deployment platform was Linux and MySQL. Zope itself works just the same on Linux as on Windows, I've found, so I wasn't expecting any trouble. But that's because I've come to take for granted the datasource neutrality of ODBC. And the sad fact is, that neutrality is far from routine in the Linux world. With respect to Zope and MySQL in particular, you've got to:

I'd never built a Python extension before. I tried and failed. I'm sure there are readers of this column who have tried and succeeded. I applaud those of you who have. But wouldn't you rather have spent that time doing something useful with Zope, rather than maneuvering yourself into a position where you could begin to do something useful? Life's too short for that.

Many Databases, Many Drivers

At the end of the day, I switched from Python to Perl and got my application working in short order. Why? Perl's ODBC-like driver manager, DBI, was installed on my deployment machine, as was DBD::mysql, the MySQL adapter. In general, thanks to the heroic work of DBI architect Tim Bunce and a veritable army of driver-writers, Perl users enjoy many of the benefits of ODBC. But do you see what's wrong with this picture? None of the labor invested in making DBD::mysql for Perl carries over to Python, or PHP, or Tcl, or any other tool or application that wants to talk to MySQL. Each of these environments has to define its own database-abstraction layer, and then hope to motivate developers to build adaptors for a whole range of databases. Sometimes this happens; often it doesn't. The result is spotty coverage of datasources. In the databases newsgroup I asked:

Why keep on with this crazy combinatorial explosion of apps on the one side and dbs on the other, all of which need to be connected in pairwise fashion? Note in the Windows space, there only needs to be one Zope ODBCDA, which gets you to a world of stuff. I really wish the Linux/Unix space would sort this out.

Look at Perl's DBI for example. It's a robust effort, years along, that is exactly the driver-manager/database-adaptor model that's needed. But only for Perl! Then Python comes along and has to do the Python DB-API, and hope they can get a critical mass of db developers to support it as well as the Perl developers support their "universal" API.

I claim that if you add up the total effort invested in wiring Perl, Python, PHP, Tcl, and lord knows what else to Unix databases, it's a lot more than it would have taken to get a robust ODBC-like model that could be supported once by each of these environments, and once by each db vendor.

Alastair Sherringham replied that, in fact, the issue is being sorted out. He referred us to some URLs that document various ODBC-for-Unix efforts:

In fact I'd heard of these, over the years, but I had to wonder:

Why does a universal db abstraction layer feel no closer to being a standard part of Linux today than it did four years ago?

I wonder if the open-source process -- at least as we've seen it so far -- doesn't gear up to identify and achieve these kinds of strategic objectives.

I'm not trying to be critical or dismissive of much heroic effort. I'm just genuinely curious what it would take to make multivendor db access as straightforward on Linux as it is on Windows.

A little more digging turned up another interesting URL:

http://www.openlinksw.com/iodbc/

In January 1999, OpenLink Software announced that it would manage the open-source project known as iODBC (independent ODBC), a port of Microsoft's ODBC driver manager done originally by Ke Jin. Since OpenLink Software is a savvy company with tons of database know-how, this sounded like a hopeful prospect. Perhaps OpenLink's stewardship would, finally, advance the state of the art in Linux database access. To find out more, I called OpenLink's president and CEO, Kingsley Idehen.

An interview with OpenLink Software's Kingsley Idehen

Udell: I can't keep track of all these Unix ODBC initiatives.

Idehen: There are three main threads. First there's iODBC, the open-source project started by Ke Jin. That's the one we're backing. Then there's the Merant ODBC SDK, which Merant [the union of Micro Focus and INTERSOLV] inherited from Visigenic. This ODBC for Unix product was based on code licensed from Microsoft and ported to Unix. It isn't open-source. Finally there's Unix ODBC, which also uses iODBC and adds the GUI configuration pieces that support user-friendly installation and configuration of drivers.

Udell: Why did the impetus for ODBC come from Microsoft, rather than the Unix community?

Idehen: It didn't. The SAG CLI [SQL Access Group Call-Level Interface] was originally put forward [circa 1990] by Unix database vendors -- Oracle, Informix, and so on. Microsoft joined later. But Microsoft understood, as the Unix community never did, how a graphical UI can broaden an audience. User-friendly installation, configuration, and testing were features of Microsoft's implementation that helped make it a huge commercial success.

Udell: Why hasn't iODBC gained much traction yet?

Idehen: The problem is that it was released under the GNU GPL [General Public License], not the LGPL [originally Library General Public License, now Lesser General Public License, see http://www.gnu.org/copyleft/lesser.html]. At the time we began supporting iODBC, I didn't understand the difference. But when we shipped Virtuoso [OpenLink's commercial universal-database product] on top of iODBC, people told me: "Kingsley, iODBC is GPL'd, so you have to make Virtuoso's source available too." With LGPL, you can ship a commercial product that links to open-source libraries. Now that iODBC is under LGPL, I hope we've removed a major barrier and will see substantial uptake of iODBC tools and applications on Linux.

Udell: What about UnixODBC?

Idehen: While Richard Stallman and I were arguing about this LGPL issue, Unix ODBC started as an LGPL project. And it moved forward in two key areas. First, whereas iODBC was a traditional Unix app with no GUI, Unix ODBC aimed to do the graphical pieces as well, using Qt, and to integrate these with KDE. Second, Unix ODBC advanced from ODBC 2.5 to ODBC 3.5.

Udell: What are the current plans for your iODBC project?

Idehen: We are about to put out a version of iODBC that complies with ODBC 3.5, but that is also backward-compatible with ODBC 2.5. And we're adding in the graphical pieces, but we're doing them with GTK so they'll work with KDE and GNOME. In about a week or two, you'll see a beta of this. Hopefully the Unix ODBC crowd will realize that there's still plenty of work to be done. To make iODBC really compelling, we've got to do a first-class implementation that includes Gator [the ODBC Test Tool], and driver-manager support for tracing so people can benchmark and debug their applications.

Udell: Why are you doing this work?

Idehen: Lots of people want to write applications, with Python or PHP or Tcl or whatever, that talk to data. With a standard Linux ODBC driver manager in place, we're going to sell more of our high-performance ODBC drivers, and also more of our applications like Virtuoso.

Udell: Any final thoughts?

Idehen: People in the Linux community fail to see the importance of data. It's the weirdest thing on the planet. What Microsoft understands, very well, is that at the end of the day it's data binding that makes a platform compelling to a business users. Microsoft gets that. Linux doesn't.

Jon Udell (http://udell.roninhouse.com/) was BYTE Magazine's executive editor for new media, the architect of the original www.byte.com, and author of BYTE's Web Project column. He's now an independent Web/Internet consultant, and is the author of Practical Internet Groupware, from from O'Reilly and Associates.

Creative Commons License
This work is licensed under a Creative Commons License.