Michael Tiemann on the future of Fedora

Tall and lean, wearing a distinctive red Fedora, Michael Tiemann is a familiar presence at Linux and open source conferences. He was Red Hat's CTO for a while, and is now Vice President, Open Source Affairs. I think of him as a hacker/economist/entrepeneur, three roles that he wove together early on as the co-founder of Cygnus Support (later Cygnus Solutions). His chapter in the 1999 book Open Sources: Voices from the Revolution tells it like it was: "I was tasked with growing the top line by day, and helping complete the work for GCC 2.0 and G++ by night."

At OSCON this year, I sat in on Tiemann's Future of Fedora session. I expected to hear an argument that would blend economics and open source culture, and that's just what Tiemann delivered in this excerpt I transcribed on the flight home:

We had a product that was kind of good for the people sitting in this room, the early adopters, but we got the 80/20 rule backwards. We could not sell any enterprise products. I mean, sure we could do web servers, sure we could do file servers, and all sorts of other fun stuff which impressed the Gartner group: "Oh, Linux, it's definitely ready for being a web server." Some guys will never learn. But the reality was that if you actually went into a place like Morgan Stanley, and they said hey, what can we do with it, if the year was 2000 then the answer was, well, it's a pretty darned good fileserver, it's a pretty darned good webserver. But would we run a database on it? No. Would we run a trading system on it? No. One of the reasons the answer was no was that they depended on some proprietary software here and there, and unfortunately no two enterprise applications ran on any one version of anybody's Linux. If you wanted to run Oracle you needed the Oracle patches, if you wanted to run Veritas you needed the Veritas patches. And if you wanted to run Oracle on the Veritas filesystem there was nothing that actually worked, because these patches conflicted. We were able to address only a small part of the enterprise market, and we were doing it for almost the entire cost of doing it right, and we said to ourselves: that's wrong.

If we wanted to actually get to the place where people could spend money -- and if you've looked at our recent progress, we're pretty proud of the fact that instead of reporting gross margins of 48%, which no software company has, because that's so low, we now have gross margins of over 80%, which is the desirable range for a software company. If you're looking at enterprise software companies and you see a gross margin of 91%, which is what we have achieved, there's a small handful of other companies that have achieved that, and they're all at the top of their class, that's where you want to be. You can't get there, no matter how smart you are, and how good you are, with these guys [the innovator/early adopter side of the chart], they just don't have that kind of money. But these guys [the enterprise side of the chart] do.

So to do that, we completely reinvented our engineering model, it was a big painful thing, we had to extend our release cycle, and while we were able to go and create this massively long release runway, so that Oracle, and Veritas, and BEA, and all these other guys could actually land on the same release, and where we could say -- with a straight face -- we will support you for five years. That was all cool, but you'll notice that we left some very important territory empty, namely, the people in this room, the innovators.

How do we return to the commitment that we weren't doing very well with Red Hat Linux, but that we were forced to completely abandon while we reinvented our company? And the answer is, we came up with this thing we call the Fedora project.

The idea of the Fedora project was to return, not only to the roots of what we set out to do with open source, as something for innovators and early adopters, but also something that could feed into, and improve, the way we develop our enterprise products.

What we did was -- the first smart thing was to say, we are not going to tie any revenue to Fedora, and that is a position we have succeeded in maintaining. Regardless of today's competitive economic conditions, we don't sell Fedora. And it's very important not to sell it, because that way we can be completely open about what it's going to be, and open about what it should be, and involve the community in that. We decided the release early and release often model was right, and on average now we're basically doing it every four months. We have a roll-forward model which we believe is the appropriate way to manage patches, for the purpose of incenting innovation.

Now the press, god bless them if they are here, have been one of the worst enemies of the Fedora project, and the reason is that they refuse to understand what it is. If Red Hat has been getting it wrong, then mea culpa, apologies, but what the press reports is: "Red Hat has forked Linux into two different versions." I guess you could look at it that way. But the reality is that we liberated the innovation platform from the production platform, and we made it so that the innovation platform could be a better bridge to the community, and a better way of conveying innovation to the production platform.

The way I think about it, and the way I encourage people to think about it, is that Fedora is a future version of our enterprise Linux platform -- not so much a separate release as simply a view into its future.

What does it give us? Well, it lets us go and make some interesting progress on improving the desktop. I know that opinions have varied on different approaches to the desktop. We're trying to solve problems that are very immediate today with the security and manageability of the desktop platform. And there's a whole bunch of other great stuff that a lot of other people are doing which is complementary to what we're doing. We are trying to fit into the way that enterprises are designing the client environment, with respect to how do those things boot, how are they managed, how are they provisioned, how are they monitored, how they fit into your disaster recovery scheme, how do they fit into an accounting scheme.

Another thing that we're taking a lot of flak for is SELinux [security-enhanced Linux]. SELinux is this secure technology which had been developed by the NSA, and was really waiting to find a home. One of the things that Red Hat can provide and decided to provide for this SELinux technology was a platform that was reasonably ubiquitous. Not everybody runs Fedora, but a lot of people do.

Fedora is a way of taking a set of packages that represents the leading edge of what a distribution can be, and letting a large number of people have a common experience. So we went to the NSA and said, "We think security is important, we think what you've done is important, we want to help."

Mandatory access control is not something new. Trusted Solaris has it, trusted IRIX has it, trusted AIX has it, but all those things did it in a way that has a standard version of an operating system as opposed to a trusted version of an operating system. And these were two different things, these were forks, because -- if nothing else -- one was more expensive than the other. And if something is more expensive, the US government has a legal obligation to avoid spending money on it. That's part of value for money, so if there's any possible way you can avoid spending money on security, by law you must. And so because people would avoid spending money on security whenever possible, that meant there was a smaller available market for secure stuff than for non-secure stuff. And because there was a smaller market, fewer people wrote applications for it. And because fewer people wrote applications, even if you wanted to deploy it, you couldn't, because the applications weren't available. That sucks.

What we thought was, with open source, we can make security ubiquitous by making it cost nothing more. We can take some wonderful security, and integrate it into something that's basically free -- Fedora is free, it's free and open source software -- and we could let people make the decision of whether they wanted it, and they can allow the application people to imagine that anybody who has a version that includes SELinux is a potential customer for a secure solution. And the NSA said, "That's what we were thinking too, let's work on this together."

So, it's been a big pain in the neck, a lot of complex integration points. Half the people on the kernel team hate us, the other half are reasonably impressed with what we have been able to get working, but to make a long story short, what SELinux has given us, and what Fedora gave us, is the platform for moving something forward which, quite frankly, none of our old models would have.

If we had to look at the first thing Fedora got right, or made possible, one of the things was we were able to take some technology and try it out before anybody else was comfortable trying it out. What has security-enhanced Linux given us? An ability to partition things into security contexts, such that even if you break one of the contexts, you don't break the whole system. A wonderful example of this is Russell Coker, who now works for Red Hat, put two machines up on the Net, with remote root exploits posted on his website. The remote root exploits were: "Here's the IP address, here's the root password." Pretty simple to execute. These two machines have been on the Net now for a year and a half and, as far as I know, neither of them has been compromised outside the security contexts that were defined. Which is a stark contrast to all the hundreds of thousands if not millions of Windows machines compromised every week.

Understandably, given Red Hat's realignment of priorities, Tiemann seemed a tad defensive making these arguments to the open source crowd. On the whole, though, I find them to be rational. If Sun does create an open source version of Solaris, the goal will be the same: to create a platform for innovation, and a means of conveying that innovation into the enterprise product.

I'm interested in reactions to Tiemann's arguments -- beyond the by-now-expected trash talking, that is. So I'm posting his remarks here, in order to capture responses that I can monitor at this Bloglines URL and this Feedster URL.

Former URL: http://weblog.infoworld.com/udell/2004/08/04.html#a1053