Malcolm Gladwell on the instability of preference

A conflict prevented me from attending PopTech this year, but fortunately Doug Kaye was there with his microphone. The talk I most wanted to hear -- Malcolm Gladwell's -- was posted over the weekend. In this two-minute clip, Gladwell sums up the dilemma of the Aeron chair, a wildly successful product that might never have seen the light of day had the Herman Miller company believed its market research. From this parable and one other -- the New Coke fiasco -- he draws several conclusions. One is that preferences are highly unstable. Another is that, when you ask people to explain what they want, their preferences tend to shift toward the conservative, familiar, and easy to explain.

As the author of a failed design that was radical, unfamiliar, and hard to explain -- yet very effective in an important way -- I'm fascinated by this dilemma. The design in question was the Safari Books Online search system. In its original incarnation, this product had a feature that came to be called "sticky search." The idea, which I'd first used with the BYTE CD-ROM, was to unify navigation and search. After a search, the results page reported the top-ranking items in the usual way. What was unusual was the method of exploring the rest of the result set. Rather than invite you to page through batches of decreasingly relevant results, the system used the table of contents as a filter. It constrained your view of the library to just matching items, and mapped relevance indicators onto those items, but was otherwise navigable in the normal way.

Some people liked this as much as I did. Once you got the hang of it, you could scope out the result set in a breadth-first rather than depth-first way. That was important, I argued, because once you exhaust top-ranking results, raw relevance becomes a less useful metric. And various cues embedded in the content -- category labels, chapter and title headings, the gestalt pattern of hit counts mapped to a book's outline -- become more useful.

But the "sticky search" behavior was unexpected, and ultimately the feature was judged a failure. The reason was partly logistical. As the library grew from hundreds to thousands of books, the multi-book table of contents became unwieldy. But a major reason was that people expected search to work "just like Google," and this didn't.

Is that a stable preference, though? I suspect not. If Google could usefully contextualize search in ways complementary to raw pagerank, I'm sure people would soon discover a preference for that, and come to expect it.

Since the chaotic information space searched by Google is infamously hard to categorize, though, such innovation may yet flow from more specialized engines such as Safari. Which, as a matter of fact, still retains a vestige of "sticky search." Here, for example, is a query for books about .NET and XML that maps the 391 matching books to Safari's taxonomy. And here's that same query focused on just the SQL Server category. Look in the Browse by Category box on the left to see the "sticky" aspect of these searches.

So if preferences are unstable, how do you make them tip? Like many others I'll be reading Gladwell's next book in search of clues to that $64,000 question. But I won't be expecting easy answers.

Former URL: