The solution I cobbled together speaks volumes about the fundamental openness of Web applications. To find out how Gmail creates a distribution list, I logged in, created a list interactively using Gmail's form, and captured the resulting HTTP transaction using one of the handiest tools in my Web developer's kit, Firefox's LiveHTTPHeaders extension.
The next step was to replay that transaction outside of the browser. I rearranged its elements -- an URL, a chunk of HTTP POST data, and a set of HTTP headers including a cookie packed with crucial name/value pairs -- as a command-line invocation of another of the handiest tools in my kit: curl. [Full story at InfoWorld.com]
In this week's column, I discuss how the ability to capture and replay HTTP traffic enabled me to discover and exploit an implicit Gmail API. But there's a general principle underlying this hack, and it seems to me that after all these years we've barely begun to exploit it.
Consider the scenario described in this item, for example. I would love to be able to recapture the sequence of HTTP transactions behind a particularly interesting search scenario, but I never logged them. In the comment thread, aristus notes that one solution is a Firefox plugin called slogger, which I've used on and off since, let's see, tap, tap, tap, October 2004.
I haven't used slogger for a while because it quit working for me on OS X a while ago, and I haven't been able to resurrect it since. But slogger notwithstanding, there's a much deeper and more general thing that ought to be happening on every web-enabled system. It ought to be trivial to attach an observer and/or filter to HTTP pipelines. Among other things, it could shovel data into a search engine so that I could instantly recall a remembered transaction by search term, by date, or by site.
I mentioned this on a recent call with the folks reponsible for Vista's desktop search. When they mentioned extensible "protocol handlers" I got really excited, imagining a general mechanism for echoing HTTP (or SMTP) traffic through a search indexer. It turned out that isn't what they meant. They were talking about supporting file formats, not protocols. But I've always thought echoing HTTP or SMTP traffic through a searcher is a great idea, and I still do.
HTTP intermediaries are also an incredible untapped resource for developers and testers of software running in environments that range from plain old HTML-over-HTTP to formal XML web services. For the latter domain, the folks (disclosure: my friends) at Mindreef are doing really interesting work based on the ability to capture and replay SOAP packets.
I've long envisioned a general-purpose ipchains-like capability, for all operating systems, that would make it trivial to attach observers, filters, and transformers to bread-and-butter protocols like HTTP and SMTP. Is it just me, or is this a gigantic missed opportunity?
Former URL: http://weblog.infoworld.com/udell/2006/12/07.html#a1573