The deep structure of code

It's been a while since I've written code in Java. But this week I updated my installation of Eclipse, dusted off an old Java program, and saw that ancient artifact in a whole new way. Now, I've never been an IDE (Integrated Development Environment) kind of guy. It's always been emacs for me, and in InfoWorld circles I'm not alone in that preference. Both Chad Dickerson and Maggie Biggs have also made the case for the no-frills approach. At the end of the day, programming still comes down to pushing lines of code around in text files. Until we change that paradigm, I've always thought, code editors built to push lines of code around in text files are going to be hard to beat.

...

At one time or another, every programmer has imagined what it would be like to work directly with the deep structure of code. Some of the best minds in the business are working to make that happen. The legendary Charles Simonyi, who left Microsoft a couple of years ago to pursue his vision of intentional programming, says deep structure is at the core of the toolset his new company, Intentional Software, is building. Sergey Dmitriev shares Simonyi's vision, and his company -- JetBrains, creator of IntelliJ IDEA -- wants to do something similar with its next-generation toolset.

These projects are still under wraps, but another champion of deep structure is working out in the open. Jonathan Edwards, currently a visiting engineer with MIT's Software Design Group, has built a prototype system that he is demonstrating in a screencast at subtextual.org. There are big ideas at work here. In Edwards' prototype, programming, testing, and debugging are just different ways of interacting with a program's tree structure. Edwards' 2004 OOPSLA paper, Example-centric programming, explores one of the benefits of this arrangement: the examples (or "use cases") that drive program design are worked out the context of the living and evolving program.

We've all heard this stuff before. I may yet go to my grave without emacs ever having been pried from my cold dead fingers. But it's worth pondering, now and then, what we could do with tools that didn't think of programs as strings of text. [Full story InfoWorld.com]

Many programmers swear they will never use a system that doesn't let them see their programs "as they really are". However, this position ignores the fact that it takes several hundred thousand lines of device drivers, the operating system, and a graphical interface to turn magnetic spins on a disk into pixels on a screen. If we remove just one of those layers of abstraction, our programs would look like this:

/  *  * \r \n  |  *  |  T  h  i  s  |  c  l  a
s  s  |  p  r  i  n  t  s  |  <  e  m  >  o  d
d  |  n  u  m  b  e  r  s  <  /  e  m  >  . \r
n  |  *  |  S  e  e  |  t  h  e  |  <  a  |  h
r  e  f  =  "  {  @  d  o  c  R  o  o  t  }  /
c  o  p  y  r  i  g  h  t  .  h  t  m  l  "  >
C  o  p  y  r  i  g  h  t  <  /  a  >  . \r \n
|  *  |  @  a  u  t  h  o  r  |  G  r  e  g  |
W  i  l  s  o  n \r \n  |  *  |  @  v  e  r  s
i  o  n  |  1  .  2 \r \n  |  *  / \r \n \r \n
p  u  b  l  i  c  |  c  l  a  s  s  |  O  d  d
s  |  { \r \n  |  |  p  u  b  l  i  c  |  s  t
a  t  i  c  |  v  o  i  d  |  m  a  i  n  (  S
t  r  i  n  g  [  ]  |  a  r  g  s  )  |  {  |
| \r \n  |  |  |  |  f  o  r  |  (  i  n  t  |
i  =  0  ;  |  i  <  1  0  ;  |  +  +  i  )  |
{  |  | \r \n  |  |  |  |  |  |  i  f  |  (  i
|  %  |  2  |  =  =  |  0  )  |  { \r \n  |  |
|  |  |  |  |  |  S  y  s  t  e  m  .  o  u  t
.  p  r  i  n  t  l  n  (  i  )  ; \r \n  |  |
|  |  |  |  } \r \n  |  |  |  |  } \r \n  |  |
} \r \n  } \r \n

Remove another layer of abstraction, and we would have a table of numeric character codes; another, and we would have a stream of bits. It is therefore hypocritical to object to adding another layer (unless the person raising the objection still uses a text-only web browser like Lynx [Lynx]. It also ignores the fact that fewer and fewer documents are stored as byte-per-character ASCII; increasingly, documents use some character encoding scheme to represent Unicode [Spolsky], which editors then interpret on the fly.

Somewhat to Greg's chagrin, his article was recently Slashdotted. In response he has reiterated that while XML might provide a standard representation of programs at one level of abstraction, the goal is not XSLT writ large. Sure, you can use a text editor to wrap angle brackets around programming-language keywords, but as the Slashdotters point out there's no real traction there. What most of the Slashdotters ignored was the idea of tools that work directly with the deep structure of code -- transforming it, animating it, extending it.

I've talked with Greg a few times about this topic, and to be honest my end of the conversation was mainly footdragging and devil's advocacy. Bootstrapping an ecosystem full of tools that work this way does seem like an insurmountable problem. Still, times change. Refactoring IDEs are a relatively new thing, but a generation of programmers who have grown up using them will bring new expectations to the game.

Former URL: http://weblog.infoworld.com/udell/2005/01/19.html#a1154