Programs as Text Files
We’re used to thinking of programs as text files: the first step on the road
to being a Real Programmer
is pick a text editor “that reflects [your] personal style”. In old-school Unix
emacs, but even modern IDEs are really a tangle of
enhancements wrapped around a text editor.
And code style is a subject for endless debate. 80-(ish)-column typewriters led to 80-column punched cards led to 80-column terminals and now 80 column code is enshrined in everyone’s coding standards. Even punctuation and whitespace turn out to be fraught, and endless arguments ensue.
Back when monitors were small and computer time was expensive, we used to print out listings so we could take our programs away to think about. When did you last print out a program?
Programming beyond Text
But that’s not the only way to program: Scratchlikes allow you to edit a syntax tree with drag-and-drop actions and some Visual Languages may even do away with the syntax tree! This can be great for avoiding getting bogged down in typing and syntax, but they do place a lot of reliance on mousing, annoying fluent typists.
Languages with very little syntax, like Lisp, are very well suited to writing such an editor. JSON is only a little harder. Python is not going to be all that easy. The trick will be to balance keystroke driven operations with automatic tree operations, so that you can type code naturally but then drag-and-drop subtrees as well. The language grammar may need to be extended slightly to allow comments etc, but this could be a good thing: if you’re going to have a markup language for code documentation, why not make it a first-class part of your code and subject to syntax checking along with everything else?
I haven’t used it in years, but Oxygen XML did a pretty good job of this: once you specify a schema, XML documents are very tightly restricted in what can possibly go where so the UI was able to guide you pretty effectively as I recall.
I’ve been playing around with this using HTML5 and Ractive, trying to make a fairly generic tree editor called “tred”.
I think this idea applies especially well to MicroPython …
Programming beyond files
The above assumes that you’re going to work by parsing your code into your editor, manipulating the AST and then (when you hit save) serializing the AST back out into a form which can be parsed back into an AST by a compiler. On the upside, that fits in with your existing tools, but it also seems rather wasteful
Since the code is always in a syntactically valid form (it has to be, after all: it is being manipulated as an AST) it should be also possible to write an incremental compiler which keeps your output code / bytecode up to date as you edit your source code. I’m a big fan of Software Development at 1Hz, why not take that to the next level and have your tests constantly rerunning themselves whenever you touch code which they depend on?
One of the arguments against doing this is that we have so many good tools for dealing with plain text files: Diff3 for example. But I think we could make even smarter tools with a better underlying data structure. The Lisp world uses homoiconicity to its advantage: treating programs as data and data as programs is easy when your language is
Perhaps this is taking it all a little too far, but I love the idea of the whole
filesystem being replaced by an append-only tree of nodes, treated homogeneously all the way
from the root of the filesystem to the individual atom of a parsed program. We’d lose
a lot of our old friends like
diff3, but think of what we’d gain …
The trickiest thing with the editor has got to be coming up with a mapping of keyboard and mouse operations to tree transformations. Subtrees can be dragged and dropped into place on the tree, and maintaining the typical mappings for Ctrl-Z,X,C,V make sense to make it easy for users. Other keys not typically used in syntax are Escape, Enter, Home, End, PgUp PgDn, Insert, Delete and the up/down arrow keys. What keys do depends somewhat on context too: the aim is to have as few extraneous keystrokes as possible.
|Enter||Depends on contact: First Child / Next Sibling / Next-Sib-Of-Parent|
|Delete||Delete current node|
For typing a language like Python, the syntax can follow your typing, to automatically
parse code as you type, much like a Pratt parser. To keep the syntax tree valid while
editing code, ephemeral
0 nodes can be temporarily inserted.
With just one character, we can’t tell much, it could be an identifier so we’ll make it an identifier.
if is a keyword so it can’t be an identifier. We can see this, turn it into
a conditional element with an empty condition and action
if None: pass
… and then select the condition so it is overwritten as you type:
if a: pass
… keep typing and you keep entering stuff for the condition:
if a == b: pass
… hit tab to move to the action, which since it is ephemeral (
pass doesn’t actually do
anything, so you can always replace it).
if a == b: print()
… we see the open paren and now we know that’s a function invocation … so now we’re inserting arguments into the list of arguments. Hit “ and you’re entering a string, which will handle escaping and so on for you …
if a == b: print("hello")
Hit enter to pop out of the string into arguments list, and again to pop out of the
print to the next statement in the
if, then again to pop out to after the