Respecting the User's Input

I had the privilege of working at a Computer-Aided Design (CAD) company, Calma, in the late 70s, and thus I experienced first-hand the early years of computer graphics. And it was at Calma that I learned a principle that has served me well ever since: respect the user's input.

The users of the Calma workstations used a CalComp Talos table digitizer. You would tape your sheet of VLSI artwork down on the digitizer and then you could digitize the traces (conduits of electrons on the surface of the chip you were designing) using a puck. Digitizing was a tedious process and eventually this process was eliminated by allowing the designer to build their design directly on the CAD system in the first place. Which is obvious, of course. But computers were really primitive back then and so this was a necessary stepping stone.

It was considered massively incorrect to require the operator (what the user was called back then) to digitize a point more than once. The problem occurred so often, that the users called it double digitizing. This was clear to me, since when I wrote something, I was also its first user when testing it. And double digitizing was incredibly bad because it required extra work. And it didn't respect the user's input.

As time went on, I became the author of the graphic editor (GED) on the Calma GDS-II system. You may not recognize this system, bit it was used to design the Motorola 68000, so you probably know its effects. GED fulfilled the promise of digitizing the geometry within the system, although it wasn't unique, and we all considered this requirement to be obvious. When I wrote it, I paid particular attention to caching the user's input so I wouldn't require anything at all to be entered twice. Even a closure point. Because by then, tablets were being used (but with wire-connected styluses) and often it was hard to hit the same point twice. So, I invented a snapping distance specifically so you only had to get close to the starting point and still specify closure.

Constraint-based systems were still in their infancy in 1978.

Programming

As I did more programming, I became aware of another corollary to the principle of respecting the user's input. It was called the Perlin principle of modular programming: we hide the similarities and highlight the differences. Sometimes the similarities are simply encapsulated in classes or modules. Usually the differences become parameters to the modules, or the differences are wrapped up in the way you call the modules or use the classes. Here the user is the programmer and their input is respected in some ways because they won't have to write something twice. There is another way to respect the programmer's input: transparency. When an API (application programming interface) is complicated, then it becomes less transparent and the possibility of bugs can increase. On the other hand, when an interface is clean and well-defined, the programmer doesn't have to learn too many things, or understand the assumptions made by the writer of the API. All these things make fewer problems and also fewer chances for bugs, because the user's model of the code is very close to the way the code actually works. In this way, the programmers intent is better respected.

MIDI

Later on, in the late 1980s the user input became more complex. In 1988, I built a program for recording and playing back MIDI data from keyboards and drum machines. I called it MIDI '88 and it's a project that really has never seen the light of day, except for in my studio. Well, I did show it to Todd Rundgren once. And, of course, John Derry.

To get the MIDI data, I had to interface to a driver and write an input spooler. This spooler, implemented as a ring buffer (seen to right), kept all the data around without dropping samples and events. In this way, I was respecting my input to the keyboard and allowing me to record it. I recorded the events with a millisecond-accurate timestamp. And this was the beginning of my quest to accurately preserve the user's input and faithfully reproduce the user's gestures, allowing the style of the user to come through.

Tablets

When I first got a Wacom tablet in 1990, I sought to do the exact same thing: create an input spooler for the tablet data, and timestamp the data accurately. But with the Wacom, there was pressure data also that needed to be captured. And then there was tilt and bearing information.

The input spooler captured all these things.

But I soon learned that the data I got from the tablet wasn't perfect. It didn't really accurately capture the user's input at all. One source of the error was temporal aliasing. So I learned that input data often had to be massaged.

Some tablets had very few samples per second and others had quite a few. But, if you assumed the data from these tablets are regularly spaced, you could get kinks and ugly straight lines in your brush strokes. So I invented a limited FIFO that smoothed out these time-and-space irregularities. And I had to pay special attention to the tight turns in the path of the stylus. Changing the extrema of a brush stroke was highly undesirable since it made common actions like cross-hatching very hard. And sketching simply looked wrong if too much processing was done on the user's input. Usually, but not always, the extrema of the stylus' path was a place where velocity was at a minimum.

But, conversely, when the stylus was moving fast, less control was exerted on the actual position of the brush. So I could afford to use parametric cubic interpolation to create more points in the brush stroke in fast-moving sections of the brush stroke. This was a good thing, because there were fewer points per inch in fast-moving sections due to the fixed sampling interval: when your hand moves the stylus faster, the sample points are spaced farther apart.

All this made for smoother scribbling in Painter.

When John Derry came to Fractal Design in 1992, his enthusiasm for mark-making actually meshed quite well with my desire to accurately capture the input. It made Painter a very good tool indeed for an artist for these reasons.

We perfected the motto with respect to mark-making: to accurately capture the nuances of human expression and faithfully reproduce them, allowing the artist's style to come through.

It is this statement that I stumble through towards the end the video in my early post Creativity and Painter, Part 1. Ah, the early days.