Different Perspectives

You can look at things from different points of view. And this can help in almost everything you do.

Look at each new point of view as a degree of freedom in understanding. The more degrees of freedom, the more independently your thoughts can move.

With each new degree of freedom, concepts that you once thought of as constant can now change. Perhaps things you once took for granted can become emergent truths of your new frame of mind.

Or become surprisingly refutable.

One thing is certainly true: if you don't at least try to look at things from different perspectives, you become stagnant.

Degrees of Freedom

Look for a dimension along which there is some give and take. Perhaps things are not completely fixed and immutable.

When technology changes, for instance, slippage occurs and a new dimension of possibility opens up. This can cause disruption.

Once it was cheap to send megabytes through the Internet, intangibles like music and movies could be easily sent.

Record stores and video stores died.

So find that degree of freedom before it finds you.

Human Interface

With true-color displays and greater ability to compute, transparency became possible in interfaces. A new dimension opened up: opacity. But in this case, a little overuse quickly showed that things could get confusing.

Nonetheless, in modeless situations that are optional, such as heads-up displays, inspectors, and the like, transparency does still get used.

In UI, transparency really refers to whether the use of a control is obvious. Aside from this attribute, ergonomics and simplicity are also paramount design considerations.

To make an interface item cool without regard to its function is simply gratuitous. I have certainly done that from time to time, but I suspect those days are over. Or are they?

The use of three-dimensional interfaces is another interesting design consideration.

There have been a few attempts at creating fully three-dimensional interfaces, such as SGI's button-fly interface. Our approach to three-dimensional interface is much more sophisticated today, and integrated into our workflow.

Two-year-olds and grandmas alike have been brought to the computer as never before by glass and multitouch.

Nonetheless, Minecraft is quite popular, as is Spore and The Sims, all three-dimensional worlds manipulated directly. It's important to teach three-dimensional thinking.

And it is true that some interfaces appear to be three-dimensional, like Apple's cover flow. Put simply, magic counts.

But form still follows function.

Fit Together

Things must fit together and dovetail perfectly. Seamlessness counts. When human interfaces are inconsistent, something feels wrong. Even to two-year-olds and grandmas.

Both workmanship and workflow fall into one category now: fit.

Once again, when you consider fit as a guide rule, suddenly things might reorganize themselves in a wholly different way. A new pattern emerges because you discovered the right degree of freedom to work from. You found the right perspective.

Did things ever line up quite so right?

The value of your creative output is at stake.

Three-Dimensional Design

It takes awhile for a design to unfold in my mind. It starts with a dream of how something can best function, and, with real work, iterates into the optimal form for that workflow. Yet it's not until it assumes real form that I can say whether I'm satisfied with it.

When designing, I often consider the benefit of workflow I have experienced in the past. Consider maps. When I was a kid, driving across the US in summer, I collected maps from gas stations (back when they still had them). I was trying to collect a map for each state. This is when I became familiar with the basics of functional design. A map had to be compact, and yet describe many places with sufficient accuracy for navigation.

I observed how both sides of a map were useful for different purposes. How many locations of interest were indicated with icons. A legend indicated what the icons meant. This was a time of real curiosity for me. Of essential discovery.

Such hobbies as building geodesic domes and technical illustration kept me focused on function for the longest time. But eventually, in high school, I discovered Graphis, an international magazine of graphic design. This struck a chord with my innate drawing talents. And suddenly I was also focused on form.

And then it was impossible function that caught my eye. At Fractal Design, I continued this design philosophy. Here is an illustration from those days, reinterpreted in my modern style that expresses form. A wooden block penetrates through glass. This is ostensibly impossible, of course, but it was in tune with my sense of materials and their simulation in UI.

At the time, I was lost in a maze of twisty passages, all alike: the labyrinth of UI design.

John Derry and I were concentrating on media, and had been since Dabbler introduced wooden drawers into paint program interfaces. Like the paint can, it was a return to the physical in design. Interfaces needed something for users to conceptually grab onto: a physical connotation that made the interface obvious to the user.

One project I was developing at the time was Headline Studio. This was an application specifically intended to create moving banners for web ads. It concentrated on moving text. So when working on a hero logotype, I sketched out these letters. The idea was that, in a static illustration, the letters might appear to be walking in. And the addition of the cursor was a functional cue. This ended up being pretty much what we used.

Every bit of Headline Studio was designed in minute detail. This included many designs that were not used. For instance, I show here a palette that was rejected because it was thought to be too dark.

This brings up the subject of visual cues. To visually cue the user to thinking of a palette as something to adjust the image, we chose simpler designs that those we used for windows. But sometimes we went overboard on palettes, as you know from the Painter UI design.

In the Headline Studio timeframe, we started thinking about three-dimensional UI. We considered different three-dimensional functions. For instance, we considered the window shade.

A window shade is hidden when you want to see out, and you pull it down when you want to block the view. At the time, there was a trend to make a window collapse to just its title bar when when you double-clicked it there. I considered that to be an extension of the window shade.

And by extension, we could turn palettes into window shades so their controls could be accessed only when they were needed.

Eventually this technique was replaced by the expanding list with the disclosure triangle. We liked this because when the list was closed, certain crucial data could be displayed in the list element. The user could thus discover the current state of the most important controls in a quick glance, even when the list was closed.

You get a bit of that here where the current color is displayed even when the palette is rolled up.

And like a real window shade, a small amount is shown to grab and slide down. This sort of technique would work even now in the multi-touch era.

You can also see a nod to the three-dimensional look, because the palette bar has depth. This makes it more sensible to the user that it can somehow contain the rolled-up shade.

The real cost of producing a three-dimensional UI is the need to develop an visual language of controls. Take for example the humble check box.

It has been a box with an X, a box with a check coming out of it, even a simple bump that becomes a different-colored indentation. Eventually the box with the X became a close square in a typical window (though Mac OS X uses little colored balls. Which really are very nice, I think. The close ball uses an X, of course).

But the check box is really an on-off item. It could easily be a ball in a box that just changes color when you tap on it, for instance. On and Off? Red and Green? Or it could be a 1 and a 0.

You become endlessly mired in an array of choices when it comes to this necessary visual language. And some things just don't make sense. Eventually we came to the conclusion that objects were more useful than icons. Because the objects become more readable and their behavior is already known.

When we came to sliders, we realized that they were also used as visual indicators. Having played a pipe organ from time to time when I was a teenager, I found that drawbars might make a nice physical metaphor.

Here is a prototype for the actual sliders themselves. One of the metaphors used was like a ruler with a dot at the end. This dot marked a grab-point. You could tap and grab at that location to extend the slider to the right. This would increase its value. The marks at the bottom give you an indication of the magnitude of the slider's value. Another more drawbar-like metaphor is the glass semicylindrical rod. You can see its magnitude based on the number of lines you cross (and which refract into the rod as you drag them over).

This was an example of form leading function, but it was compelling enough to experiment with. If you turn this one into a real control, it must be possible to have several of them, like drawbars on an organ.

Another way to look at them is as a bar chart. Each parameter has a magnitude that is indicated by the length of the glass rod. The interface is three-dimensional, as you can see. The section to the left of the bars is thick enough for the bars to be embedded into.

Probably the inclusion of even more shadows would make it visually more interesting and also more easy and obvious to interpret.

These are re-drawings of my original sketches from 1999, colored and rendered using a woodcut look.

The idea of using a sticky note that sticks out of the edge of a three-dimensional pad was one simple physical construction that seemed useful. But how? In real life it is used to mark a place. Sometimes it is used to specify where in a large document you need to sign.

Either way, it was similar to a bookmark in the web: a quick way to get back to a specific place that you want to remember

The pad signifies a multi-page document, like a PDF. So, how might this be envisioned in actual use? I actually drew out a few examples. And here is one.

This shows an idea for a storyboard project. The storyboard is the multi-page document, with frames showing in sequential order. Different scenes might be marked using colored tags. The blue arrows allow the user to sequence through the pages in the normal linear ordering.

Probably the colored tags would live in small piles like a sticky pad. The user can click and drag a sticky note from the pad to tear one off and continue to drag the note to the document for use as a placeholder on the current page.

A nice, clean three-dimensional interface for non-linear access to a linear document!

Here's another three-dimensional interface, used for a document window. It's kind of a gratuitous use of 3D though, as you can see. Still, it features an infinitely thin document, like paper, stretched in a frame made up of the scroll bars and the title bar.

Perhaps the red item in the corner is a close box.

Down in the corner is a kind of tactile device used for adjusting the window size. All of these parallel what a window has in it right now, of course, and has always had in it.

It's all about using a different visual language for the UI elements, which is something you have to choose before developing a UI in general.

Here is another, more generic example, devoid of the accoutrements of a title bar. It shows that it might be possible to put transparent stuff into an interface as well.

It is unlikely that I had any idea why I wanted a transparent element in the interfaces (I have colored it green to single it out). It is another example of form leading function.

I am still interested in how such an element can be used, though. It does look cool. It is also possible to make the document itself transparent. This might even be a nice frame for a layer in a touch environment. Consider touching the layer, and then having some controls appear around it. In this case, the three-dimensional interface makes more sense since they are like objects that appear on touch command.

But you can consider elements like the blue arrows in the storyboard example above. They could be made transparent easily, with no real loss of readability. And that would look cool as well.

And what, I wonder, is the shadow being cast on? The elements seem to float in space in the example. It is an example of a visually interesting impossibility. If we were going for true realism, this wouldn't qualify.

And that, in a nutshell, is one of the endearing qualities of three-dimensional UI. It doesn't have to simulate something totally real. It can be magic, simply transcending reality.

The amazing thing is that, as a user, you still get it.

When it came to the Headline Studio packaging, I needed to come up with a way of showing animation on the box: a completely non-moving way of showing animation. I came up with several ideas, but this one stuck in my mind as a good way to show it.

Once again, three dimensional design becomes a useful tool, because it helps to replace the missing dimension of time.

Overly-ambitious Projects

Sometimes we underestimate how difficult a task is. Like nearly all of the time. But this is not why we choose overly-ambitious ways of solving problems. So let's talk about why.

I have tackled several overly-large projects and sometimes I have chosen to split them up along natural lines. When handling large projects, not all of them are easily split up by top-down decomposition.

Let's first look at a project that can be split up: Overhaul Painter's user interface (UI). First off, why did we do it?

Painter's interface was getting too complex, and needed organization. It also needed to be easier and more accessible. More items in the interface simply called for a better way of organizing it. So this was a redesign borne out of necessity.

But by then with the features of Painter 2.0 and also Painter/X2 in one program, it looked like a big task. We needed to split it up.

This could be divided up, a little bit, and, in the development time frame we had at our disposal, John Derry and I sought to split it up between design and implementation. Just so we could get to the independent implementation part of the job. It sounded like an easy task: just basic planning. But it wasn't easy at all.

Redesigning the Painter 3 Interface

First we examined the items that needed to be in the interface. Then we organized them according to function and sometimes according to property. We worked out some basic areas: choosing brushes to use, choosing materials to paint with or design with, adjusting brush properties, and other. You see, things like selections (which had just been renamed paths from the original term, friskets), layers (which were called floaters by some unfortunate consequence that they had come from floating selections), and scripts had no obvious organization.

We were looking at a five-color-icon organization for these palettes. Based on the problem of icon hell: too many icons made for a totally indecipherable interface. I think all interface designers go through this.

We knew colors could be detracting, so we laid out the palettes in predominantly dark gray, with a texture to them. We wanted things to be three-dimensional and basically understandable because they were analogs of the real thing. It is possible that color was a distracting feature of the user interface and added to the complexity and detracted from the accessibility. The modern version of Painter (12) use icons that are grayscale instead. The texture that backs the palettes is a precursor of skeuomorphic design.

You see the Materials palette. With colors, papers, grads (graduations), sets (color sets) and weaves. OK the weaves was less likely to be used, for sure. But the color picker redesign was clean, effective, and artist-centric. No problem understanding it at all. Actually, it became an iconic symbol of Painter.

The brushes were actually pretty easy. We figured five brushes was a pretty common number of brushes to work with. And may even be overkill. But these were brush categories, not variants. So this was a mistake, I think.

You see the Brushes palette, but this is an early mock-up with the pop-ups not yet functional. Our idea of putting the method up front was probably wrong as well.

But the topmost pop-up was the brush variant, and this was certainly a good thing to have here.

As time went on, it was the individual customized brush variants that the artist actually used and often switched between, and that should have been the list to choose from. Painter 12 gets this right, of course. Good job, Corel!

The brush controls turned out to have a lot of sections. But in Painter 3, we had an idea of having five icons per palette, so we worked the properties into ten areas. Five were in the Brush Controls palette. This became a fairly good split, with size, spacing, bristle, randomness (now called jitter), and nozzle sections, for the image hose.

But there were many more controls, and these got organized into the Advanced Controls palette. This contained sections for the rake brush, the color well (which affected mixing of paints in the canvas, and pick-up of color into the brush), looks (brush looks, which were a customized variant plus a paper texture), sliders (which allowed for many features to be controlled by the expression contained in your brush stroke), and water (for the water color brushes).

So far so good? No. We had already made a lot of bad decisions, I think. The next palette was the Objects palette. Here you can see the sections for paths (selections), a path list (for storing paths), floaters (this was the prototype for the portfolio, a place where you can store layers for reuse within a project), the floater list (like a layers palette), and scripts.

Painter was by now recording everything you did. So you could play it back, and also have a backup of your work. Of course, it got recorded to your Painter folder, in a Scripts subfolder. No - we weren't spying on you. We called these Sessions in Painter 3.

Materials (in the Materials palette) were an interesting case to the interface. Papers, Grads, Sets, and Weaves were all materials.

The Papers section shows a drawer, a user interface drawn from Dabbler and created by John and I. The current paper is shown, and some controls are present to adjust it for use.

So materials used a drawer with material swatches in them. The drawer face acts like a Recent Items list. So the most common items will be present to use again and again. When you open the drawer, you can access more, and also go to change the library. Eventually the Materials palette got bigger, with patterns and nozzles making it a bit more useful. But I think that was Painter 4.

The final palette was the Controls palette. And for this palette there was a separate pane for each tool. So you could think of this as a mini-inspector for tools. Useful features were located here. For the brush, the opacity and grain sliders were present. Also freehand vs. straight-line radio buttons were there so you could easily make straight lines. The color swatches were useful for switching between the foreground and background color.

It was really a whole redesign.

This took most of the time. Then individual elements were designed by the pixel-meister (John) and I designed the clever way of putting it all together, called the complied interface file.

So finally, we were able to split up the work. But in so doing, we had bit off more than we could chew in the original design and decision work.

In Painter 4, we did redesign it a bit, and it was more streamlined. But the redesign was also motivated by having more things in the program.

Once we had it planned out and split up, the implementation took at least a month of grueling work by both of us.

Multiple Undo

One project that was way too ambitious was the implementation of multiple levels of undo for Painter.

This project was so large that every single part of Painter had to be redesigned and retooled so that each operation could be undone. This was particularly difficult in Shapes, the vector illustration layers built into Painter during version 4, which became a graph theoretic problem of unprecedented proportions. This was because of grouping, ungrouping, and the operations of cutting up and joining shapes. Making holes. It just went on and on.

There wasn't any clean way to split this project up. But I did implement a set of primitives (kind of a library and an API) for building undo into each operation.

At one point, I sat down with another programmer, Priscilla Shih (now Priscilla Shih Cinque) and we went over huge areas to the program that needed to be retooled for undo.

It was kind of hopeless, we realized. Now there were two programmers that were overwhelmed by the problem. And I did a particularly crappy job of describing the problem. Perhaps it was just too incoherent. I liked in those days to keep a huge amount of details in my head and I made a gigantic list of primitives to tackle for each of us.

In the end, multiple undo took two revisions of Painter to implement it properly (versions 4 and 5) and eventually it was possible to simply implement undo on each new feature we added. I'm sure the legacy of multiple undo sticks to it with every revision that gets done.

In a future post, I will discuss the dangers of add-on programming. The issue of undo will return as a cautionary tale of something that should be designed in from the very start.