The Miracle of the Paint Can

On September 2, 1995, I began work on a poster image for Painter 4. Our theme was decided: Painter Through the Ages. This was partly because I had implemented the mosaic tool. We began securing artists for the poster, which was to contain the usual 5 images. Our timetable was short, so we picked out some friends, Hal Rucker and Mark Jenkins from Rucker Design, the same designers that brought us the paint can design.

John Derry and Steve Guttman decided to do the other two pieces, because I was doing one of them this time. Remember, I was left out on the Painter 3 poster: too many things to do.

For Painter 4, I wanted to use the new mosaic tool. This was so I could show off the feature, partly. But, even more importantly, I needed to beat on it. I wanted to accomplish a real project so I could shake out the bugs in the mosaics feature. I had to be my own biggest critic.

I began implementing mosaics in the summer of 1995, while staying at the Hotel Skagen, at the tip of Jutland. Shortly after spending the summer solstice in Roskilde, after checking out the bonfires by the fjord, Ruth and I drove to Skagen to stay and rest. I had brought my black PowerBook 540c with me and I spent lots of time in the hotel working on mosaics. And we also spent time walking around the beaches at Skagen, looking for amber.

The mosaic problem was a very difficult one, because it involved chopping vector-outline shapes to other vector-outline shapes. Also, because mosaic has a grout, it also involved reliably creating outset shapes for each mosaic tile. These are hard problems, because they involve general polygon Boolean operations.

In July, I had tested the mosaic tool back in the Fractal Design offices in Aptos, using a comp'ed up version of the product, clipped from pictures of real mosaics from Ravenna.

You can see this to the right, here.

The letters' styles come from the Byzantine era, almost 900 years ago. It's actually amazing for me to think that something can last a thousand years.

This was just a test run, but I was seriously trying to create something with the tool. At this point, there was only very little control over the width.

I used this image as a clone source and sketched over it using the mosaic brush.

It's actually interesting to see how similar the tessellation is between the letters. Perhaps I used them as a guide.

With some trial runs, I turned back to the code and added the ability to vary with based on pressure, created a control panel for mosaics, and figured out what the modality of the mosaic control panel was to be. It was an unusual tool, because of the vector information that comes out of the brush as you draw. You get the point: lots to do!

Once September rolled around, I realized that I needed to be very careful and plan ahead in order to create a design with mosaics that was coherent enough to be seen as a Byzantine mosaic.

I drew a version of it on the whiteboard in the office shared by Tom Hedges, John Derry, and me. Suggestions from the various team members were made based on this strawman depiction of the idea.

Next, I created a small-size proxy that showed the basic organization of the piece. It was a very rough version. I was indicating white circles behind the heads, which showed saints in the original mosaics of the Byzantine era.

The paint can was to have a similar treatment, originally. Also, I showed John Derry (to far left) holding a burning ice cube, a reference to the theme used in Painter 2.0.

Initially there were only going to be four of us, but we thought to include the venerable Glenn Reid, who had just come onto the R&D staff at Fractal. So I had to modify the original design, but that was hardly a big issue.

The main problem was to get each of the figures in place. I decided to use layers to do this, because then I would only have to capture a photo of each one time. But I would have to make sure that each person was captured with the same lighting, of course.

So here you can see the original captures, done with a cruddy video camera, attached to a PowerMac 8100/80av. I supplied the facial angle for each person, and, as you can see, they were all being taken in the same location, so the lighting would match. The outer people (John Derry and Steve Guttman) I had hold their hands in a position of classical adoration.

I wanted the folds of the fabric in the shirts they were wearing to be visible, so I could use them in creating the folds of the garments that the figures were wearing. All of the portraits were done at appallingly low resolution, because it only had to be the resolution of the mosaic tiles themselves. They all worked except for Tom's frame grab, because he was staring at the camera in a classic thousand yard stare. Hmm.

Then I masked each of the photos and composited them in layers into a single image. This allowed me to move them around and adjust them to create a better pose. I adjusted their brightness and contrast a bit to create more visual cohesion.

I also adjusted each person so that their actual height was indicated as well. Tom Hedges was quite tall, so he got to be taller than the rest of us.

Here you can see how Tom Hedges' image sticks out because of his stare directly at the camera. But initially I thought I could work with it.

My next task was to choose the garment styles for each person. Their styles were taken from the mosaics of Emperor Justinian and his bishops and officers, depicted in the Basilica of San Vitale in Ravenna, which I have seen personally. My sash is actually designed based on that of one of his bishops.

But in mine, I replaced the cruciform figure with a 4 and a crude Fractal Design logo. My initial design for John Derry's robe was rather freeform. But you can see how I used the existing fabric folds to create the look of the folded garments. I extended them down using my brush.

I also adjusted Glenn Reid's height so that he showed the appropriate height. It also made his garments a bit more visible, which was another goal at the time.

I painted most of the robes on the night of September 12th, and I was up past 11 PM doing the brushwork. I used a small-radius airbrush that was adjusted to 90% opacity to sketch in the colors, which were approximated from those used in reference mosaics from the fourth century. I then used Just Add Water to blend the colors together and create smoother fabric. The gold background was added, a reference to the common practice in Byzantine mosaics.

But I had to fix that look on Tom Hedges face! I broke out the video camera again and persuaded him to stand for another portrait on September 13th, the next day.

The next iteration of the photo was done by 5:00 PM on the 13th: I had replaced Tom's head with a better frame grab that had a much better expression on his face. It was also focusing in the right direction, and I had reworked his shoulders to accommodate the new head position.

I placed an image of Painter's paint can in the image and put a nice glow around it, making it seem a little magical.

All I had was an image of the Painter 3 can, taken from a photo shoot we had done professionally. But I could see it all coming together. I made it unusually large to accentuate the fact that it was in front, and floating.

I was really not satisfied with John Derry's robes. I looked back at the San Vitale images I had in an old Dutch book on Mosaics I had found in Amsterdam the year before. I found a reasonable style to rework it into: a bishop's robes.

John is on the left. It was an arduous process reworking John's robes to make something that matched his style. I had noticed that the bishops often had a section in front drawn over and pinned near their collarbones. And on the font section they often had diagonal rectangular patches of color.

For John, I created a red pattern featuring a checkerboard pattern, one of John's signature style elements.

For Steve, on the right side, I knew I needed to extend the robes lower and complete the double-white borders of his sleeves. I did that, also creating subtle shadows under the hands to make them a bit more realistic.

I began to realize that real Byzantine mosaics didn't feature people with watches and glasses. But I wryly decided to leave them in, to give the piece an interesting tilt. And I had decided that a floating paint can was even more magical if it was floating above something.

So I created a red table with draped fabric, in order to make the floating even more realistic.

I had to put the shadows of John's and Steve's hands on the red velvet tablecloth, as if the shadows were cast by the magical light from the paint can.

At this point, I put in the halos as layers behind the people. I also put some Byzantine-style text behind the figures. The text is latin, and says "Natural Media Forever". It was kind of my Strawberry Fields Forever moment, so to speak. I had to be careful about the thickness of the letters, because I knew they would be rendered in mosaics. I also used my experience with the images near the top of this post to sculpt the letters. They were done three months before.

Above, a blue field with green circles suggests the vaulted ceiling of the Mausoleum of Galla Placidia in Ravenna.

I had to further extend John's and Steve's robes to the bottom of the image, which was designed to be the right shape for the Painter 4 poster entries.

It was a very lengthy task, and by midday on September 14th I had completed the proxies. But that was, unfortunately, only the beginning!

The next step was to clone the image using Effects:Esoterica:Make Mosaic which is still available in Painter 12, by the way for those of you who see this and want to travel down the same road). That night at about midnight, I started making the mosaic. The first thirty minutes were spent detailing the faces of John, Glenn, and Tom, and also the paint can. Then I gave up for the night.

The next morning, in the office, I started working on the image and I worked all day. At about three in the afternoon, it crashed and I lost all of my work for the day! I had finished most of John and his robes, my face and Steve's face. It was painstaking work. So I spent a couple of hours figuring out what went wrong: there was an error in the mosaic vector data structure pack and unpack. So it crashed on save!


To left, you see how much progress I had made. This image can be opened, but the mosaic data cannot be recognized. Imagine my chagrin!

Six hours of tedious, painstaking work was lost.

Anguished, I fixed that problem and went on. Since I and only I was responsible for the error, I had nobody to complain to but myself on this project!

Such is the life of a developer who uses his own tools on grandiose projects.

It is probably good that I found this error, though. Better that it happens to me than to a user of the product after it has shipped. In all, I found several bugs in mosaics during this process, and added several features that made it easier for me (and easier for future users as well).

Starting again from my only backup from 12:30 AM early that morning (to me the night before), I continued undaunted. Well, I would like to think I was undaunted but actually I was weary and crestfallen. And by 6 PM I had only finished additionally my face.

You see the state of it at the right.

Needless to say, at this point I started saving backups every hour.

I went home and continued my work there after a quick meal. I'm sure I was hell to get along with at this point.

Next I did the halos, which were fairly easy work, starting on the outside of the rings and filling in from there.

I continued to work, and by about 11 PM the night of September 15th, I had finally reached the point I had reached at 3 PM that afternoon when Painter had crashed.

If you look closely, you can see that I had to do it all over again, and made several different choices of placement, particularly with Steve's face. And the color of his glasses.

I was also much more careful in the placement of the mosaic tiles in the rendering of John's robes, which I wanted to be eye-catching.

You may be wondering why I left the faces and hands in grayscale colors. For the most part, this was what the Byzantines did in their mosaics. I did color the glasses though!

By the time I went to bed at about 12:30 AM, I had finished almost all of the central figures, as you can see here.

So far I had worked thirteen hours on the mosaic part of this task, and six of them had been wasted!

The strategy of saving once an hour was paying off. There were other crashes but they only caused me to lose a minimum of time. And they gave me an opportunity to fix other problems, as was my responsibility.

For the background, I wanted the glittery golden look of the Byzantine mosaics, and so I used color variation to create the look. I painted in a green background below, and finished Steve's robes.

Also, it took me several hours the next day to paint in the letters in the arch above the main figures. The letters were rendered in the crude, yet stylish typography used by the mosaic artists of the fourth century.

I still hadn't figured out what I wanted to do with the area above the arch. It was certainly unfinished.

At left is the state of the mosaic by about 5 PM on the 16th of September. By this time, I had spent about 20 hours of time on the mosaic part of this task.

I went home and, working from there, I spent another two hours recoloring the background at the bottom and drawing in the symbols above the arch. At the left, I put in a P4 (for Painter 4), and at the right, I put in a burning ice cube.

I cleverly allowed the dripping water from the burning ice cube to overlap the arch on the right hand side.

Two green diamonds, another common Byzantine symbol, filled the center portion above the arch.

I decided that the faces truly did lack some color, and so I added in a pale skin tone to their shading. Each mosaic tile has its own color assigned to it, and you can change tile colors after you place the tile.

I painstakingly redid every awkward mosaic section I could find, to make it more like a real artisan had placed the tiles, one by one, by hand.

Even still, there are things I would do to improve this picture. There is one errant green tile just next to the top of the U in NATURAE. That should be fixed!

By 6PM on September 17th, I was done working on the mosaic portion of the task. With some 22 hours of work on the mosaic. And probably ten or so working on the proxies before.

At this point, my main task was to resize the piece to be used at the resolution required by the poster. This was possible because the mosaic was vector art. It was stored in sub-pixel vector locations and rendered from that form into pixels.

So all I had to do was resize the picture and the mosaic art was also resized in place.

Then I used the mosaic information to create a mask (using Render Tiles Into Mask) and that mask was used to create a three-dimensional look to the tiles with Effects:Surface Control:Apply Surface Texture.

Whew! Only 29,000 tiles later, the mosaic was finished!

Painter 4 was a real success for us at Fractal Design: the world had never seen anything quite like it!

And, thankfully, several bugs were fixed in the process: at my expense!

Such is the life of a developer.


Here is an interlocking P and 4, designed based on the interlocking royal monogram of king Christian the 4th of Denmark. I saw this in the Cathedral at Roskilde back in June. A mosaic monogram was the result.

Ahhh, we were clever back then!

The Most Useful Painter Feature

In early 1993, we were finishing up Painter 2.0, I had several things on my mind. The Van Gogh and multi-bristle brush styles were made to be real-time.we had added sharpen, resize, soften, color separations, and previews to most effects so you could visualize the results before you committed them. Saving the user time and allowing for more flexibility was the theme we were working to implement. Painter 2.0 also implemented full scripting and playing back of scripts at a new higher resolution. New effects like apply lighting, glass distortion, and marbling brought even more new looks to designers. In short, we had expanded Painter on all fronts simultaneously.

Painter was being finished, debugged and cleaned up, mostly by Tom Hedges, when I broke from the development process to work on new features. This didn't sit well with Tom, but by then he had learned to give me a bit of leeway in my implementation. Mostly because I was on a roll.

We were proud of our new work on Painter 2.0 and I knew we would have a great story, and plenty to sell. But rust never sleeps, so I continued development.

Back in 1984, when MacPaint came out, it introduced the concept of a floating selection. You select, and then click into it to move it around. In April 1993, a friend, Daniel Clark, asked John Derry and I if we could implement multiple floating selections. If each one was floating and on a separate layer, then a lot of the design work might be made easier.

That was the beginning of the process of implementing layers in Painter, the most useful feature ever. We had heard that Micrografx had implemented multiple floating selections, but we found that their selections weren't antialiased or sub-pixel aligned. They had no list of floating selections. Accessing one was tough, and required sequencing through all of them. I found it to be a useless approach. They didn't even allow you to paint into the mask of the floating selection and extend it.

So we began working on a better way to do it. We knew that the users needed to have continuous tone layers with soft-edged masks, and sub-pixel accuracy to the mask edge. Fortunately our friskets (what we called selections, and which we're used for airbrush masking) already had this. Our feathering used a real Gaussian convolution.


And the result was Painter X2. It began working about 9 days after we talked with Daniel Clark.

We called it Painter X2 because we felt that multiple layers easily doubled the user's ability to design more freely. In those days we called them Floaters. The name was a mistake, because eventually Layers became the standard jargon for floating selections. I implemented grouping of layers, and a nice list with an open/closed state for the group of layers. I also added compositing methods, ways of compositing a layer with what was behind it.

Even more interesting was the Portfolio, a place where layers could be stored and reused, like clip art. This was John Derry's idea. We figured that designers had objects that could be used again and again, and that, given an original layer, many things could be tried on it. Without destroying the original piece.

In fact, it was this philosophy that we were trying to further: the ability to try things and to change your mind again and again and again. Picture a designer, a client, and an art director. The designer has to keep their options open because they need multiple designs to show their client using the same basic artwork. Also, the designer needs to keep their designs fluid to quickly respond to suggestions and design decisions of the art director.

This signaled a move of Painter towards the design ethic more than ever. Not only could you use the traditional tools of the designer, but we could also emulate paste-up better than any other application at the time.

It turns out that Photoshop introduced layers in September 1994, long after Painter X2 shipped.


Our logotype designs for X2 were bold and interesting. Here I show an X2 sketch that featured Neuland Inline-type letters and an early catchphrase, true 2 form. We were always thinking about the catchphrase. The concept of changing your mind again and again made it onto the box.

As shown above, Painter X2 came in a box, and was sold as an expert extension to Painter, intended for the expert designer. Of course, the features were all rolled into Painter 3 when it came along. And Painter X2 came so soon after Painter 2.0, that we were struggling to get a way for users to justify the expenditure. We had that problem a lot, when you consider Painter 1.2, Painter 3.1, and Painter 5.5 were all intermediate releases. We really wanted a new release to be loaded with new groundbreaking features, so we often saved the new numbers for what we considered to be major releases.


This sketch shows that we were considering our new features to be industrial strength, with a metallic beveled-edge logo form and even rivets in it. Other concepts included the atomic X2, where electrons whirled around the letters. And, strangely, even a radiation symbol with X2 underneath it.

What I was thinking, I will never know!

Perhaps there is even a hint at the biohazard symbol. But I guess I didn't quite know how to draw it. And that was probably a good thing.


In this sketch, we see a part of the final logo form for X2, with it's angular letters, the raised 2 with a triangle below it.

Other things were tried such as negative space letters, vertical alignments, and other such crazy treatments. I like the flying paint can that's dripping paint onto something that looks quite like a Mac IIci.

In the end, the angular letterforms and the beveled metal look won out and we had a logo form for X2. But John Derry was convinced we needed to do something new with the paint can. Not necessarily with the can we were actually shipping, but something that made it more animated.


Some of the ideas were just plain wacky: popping it's lid, a stir-stick with a tagline stir up your imagination. A can dripping with paint. And all of them with faces! I wasn't sure where he was going with it, but we did consider it for quite a while.

One of the cans is astonished, and exploding with paint. Perhaps like a user exploding with ideas when presented with the new range of ideas that come from the freedom to design like never before.

By the way, the stir stick has the letters OSH on it, which stands for Orchard Supply Hardware, a hardware store in Silicon Valley and also in Capitola and Watsonville where we sometimes went to look at paint cans and paint in general.


OK I will admit we were a bit out there! At the top is a triangle with an eye in it, like on the US dollar bill, but it's carrying paint cans. In the center is a paint can running away from freshly having applied a coat of dripping paint to a box clearly labeled ADOBE. At the bottom, a lens in a frame, part of the symbology used on the Adobe Photoshop box, with a pitchfork and a barbed devil-tail, examines a roll of 35-mm film on the ground. And it seems to be dragging a mouse. Note the blank space behind it where the frame has been lifted off the wall.

I think all that time designing and coding was taking its toll. This is John Derry's sketch, but I'm sure I felt very much the same at the time.


We even started thinking about X3 as you can see. Like I said, rust never sleeps. I think the most interesting thing here is the bill of floating selection rights, the declaration of floating selection independence, that your floating selections at no longer slaves to the deselect command (which previously always dropped the floating selection back into the canvas).

We were definitely onto something new!