Creativity and Painter, Part 2

The War for Painter 6

In early July of 1999, John Derry and I got together with Metacreations marketing people to create the theme for Painter 6. If you remember, this was a major revision of Painter. It had the next generation brush engine where every bristle was simulated. It had impasto, a new 3D-shaded way of viewing your painting and building up paint. It had airbrush, with full support for 6-axis control of the Wacom pen and a new spatter model. Essentially, it had all the features of the brush testbed I had been working on for a year or so on my own. And note that impasto was Tom's Big Feature.

Something this big in terms of creativity was, for Painter, a real return to painting and brushes.

So John and I put our heads together like we did for Painter 3 and 4 to create the best possible image for the product. We actually tried on several different things, and I have the notes to prove it!

But the funniest thing we did was to do a write-up on war and other violent themes. How could that apply to creativity? How could we do such a thing? We were that far out. When these ideas got drawn up, I was present, with John Derry and Mary Zimmer.

Among the catchphrases, themes, and ad ideas that we circulated were:

• make art not war
• the artist's finest hour
• your creative victory
• a paint can with a bandolier of brushes and pens coming out of it
• Painter 6 your personal creative weapon
• win the war of creativity
• the brush is mightier than the sword
• better brushes than bombs
• target new clients (perhaps for the pro magazines)
• "camo" paint can (manual is a field manual)
• iwo jima photo with soldiers raising a brush instead of a flag
• smart bomb photos. closer... closer... splat! painted lush oasis.
• baby with a toy gun in one hand and a brush in the other. theme: which would you rather do? most of us decide pretty early which we prefer
• Painter to the people (militant/revolutionary)

Yes, John and I were at the limits of poor taste, there's no doubt. At some point we decided against it, because marketing just couldn't get with the program. I'm glad they couldn't.

What other ideas did we try on the way to the final theme?

The worst was this: The phrase "we've got your art" in ransom-note letters, clipped out of a magazine. With a picture of an unfolded handkerchief. Inside is a severed brush head with some red paint.

OMG.

At some point, we got back on track and started to churn out more mainstream ideas.

• pry open your creativity (style, art)
• only Painter
• always Painter
• paint unique
• uniquely Painter
• paint distinctively
• originally Painter
• Painter quality
• Painter for life
• pure Painter
• Painter power
• Painter style
• Find Your Style
• Beyond Creativity
• paint with an attitude
• show your style
• advocate the artist
• pop the can (nice 3x3 arrangement)
• spill the paint
• the canvas is your playground
• do the can thing

Personally, I liked Beyond Creativity. But that was just me. In the end, we used Only Painter, which signified uniqueness and an otherwise unattainable quality that marketing was going for.

For the paint can art, we had some ideas as well, but we kept coming back to showing a nice selection of tools, rendered by artists. I personally rendered the pencil and the technical pen in the set that ended up on the can. John Derry did the brush (using Impasto) and the airbrush as well. A third artist (whose first initial is B, but I can't remember much more) did the palette knife, the scratchboard, and the felt pen. In the end, my technical pen didn't make the cut. Oh, well! But the can design started with John Derry and myself.

In another stroke of creativity, we had dreamed up a four-way split can with the tagline Find Your Style. An additional catchphrase was With Painter 6 - you can. We were clever in those days. But it seemed like it would require a different form factor for the poster, and marketing was against it.

In the sketch you can see that we went through a few catchphrases before that one stuck.

We were so full of ideas in those days that we literally threw away 95% of them.

Net Painter

At Fractal, we were interested in collaborative painting, and I pitched this to Tom Hedges in 1996. He spent a huge amount of time working on a version of Painter for Internet collaboration we called Net Painter internally.

It was based on a low-level protocol called UDP, that TCP/IP is based on. At the time, Tom worked with a friend named Stuart Cheshire to figure out the protocol and the main means of transmitting the data and achieving reliability. It's odd, because UDP means "Unreliable Data Protocol". They explained that you have to first assume it's unreliable, and then verify the messages that got sent.

Net Painter was to allow the painters to capture the moment and draw, and then others could interact and work on the same document. It had a stoplight protocol visually, so you could understand what your status was. Also, it had a list of users that you could see, like users in a chat list. When their stop lights went yellow, it would indicate that they wanted to draw. And text would come from each user as well, so you could have a conversation. Not necessarily in a chat mode, but, really asynchronously. I used it many times, and we used it across the ocean on occasion.

So it became an environment for collaboration across the entire world through the Internet, which was rather low bandwidth back then. To implement it, we sent script items across the net, like a recording in Painter. Painter at that time had already begun to record every nuance of your artistic process, so you could always play back your session, remember? This was actually part of Net Painter: a secret plan.

But Tom was always having difficulties back then with his lymphoma and also relationship problems with his wife. And this project, though it worked, was never fully realized I think.

So Net Painter was kind of a whiteboarding application.

We wanted to be able to ship files across as well, so we could share more complex things.

I believed the idea of a stoplight from Tom's Net Painter was stupid, though. Why bother to have something that halts creativity, when you can be spontaneous! This is the way I always wanted it to be, with two people or more working at once. It was a curious prototype.

Sure, the time lag can make collaboration hard to do in real time. And that can cause other issues of confusion as well. But I think that most of the time, this will not be true. It is for the same reason that most source changes on a multi-developer system can be disambiguated and merged back in. Only on a painting, which is two-dimensional, the collisions might indeed be less frequent.

2011 - The Year of Steve

2011, now almost over, was the year of Steve Jobs. These are my memories of Steve: the times he called me, the times I sat with him and talked, and a surprise visit. The memorial.

I have included the (somewhat edited) text I sent to a friend in August through October of this year. I have added more that I can remember, also

Memories

I remember Steve Jobs.

The first time we met, was an event set up by Kai Krause. Kai wanted to show our interface improvements to the big kahuna. I showed my idea processor (I will write a short blog post on it sometime).

But Steve's first words were that it wasn't scalable, and he hit the nail on the head. He was like that. His appreciation of Kai was that he was there to show stuff off, but that he hadn't really done much. He got it instantly, he told me later. He bided his time, and soon the show was over. He apparently sensed that Fractal and Meta Tools had just merged and that the personalities weren't really compatible.

A few days later I was in the Seybold show in San Francisco. Our booth was huge and overflowing with interest from the crowd in the Moscone center. We were showing Painter 6, with the new brushes and effects. I was in our small press room talking with John Derry, behind a closed door, in-between press appointments, and one of our marketing people ushered in Steve and then left us alone. Steve said that we had to talk, but in a more private place. He was so impressed by what we had done and he wanted to make sure our products stayed on Mac. We made an appointment for me to come into Apple: his real intention.

When I did, I met with Steve in his office this time. I was overdressed, in a button-up shirt and dress slacks. He was in shorts and a T-shirt. Birkenstocks. We talked. He said: Apple really needs people like you and I think I can make you an offer you are going to like. He was very nice. We talked for at least a half an hour about the state of the Mac and life, and stuff. I set up a meeting with Avie Tevanian and also with Bas Ording, his UI guru. Avie was a shark at negotiations, and I was a bit put off. He ran R&D. Bas was a fast-thinking and brilliant UI designer and he and I hit it off instantly, trading UI stories. My demo went well, and he shared some of his thoughts with me. Bas and I still speak today sometimes though we work in completely different parts of the company.

Steve kept calling me at home. And he gave me his home number. I did call him there once. This first time, it didn't work out. I was unimpressed by Apple's advances in the stock market. They actually went down while I was trying to decide. Painter called, so I had to do it.

But it was a rush to get a call from Steve Jobs at home.

Later, when things at MetaCreations changed and it became known that we were divesting the software, he called me in my office at work. By then I was the CEO again and working to sell Painter. I went in to meet him again and we chatted about these things, the world, and another offer. Once again, to Avie Tevanian, and a new offer was struck. Avie has since retired (I assume he's a rich man).

I had to turn it down since the terms of the Painter deal with Corel required that I consult for them for an unspecified time. And I morally had to do it and not leave my people in a bad place. Though the dismantling of Metacreations was unpleasant work, I didn't want to entrust it to someone without a soul. I had to do it, because of the history of me and Painter and my people and I couldn't be selfish.

So I did my consulting gig with Corel, and made Painter 7 shine with woodcuts, and liquid metal layers and all the good stuff I always do. After the consulting gig, I decided to retire (!) for a year. At the end of it, I went to Apple with hat in hand and asked for a job. I met with Peter Graffagnino, now retired, and eventually I found myself at Apple with a damn good deal. Steve wasn't involved this time, but I noticed when Peter was talking with Steve (translation: Steve called Peter on his phone) noticeably about new products, that Peter would then mention it to me, because clearly I was part of the conversation.

One day in 2004 I arrived at Apple, waiting to make a left turn on Mariani. Steve drove up behind me in his silver SL-55, waiting for the same light. He apparently noticed I was driving my SL-600, I saw him smile a bit and his head tilted to the side a little. I think he was trying to figure out who I was. That might have been tough, since I had just cut off all my hair, so I doubt he recognized me just then without my then-iconic beard and pony tail. Even so, I'm sure he was quite content to know that his more modern SL had at least 50hp on my V-12.

I have seen Steve several times while working at Apple, mostly passing in the halls, or saying hello in an elevator. I always appreciated that he didn't shave. With my busy life, I often don't have the time either. I last saw him in the latter part of August 2011, walking through Caffe Macs in the Infinite Loop campus. He was quite thin.

But, yes, I remember Steve. And the world has felt the effect of his wisdom.

Steve Passes On

It was October 5, 2011. That morning things were tense at work: a disgruntled employee at Permanente was responsible for a shooting at the quarry down Stevens Creek Boulevard that left several 3 dead. And an attempted carjacking left another woman shot but not fatally so, thank God. They found him the next day and he was shot and killed. Apple security had wisely warned employees not to leave their buildings and to keep exposure to a minimum. So it was a tense day. When I left work at 3:45 this was the big news. When I got home, I checked my iPhone and that's when I got the company-wide email from Tim Cook, our CEO. It was a sad time.

Coming to Apple the day after Steve dies was a bit like walking around near the epicenter of the quake of 1989, which I actually did. You expected to be able to survey all the damage. But there seemed to be a big difference at Apple. In the quake, power was off, gas mains were shut off to prevent explosions. But at Apple none of that kind of disruption seemed to have happened. Sure, people talked about it and brought out their memories of Steve. I certainly did. At no place on Earth did Steve have a more profound effect; but of course his effect is felt everywhere. You could tell that people still had the fire and resolve to make great products. Thanks to Steve, it is in our DNA.

There were a number of satellite trucks and interview tents set up outside Apple on Mariani. But I seriously doubt any employees were talking to the media. At 1 Infinite Loop, the flags were at half-mast. There was an impromptu memorial with flowers and pictures and laptops set up in the gardens. There was a steady stream of people walking by.

It was a time for reflection, a time for remembering.

The Memorial

The following comes from a rolling commentary that I sent to a good friend. Then more that comes from a wrap-up after a lunch with friends.

Fifty foot posters of Steve adorn the buildings of Infinite Loop. Songs play on a speaker system of unprecedented size in this venue, which is the quad between the Infinite Loop buildings. Thousands and thousands of people stand here trying to see anything. Pianos and mixers are in evidence for tributes by I don't know who yet. Guitar amp stacks up on the stage. Security was tight coming in so there is only one entrance to this vast space which is teeming with Apple employees. It's like a Rolling Stones concert. That's how crazy it is. A helicopter passes overhead.

Posters: Steve at 25. Steve with a Macintosh. Steve as I knew him at about 50. Songs: Tupelow Honey by Van Morrison. Roll Away The Dew by Grateful Dead. I expect Bob Dylan, one of Steve's favorites. 60s songs I can't even recall perhaps Jan and Dean. Idyllic songs. Dream a Little Dream with Louis Armstrong's gravelly voice and some female vocalists from the 40s. O-Bla-Di-O-Bla-Da by the Beatles. A song by the Rolling Stones is it Rainbow? Don't know the name but it's recognizable, mostly instrumental at the start, for a minute. Some Jack Johnson is playing now, with nice acoustic guitar. 

Security warned no pictures no recordings on the way in. So I won't do that. [Well, I did, but I saved them on my camera and include them here for the first time.]

I'm standing on concrete. Not good for my hips, but I wouldn't miss this for the world. I knew Steve. I have grieved for the guy I knew. Apple senses that we all need this, and they are right. 

Now Beatles In My Life is playing, sounds like Steve's playlist. This was Tom Hedges favorite song too. I'm about to cry. Dylan is playing now. Looks like I was right. Highway 61. 

The helicopters were warned off. But the moon is still in the sky. Of course. Between the clouds. 

Now they are playing the Beatles Get Back. The cell site is swamped. I think I'm lucky the first email got out. The company wi-fi is also maxed out. These events are too big for the usual solution. Joni Mitchell Clouds is playing now, a sad song. It's life's illusions I recall. I really don't know life at all.

Rolling Stones' I Miss You. More upbeat, thank God, otherwise I really am going to cry right here between all these people. Here Comes The Sun with George Harrison's voice and the Beatles backing him up. And here's Tim. He says all the retail stores are closed now as well. Introduces Lorene, Steve's wife. Bill Campbell speaks after Tim, teary but the coach he is comes out. They played Steve's narrated version of "The Crazy Ones". Talk about the Macintosh.

Norah Jones is playing piano now on the stage here, singing. The Maleness of You. If I Were a Painter. (!) and a Dylan song for Steve: Forever Young.

I see Al Gore over there next to Tim as they listen to Norah Jones sing her plaintive Dylan song.

Now Tim is introducing Al Gore to speak. He's talking about love in the use of technology. It erases the barriers of those we need to connect to.

Johnny Ive is speaking now. 

Coldplay is here to play for us. Wow!

They are playing Yellow right now. They played it for Steve and he told them it was shit, their singer says. But it's wonderful!

Now When I Used To Rule the World. One of their biggest songs. Wow!

Now they're playing Fix You. And Every Tear is a Waterfall. It's an upbeat number.

I am now sitting by the stage. Letting my weary back, feet, and hips rest. They are playing Norwegian Wood on the speakers. I like that one.

Dylan's version of All Along The Watchtower. Takes me back. I like Hendrix' cover of it though. ;-)

I'm having a Coke. They have free drinks and I grabbed whatever they had on ice. The sun has been out and I'm a bit warm. Now Louis Armstrong's Wonderful World is playing.

I decided to leave my car. Mariani is like a parking lot. I will go get lunch with some fellow employees who knew Steve and his tough management style firsthand.

Of all the speakers, Johnny Ive was the most touching of all. His anecdotes about Steve were really good.

I cried a few times.

After the event, I went with a bunch of folks to a place called the Tied House in Mountain View. It's a mini-brewery. There were significant managers and some of my most treasured friends there who work on the cameras and imaging software. And we were all drinking the beer. And having a nice lunch, talking about our memories of Steve, and what's next. I started a conversation thread on what's next. What would be considered impossible but just so much better. What is the next thing that we should strive to accomplish. And it was fun to hear the answers. From people interested in sensors to people who know how the camera stuff should work to the managers who just want to use it and have it work right.

The event was wonderful, and all the people were broken up over it. Pretty much everyone who was at the lunch had their tearful moments.

When Tim spoke, he was eloquent, respectful, inspiring, intelligent. When Bill Campbell spoke, He was nearly in tears at the start. But really came through as a powerful speaker that really knew Steve and his turnaround of Apple and how he accomplished it. How he did things not because they were easy, but because they were the right thing to do.

Johnny Ive spoke about Steve, in anecdotes and in wonderful references to his decisive capabilities. Steve always came up to Johnny and told him, "Hey Johnny, I have this really stupid idea". And sometimes the ideas were stupid, and sometimes they were really stupid. But Steve really knew that to turn an idea into a real product took lots of time and investment in honing it and making it what the original thought was like. And sometimes these thoughts produced greatness.

When they traveled, Johnny said, he would arrive and get to the hotel and set his bags down and then sit on the bed and move quite close to the phone. Soon, the inevitable call would come. It was Steve, "Hey Johnny, this Hotel sucks!". :-D

Al Gore said that every iDevice was a little bit of love from Steve and it was all about erasing the barriers to connect each to each other. That using the iPhone was almost like love to the brain, according to researchers.

Tim said that Steve's work on Apple stuff went on until his last day on earth. He personally recounted a story where Steve called him on the day the iPhone 4S came out, when he was in a meeting with the SoftBank CEO, and Steve called him to talk about the next product".

And mixed with it all was the music. The wonderful music of the world. Which flew through the air to touch down in Cupertino this day. In person, Coldplay and Norah Jones did unbelievable jobs producing sound from their hearts. My hips hurt from standing but I think my heart would have hurt more, and for longer, to miss it all.

Different Modes of Thought

A good friend found this link earlier, about how renaissance people think: shifting from one mode of thought to another. And I found myself in those words somewhere between rational thought and experiential thinking. How does this apply to me? Do I shift constantly, or do I exist on both sides?

When I get into a rational frame of mind, I can organize and probe a complex set of information. I can make sense from randomness. I can put it all together.

When I get into an experiential frame of mind, I can create music. I can draw interesting designs. I can understand how another person is thinking and be considerate. I get the feeling that most people have to contain both sides in some quantity or another: the thinker and the creator.

Rational People

I know that there are some people who are pure rational thinkers and very little empathy. I have met a few and they are sometimes quite creepy. Some people are very deep into this and can do amazing things. They are great resources, for sure. When rational thought reigns over empathic thought, you can easily end up with a sociopath. This can happen when they are taught bad values from the very start, or if they aren't taught the right ones, I think. Such a person has to learn from scratch how to interact with others and sometimes they become very practiced. And sometimes they are misunderstood.

I had a friend named Bob Lansdon. Really he was Tom Hedges' friend, having met him at Ruddock House in Caltech, but I came to know him pretty well, since he contracted for Fractal Design. He was responsible for the logic behind the first watercolor brush in Painter. Tom did the programming, but Bob figured out some interesting things about watercolors in the physical sense. Bob was a pure rational thinker. He was constantly involved with academia and working on his PhD. He was, as we sometimes put it, an odd bird. I must thank Bob because he turned me onto Fourier Transforms and it was through Bob that Tom Hedges first learned of fractals at Calma in the late 1970s. I did a lot of work on fractal branching in the early 1980s and had access to a raster plotter. I even presented a fractal poster to Benoit Mandelbrot at a SIGgraph once. Bob was a tragic figure, because we never could figure him out. I remember one day in 1994 that Bob came into our office (Tom, John Derry, and I had the same office the Fractal Design building in Aptos then) and was muttering about his PhD. He showed us his certificate and we congratulated him but we were all working on deadline for some version of Painter. After he left, we never saw him again. He committed suicide in a park in Berkeley a couple days later. Tom and I beat ourselves up over it, and the only thing we could figure was that, when he finally got his PhD he figured his life was over and had no more to contribute.

Later, in Painter 6, I implemented a new watercolor brush, with a donor-receptor model and multiple layers for the water to transport the ink and a paper layer for the ink to dry onto. A Margolus-Toffoli cellular automata model was used in the transportation of the ink-bearing liquid. Bob Lansdon showed me their work. Also, the whole model for how watercolor actually works and how it could be simulated was heavily influenced by talks with Lansdon years earlier. But it wasn't practical when Bob was around. So the watercolor feature in Painter 6 was a tribute to Bob.

Experiential People

There are also people who are completely in the experiential mode. I have met a few actors, and they have matched this one-sided profile quite well, which is why the best of them are so good at ad-libbing: they are creative. The most creepy thing is to see a person that has to act to live, that have to be something else to "be" at all. Empty vessels that need to be filled up with something/anything. Why are they so good at portraying bad people? Poor Heath Ledger was probably a very good person, but one thing is certain: her was a really good actor. Clearly, after he portrayed the Joker in The Dark Knight, he had learned accurately to be a sociopath. I can't imagine living a character for weeks on end and not having it affect me in some way. In Heath Ledger's case, perhaps it somehow contributed to his death: pure speculation.

Actor Daniel Day-Lewis is famous for totally taking on the persona of the character he portrays. His zeal at becoming the character has led to many unusual circumstances.

What Mode am I In?

Now I sometimes will be asking myself what mode I am in. When I sit at the piano and play, I improvise. Improvisation is a creative art, so it is experiential. But I can also see the chord sequences in my head, and understand the chromatic relationships between the chords. I can feel the music, I can sing and play simultaneously, but I can also tell my fingers which patterns to play, and how the rhythms interrelate.

When I play a Beatles song, I often put my own spin onto it, interpreting it in a new way. I have long since given up on performing them in the original mode that they were recorded: once you have done that, then you yearn for the song to mean more to you. It just has to be changed up.

There's no doubt that I exist in both the rational and the experiential mode simultaneously.

When I program, managing a complex project, I exercise my rational thought processes to the limit. But I'm always looking for a new way to look at something. This is the creative side peeking through.

Again, both sides are operating simultaneously. Perhaps one side dominates, but they are definitely both there.

Hackers

Hacking is not new, nor are the motives for hacking. But not all people know what they are, nor how the hackers act on their motives, nor how they can protect themselves or their companies from hacking. Let's talk about hackers for a bit.

Means

All it takes is a computer and a connection to the internet, right? Wrong. It takes mad skills to get anywhere in the hacking game. A penchant for puzzles. A love for spy vs. spy. A more than average intelligence. And it takes friends, either real people or just botnets. Or, just access to the right tools.

Attacks on organizations, particularly DDoS (distributed denial of service) attacks, are typically organized via social media, coordinated on Twitter, and accomplished with tools such as Low Orbit Ion Cannon (LOIC), a tool specifically designed to accomplish DDoS attacks. These attacks quickly make websites useless because their servers are overloaded with incoming messages.

The hacker's toolkit includes the rootkit, basically a way of achieving administrative privilege security level on a computer. Usually malware starts the ball rolling, perhaps installed by a zero-day exploit. This malware subsequently installs some processes designed to be completely undetectable that aid the hacker in accomplishing their tasks. Once a rootkit has penetrated a computer, that computer can then be used remotely and it becomes a zombie (or bot). When a large number of these computers have been secured, they become a botnet. So a hacker can, for instance, install LOIC onto several computers in this fashion to provide more power (and bandwidth) for a DDoS attack.

But, of course, it is possible to simply rent the computers to accomplish the same task. It's easy to rent hundreds of computers from Amazon Web Services. The attack against Sony Corporation's online entertainment services, which resulted in the compromise of the personal accounts of over 100 million customers, was facilitated in this way by users with fake names.

Tools are available online and some people just use them without realizing how they do their job. Such people are called script kiddies in the hacking world. Hacking tools are apparently available for several purposes. Keyloggers are a kind of malware intended to record each keystroke the computer's user types, including their username and password. They are often structured as a trojan horse, a program designed to look like a trusted system, perhaps the login screen. There are plenty of techniques used by modern hacking groups like the recently-disbanded LulzSec and the active group Anonymous.

Most of these tools and techniques are designed to penetrate a computer and obtain system administrator privilege. Once a hacker has this privilege then they can access or change any file on that computer. The files can contain other passwords, or perhaps valuable data such as credit card information or personal addresses and phone numbers. Or perhaps it contains private information.

Motive

The DARPA Shredder Challenge

In 1974 when I was a freshman at Caltech, there was a bit of hacking about. One blonde-haired "troll" was quite proud that he had penetrated a security kernel of a system remotely by hand-disassembling it from an IBM 370 machine code dump. Over Christmas break, some students orchestrated and accomplished the "McDonald's Sweepstakes Caper". I was in Steve Klein's dorm room listening to Pink Floyd's Dark Side of the Moon in Page House when someone walked in with a bag of McDonald's. A contest entry form was passed around and the guys discovered that the entry form said "enter as often as you wish". Even more damaging was that the fine print on the entry form didn't say the forms had to be handwritten or signed by a human, or even that they couldn't be printed separately. We thought this was hilarious! When I went home for Christmas break, they used a computer to print out hundreds of thousands of entry forms and distributing them into as many McDonald's as they could find. By the end of my freshman year, they had won 20% of the contest's prizes, including a car. Although the caper wasn't exactly hacking, it demonstrates the first motivation for hacking: it's for the honor of saying "I did this". Yes, it is very similar to the reason people climb Mount Everest.

So, honor and a sense of one-upmanship is a very powerful psychological motivation for hacking. Witness the years-long rivalry between MIT and Caltech that finally erupted in Caltech's cannon being stolen.

These days it's quite a challenge to keep secrets, it seems. The more valuable your secrets are, the more people are trying to get them. The more damaging your secrets are, the more people are trying to publish them. The more famous you are, the funnier people think it is to harass you. These illustrate three other motives: the criminal, social activist, and humorous motives for hacking. Nowadays, there is one more overarching reason for hacking, and its totally wrong: state-supported hacking. Hacking for destabilization, infrastructure attack, and for gaining the economic upper-hand are increasingly becoming common.

Indeed, some of the more infamous attacks use rootkits to penetrate special-purpose systems and accomplish political gains. The Greek wiretapping hack is one example: the perpetrators were never discovered. The Stuxnet virus, a brazen frontal attack on the Iranian nuclear weapon ambitions, has been long suspected to be Israeli, American, or Russian in origin but we may never know. It also attacked special-purpose hardware using a root kit.

Criminal hacks abound. Consider the phone hacking scandal involving the News of the World. The British tabloid hacked into the voice mail of the murdered school girl Milly Dowler in order to secure an interview with her mother. This was intended to sell more newspapers, so the motive was money; the act was criminal. But it was only the tip of the iceberg.

The release of damaging information often results from a sense of social activism. They believe they are advancing the cause of transparency, accountability, and freedom. The case of Bradley Manning and WikiLeaks illustrates this trend more than any other case, although it really wasn't hacking. For hacking-related social activism, it's better to look at Anonymous and the emergence of the hacktivist.

Opportunity

Hacking is definitely a crime. There's even a name for it: cybercrime. But is it the only crime being committed? Is there perhaps some stupidity or worse gross negligence that enables hacking and the subsequent loss of data, by creating a huge low-hanging-fruit opportunity? Oh, most certainly!

The largest presented opportunity is fame. But sometimes you can't help being famous. Sometimes it's not even your ambition to be famous. Still, when you are famous, people love to see what you are doing. This is why data about them is highly prized: to sell gossip zines. It appears to have become common for paparazzi to be in league with hackers, sometimes freelancing and sometimes connect with specific media outlets. Media outlets often offer huge sums for pictures of celebrities. My favorite is the National Enquirer, which offered a cool $1M for an Obama love tryst video.

The next presented opportunity is lack of proper security. This almost doesn't need to be explained. Anybody with a password of 123456 or qwerty probably doesn't know how insecure they are - simply because of cluelessness. There are plenty of available lists of common passwords. All a hacker has to do is try them. But truthfully, any word in the dictionary can be tried by using a password-cracking tool. There is even a list of commonly-used iPhone passwords. So it is very important to choose a username/password pair that is secure. They say to (1) use a word not in the dictionary, (2) have the password be 8 characters or longer, (3) include at least one or more numeral in the password, and (4) to include both upper and lower case letters. Using the same password for several accounts is also not a good idea. E-mail passwords are typically sent across the wires in plaintext format, so bear that in mind.

Sometimes getting into a computer is not very hard due to zero-day exploits: an exploit such as a buffer overrun that you can use right now (because it's installed in several running computers) that nobody knows about. And if they are in, then they don't need your password. So your security should go even deeper. Information stored on your computer that has intrinsic value, or is held in confidence for your customers should be encrypted. Failure to do so has led to several infamous hacks and also of loss of data in the wild. This is inexcusable, particularly in the presence of such viable alternatives as Transparent Database Encryption in Oracle systems.

A browser vulnerability, known as parameter tampering, where the browser address string is simply changed from one account number to the next, caught Citibank off guard when hackers used their computers to modify the string tens of thousands of times and access confidential data.

Finally, hackers are increasingly becoming emboldened by the opportunity of being able to easily sell their ill-gotten credit card and user identity information. Online bazaars are professional-looking sites that allow the hackers to easily connect with their buyers, who use the information to impersonate the victims and buy merchandise.

Energy, Part 1

As the world moves closer to fossil fuel depletion and global warming seems to be increasingly caused by carbon dioxide emissions, sources of green energy have become more and more important. But what are green energy technologies and how far have they gone towards adoption? What hold-ups are there in relinquishing fossil fuels? What can satisfy humanity's diverse and enormous requirements for energy? To even start to answer these questions, we need to know how energy is harvested, stored, and consumed.

Harvesting Energy

Since Einstein we have known that energy is contained in all matter, and his famous formula represents an upper limit on how much energy can be harvested. And in a few cases, we have determined how it can be extracted, with varying efficiencies. Let's look at a few. The first method of energy extraction is from an exothermic (heat-producing) chemical reaction. This method is used in internal combustion, and it releases heat, measured in Joules. For example, when combusted with oxygen the following fuels release this much energy in kiloJoules per gram:

• Acetylene 11.8
• Ethanol 27.3
• Coal 17-21 (sub-bituminous) 29-33 (bituminous or anthracite)
• Kerosene (Petroleum) 43.1-46.2
• Methane 50.6
• Gasoline 51.6
• Hydrogen 120

The reason Hydrogen has so much promise is due to its clean-combustion: water is the only by-product of combustion with oxygen. The other fuels release CO2 in varying amounts per kiloJoule of heat produced: gas is the least at 1.2 moles of CO2 per megaJoule and coal is the most at 2.0 moles per megaJoule.

The conversion of harvested heat into electricity or mechanical motion is quite a different matter.

Most people know that internal combustion engines are really controlled explosions. The addition of heat to a gas causes a significant change in its density. This change is implied by the law of ideal gases:

PV = nRT

Here, P is the pressure, V is the volume, n is the amount of the gas, R is a constant, and T is the temperature.

From this, we can see that when you increase the temperature, while keeping the amount of gas and the volume it is contained in constant, then you must increase the pressure proportionally. A massive increase in heat and a chemical reaction can create an explosion, which is a massive increase in volume. This is what happens in an internal combustion engine: the gas explodes, the massive increase in pressure drives a piston, and the motion of the pistons drives a cam shaft.

In modern cars, this is used to propel the car forwards, as work.

This can also be used to move a rotor and generate electricity through electromagnetic induction. Many power plants work this way, including the Moss Landing power plant in California.

Chemical reactions, to generate heat and drive turbines, generate about 65% of the world's electricity requirements today.

The second method of energy extraction is the harvesting of kinetic energy from matter that is already moving, such as water and air. This energy is almost always harvested using a turbine.

Have you seen the wind machines dotting the hillside near you? Those are wind turbines that harvest electricity from the wind itself, with very little effect on the environment. This is commonly called wind power, and is a renewable energy source, powered indirectly by the sun with the process of convection, and the turning of the earth, through the Coriolis force. Wind power currently generates about 1% of world electricity requirements, though the cumulative output is growing exponentially, suggesting in 20 years that an 8-fold increase will occur.

Another kind of kinetic energy that can be harvested is moving water. This is commonly called hydroelectric power. I'm sure you have noticed that water runs downhill. Here gravity itself is harvested because of the inexorable tendency of water to find a common level. Water, usually stored in a reservoir (fed by rain or snow melt) is stopped up at a dam. Some of the water is allowed to flow into a river, and in between the reservoir and the river is a turbine that runs a generator. About 1.5% of world electricity requirements are generated using hydroelectric power.

What happens when you mix exothermic reactions with turbines? The heat produced by chemical reactions is often used to heat water to produce steam. The state change that occurs when water is converted to steam means an increase in volume by a factor of approximately 1700 times. The increase in pressure means steam can be used to drive a turbine to generate energy, in a technology known as steam turbines. This process is used in aircraft carriers to great effect: it can drive a steam catapult to propel planes from its deck, or it can be used to drive the main screws to propel the carrier through the water. This process is also extremely well-suited to energy generation via electromagnetic induction. About 90% of all electrical power in the US is generated using steam turbines both in conventional coal, natural gas, fossil-fuel, and also through nuclear power plants.

A third method of energy extraction is via the photoelectric effect. This is the method that solar cells employ to harvest energy directly from sunlight. The efficiency of this technique is determined by how many photons are required to generate a single electron. Modern solar power plants, however, usually employ a different method to harvest energy. The technique is known as solar collection. In this technique, mirrors and lenses concentrate the sunlight from a large area to a small area. Once in a small area, the sunlight can be used to heat water in a closed steam system, as in solar towers. This steam is then used to drive a steam turbine, which drives a generator, which generates electricity. Usually the mirror is shaped like a parabolic trough, and instead of heating water directly, it heats molten salt and then the molten salt is used as a heat source for the power generation system.

Solar power is considered to be renewable since it harvests energy directly from the sun, the effectively continuous free energy source.

The largest solar power station in existence, in the Mojave desert of California, generates 354 MW of power. In comparison, the Three Gorges Dam hydroelectric power station located on China's Yangtze river, generates 18.5 GW of power, and, when finished, is intended to generate a total of 22.5 GW of power. As of yet, solar power hasn't yet reached a generating level of even 0.1% of the world's electricity requirements.

A fourth method of energy extraction comes from the energy contained in matter itself. Nuclear power currently works by exploiting the chain-reaction properties of U-235, an isotope of Uranium. In a nuclear power plant, the runaway chain reaction is usually moderated by water and other slow-neutron absorbing substances, like graphite. Nuclear reactors generate heat in abundance. This heat is used to heat a second, insulated water cycle and generate steam, which then drives a turbine and a generator to make electricity. But it is also possible to use molten sodium, an excellent neutron absorber, to transfer the heat of the nuclear reaction, in a so-called liquid metal reactor.

Uranium is about 40 times more commonly occurring naturally than silver, and so it is hard to prevent technologically advanced nations from acquiring it. For instance, in Israel, the sands of the Negev desert contain trace amounts of Uranium. The separation of U-238, the isotope of Uranium that makes up 99.3% of naturally-occurring Uranium, from U-235 (that makes up the rest) is complicated.

The Diablo Canyon nuclear power plant
Source: PG&E via Power Plants Around the World

Nuclear power presently generates considerably more energy than solar power. For instance, the Diablo canyon nuclear power plant in California generates 2.4 GW (7.5x the largest solar power plant) and makes up about 20% of the Northern California power grid.

Nuclear power accounts for 14% of the world's electricity requirements today.

However, nuclear power has some serious drawbacks. The disposal and storage of radioactive waste (particularly spent nuclear fuel rods) presents problems that, while they can be solved, are nonetheless controversial. The Östhammar facility in Sweden shows promise for treating this problem with the proper respect and care required for 100,000-year storage systems. The site was chosen partly because the rock at the waste-storage level is relatively free of fractures. After the rock is excavated, two tons of spent fuel is stored in 25-ton copper canisters. Each copper canister is then welded shut using a special robotic welder and robotically deposited in an individual tunnel in the repository. Then, bentonite clay is injected into the tunnel, mixed with water, to expand into place. This forms a watertight barrier that is essentially earthquake-proof. The storage repository is scheduled to open in 2025.

Portable Energy Storage

Once energy is harvested and converted to electricity, it may be stored for later use and carried around. You can consider fuel to be a portable energy storage system also, although usually fuel needs to be combusted and this makes it normally unsuitable for battery usage. But even this axiom is being challenged by the fuel cell.

Portable energy may be stored in several ways. The first is a battery, which generates electricity through electrochemistry. The second is a capacitor, which stores energy in its electric field. The third is a fuel cell, which, similar to a battery, uses an electrochemical reaction to generate electricity.

We grade portable energy storage systems on:

• capacity, which is the amount of electric charge they can store
• charge time, the amount of time required to return the device to full or substantial charge
• discharge rate, the maximum amount of constant current the device can produce
• energy density, a measure of how much energy the device will produce by weight

Each characteristic is useful for different uses. For instance, in an electric car, the energy required to start the engine and move the car from a standing start is related to the discharge rate. Also, a device must be light in relation to the amount of energy it contains, and thus energy density must be high for an electric car battery.

On Hydrogen Fuel Cells as an Energy Source

The energy density of hydrogen gas is the highest of all chemical sources, in excess of 120 kiloJoules per gram. The energy density of a lithium-ion battery, in contrast, is only about 0.7 kiloJoules per gram. Of course, nuclear material such as U-238 has an energy density of 20 gigaJoules per gram. Antimatter contains a theoretical maximum of 180 teraJoules per gram. Presently, nuclear material and antimatter are unsuitable for portable energy storage systems due to the weight of a nuclear reactor and the general unavailability of antimatter.

All above considerations point to hydrogen fuel cells as the most likely successor for portable energy systems. Per weight, hydrogen fuel cells are about three times the energy density of gasoline. The volume of hydrogen, even stored as a compressed gas, far outstrips that of gasoline for comparable amounts of energy generation. This makes hydrogen use a bulky problem.

Also, most all the world's hydrogen production emits CO2, since it uses the steam methane reforming process. So some improvement is needed to cut down on its carbon footprint.

Improving Batteries

The characteristics of a capacitor are short charge time and fast discharge rate, really the opposite of a typical battery. This is why several companies are trying to merge the two technologies to get the best of both.

Supercapacitors are a new technology which promises to replace the battery as we know it. One valuable attribute of supercapacitors is the apparent ability to charge and discharge thousands of times, making the device stable enough to outlive the device it is intended to power. The main problem with super- and ultra capacitors (battery-capacitor hybrids) is the energy density. The capacitance of these devices is directly proportional to the electrode surface area. The use of materials like activated charcoal (with its unbelievably large surface area) have increased the energy density into the usable domain. The promise of nanotechnology, such as nanotube carbon filaments, also can lead to high surface-area solutions and still greater energy density.

How Energy is Consumed

The amount of electricity used per year by humanity is in excess of 20 petaWatt hours per year. The US uses about 20% of that, and China uses another 20%. This doesn't include the energy produced by internal combustion engines, or by burning coal, wood, and kerosene for heating. When all energy consumption is added up, total annual consumption is 474 exaJoules.

It is interesting that world energy consumption decreased 1.1% in 2009, due mainly to economic downturn in North America. But that trend doesn't seem to be a continuing story, since in 2010 world energy consumption grew about 5%. In particular, China's energy consumption did not decrease in 2009, and consequently it is now the world's largest energy consumer, at about 18% of global energy consumption.

What's Next?

In part 2, we will drill down farther into how energy is consumed, and discuss what we can do to cut down on energy consumption, and what is already happening in that regard. We will also discuss the thorny issues surrounding fossil fuel usage. Also, the carbon footprint of energy production and consumption will be discussed. Which energy sources have the smallest carbon footprint? It's not obvious at all.

Creativity and Painter, Part 1

Painter 1.0, which shipped in much-less-corroded
paint can than this one, had a theme of creativity

For those of you who have read my introduction to this blog, you may have wondered what I meant by "creating Painter". This post should clear up any confusion on the subject. There are resources detailing this from other perspectives, and they are to be cherished. John Derry's interview on Painter and other subjects makes fascinating reading. He's got it right. Karen Sperling's account of writing the first Painter manual also gives more wonderful insight into the process. Some of this has also been included into the various books on Painter techniques and artistry as forewords.

Corel sells a product called Painter, currently in version 12, that is an excellent application for painting and sketching. What is little known these days is that I wrote the initial technology entirely from scratch, starting in September 1990.

Earthquake

In October of 1989 I was in the back room of my house in Aptos working on Shapes, the illustration layer for ColorStudio. Shapes was a system for controlling Bezier curves in a manner similar to PostScript. On the 17th of that month, a 7.1-on-the-Richter-scale earthquake came which dumped the heavy CRT monitor in my lap and sent me running out of the house. I could see my 7-foot grand piano bouncing two feet off the floor. I headed out the front door with my family, dodging bricks that were falling off the façade surrounding the front door. These were real full-sized bricks, unfortunately. The earthquake felt like a giant fist hitting our house from the side again and again. As I ran out from the cacophony that was the inside of our home, my foot slipped on the brickwork and I twisted my ankle painfully. My (then) wife Ruth was justifiably quite frightened and this led to a traumatization that caused her and the kids to sleep outside for a month.

It took several months to get the house fixed up. Several headers were under-spec and had to be joisted properly. Internally, the wood framing had to be re-strapped. The chimney was heavy brickwork and had pulled away from the house. If it had come crashing into the house, I might not be writing this blog now. It had to be reframed using zero-clearance and was redesigned to control a heatilator, which allowed the heated air surrounding the fireplace cavity to be circulated into the house.

The insurance company paid off and the house was repaired. But this took almost a year. When the construction people were finally out of the house, my mind was at rest and it became time to start something new. Exactly what frame of mind I was in is a complicated story.

Secret Starts

Tom Hedges,
later in life
Source: John Derry

My job at the time was really a partnership with Tom Hedges, called Fractal Software. Starting something new wasn't in the cards for me, since ColorStudio 2.0 was underway and improvements to Shapes were my official task at Fractal Software. I couldn't start anything new at work, because Tom would have blown a gasket. And the process of making the new revision of ColorStudio Shapes was in the debugging stage.

For those of you who know me, you probably already know that I'm more a developer than a fixer-upper. And Tom, of course was the ultimate finisher. His debugging and troubleshooting skills were legendary. He was also a great systems programmer.

So I was always starting something new. This time, I spent a lot of my time deriding what ColorStudio had become: a production tool. You see, production is not what I do. What I like is creativity. Sketching. Drawing. I am an incessant artist.

Let's take a step back. Our first actual project together had been an audio digitizer and software, called SoundCap (for sound capture) to drive it. Our first customers were actually Steve Capps, Bill Atkinson, and Andy Hertzfeld from the first Macintosh team. We licensed this product to a marketer in Minnesota (MacNifty, run by Mike Halvorson) and they promptly had our software reverse-engineered and replaced, so he wouldn't have to pay us the exorbitant royalties we charged him.

Grumbling about this, I started a new project initially called GrayPaint. This application had the ability to use a mouse to sketch using a pencil, using charcoal (that actually built up from lighter to darker so you could shade), and a fingertip that smeared the image. I spent plenty of time perfecting this notion of a more realistic drawing capability. Tom got the insanely great idea to connect this grayscale technology to scanners (which were able to collect sixteen levels of gray!). So he began to implement connection software for scanner after scanner. Between the two of us we had just developed the first image editing software for Macintosh. We showed this to Marla Milne of Letraset and she codenamed the project The Realist and we signed with Letraset. This project became ImageStudio, and it was our first real moneymaking software application. It was with the royalties from this software that I was able to purchase my house in Aptos.

The net result was that I was the lead on ImageStudio. So when 24-bit cards began to show up from RasterOps and Supermac, Letraset began asking us about a color version. This time, Tom wanted to be the lead on the project, and I agreed. This became ColorStudio, and ColorStudio wasn't an artist's program. I spent months working on RGB to CMYK conversion software. And then Shapes. All the programming was technical, and none of it was creative. I became bored. But ColorStudio made much more money for us, so who was I to complain? It was a coincidence that the first Wacom tablets arrived at that time.

At home, after the earthquake repairs were over and I had more time on my hands, I just couldn't stop thinking about what had been lost with ColorStudio. So in September of 1990 I began studying how pencils interacted with paper. How to create seamless textures. How these textures could be used to simulate pencil strokes. I studied how felt pens built up to make new colors. I created log-density equations to allow for the build-up of charcoal and graphite on the page. I modeled brush strokes and the physics of the motion of the hand using the (initial, crude) input from the first Wacom tablets. I had to write software to de-jitter the lines.

I did all this in secret because I was worried about what Tom would say about me doing something that didn't have an obvious monetary value. I proceeded studying chalk and ink pens, pencils, charcoal, and even oil paint for some five months before I actually even showed it to another living person.

One day I did bring it into work and I was immediately criticized by Tom for not concentrating on the matter at hand (ColorStudio 2.0, and Shapes). One week later, we got the notice from Letraset that they were terminating their software division. We knew that that meant: no more royalties. And Letraset still had the license to ColorStudio and ImageStudio.

So we brought some good friends into our small 660-square-foot office to look at our new software, which I had nicknamed Painter. Lee Lorenzen and Steve Manousos were their names. Lee had made his fortune on Ventura Publisher and was busy monetizing his software for cross-platform development at Altura Software. Steve ran a service bureau called Aptos Post with Linotype machines and printers and had become an invaluable resource to Tom Hedges and me.

They were impressed and we started Fractal Design Corporation a few months later.

The Creativity Begins

The rush to get out a product (in August 1991) led to interesting developments like cloning, but, as you already know, the finishing work on a product was more Tom Hedges' expertise than mine. Also, while Painter 1.0 got finished for Mac, Painter 1.5 was being developed on Windows 3.1, using Altura's cross-platform development environment. Tom did an excellent job of this, while simultaneously developing a system for virtual memory using image tiles. This was a period when Tom's contributions really shone.

But I got restless to make something new. Based on my observations of animation sketch artists' stations at Disney, I decided to implement canvas rotation. In Painter, this is the tool that allows you to rotate the canvas you are drawing into. This is necessary, because the artists' hands typically favor certain angles (45 degrees for right-handers such as myself) and the best sketchers actually rotate their canvas quite often to do better freehand work.

Also, at this time, we hired John Derry.

I met John at the same MacWorld conference in August 1991 when Painter was debuted. He was demoing Time Arts' Oasis. His demos were spectacular. I knew I could only just barely approach that kind of artistic flair and skill in demos. So I made him an offer. It was the best decision I ever made.

Rule Number 1: Take Notes

In those days I kept notebooks in my pocket, along with a FaberCastell DESIGN EBONY pencil. These were used to keep any ideas that I had, so I could harvest them later. Fortunately, I have kept them all!

On these pages, I kept some of my more interesting ideas and concepts. Here are two pages from my notebooks, dated June 1991; this was two months before Painter 1.0 came to market. Hope you can read my handwriting!

 

This is the original notebook I used. Note that there is a small sketch, my initials, and the date. Also note that the original EBONY pencil is still stuck into the spiral binding at the top! The title is "inimitable features of natural sketch".

Based on the next pages and dates from that notebook, I remember not more than ten days after writing these notes I had a conversation in Banff, Alberta with Ed Catmull, president of Pixar, about their new deal with Disney for three feature-length animation films (which became Toy Story, A Bug's Life, and Toy Story 2). And about the relationship with his new CEO at Pixar, Steve Jobs. That was one of the most interesting conversations I have ever had.

Later I started using a Moleskine (without lines) and a gold Cross pen. Once we hired John Derry, there were press tours for Painter 2.0 and Painter 3.0. These were watershed times for creativity.

I often used little yellow notepads for my incessant note-taking. Here is one example of a feature concerning circularly-arranged brush bristles and their contact with the canvas as you turn the brush and simply change the direction of your brush stroke.

At one point, John Derry numbered all my notes. I can't remember for the life of me why he did such a thing, but the numbers at the top left corner of each note are in his hand. I remember that this was around Painter 4. The notes clearly show the intimate relationship of lists, diagrams, and equations in my notes.

Later on, in the Painter 6 time frame, after Fractal Design had merged with MetaTools to become MetaCreations, I continued to take notes. Some of my notes were very graphically oriented, showing my technical/artist sides. Here is a particularly florid example from 1999.

© 1999 Mark Zimmer. All rights reserved.

As this page from my notes shows, I was concerned with the state of Painter 6 (list on right hand side) and  the state of the impasto brush (this was Tom Hedges' greatest creative moment, by the way!). You will also notice that the first customer ship date of Painter 6 is listed as September 24, 1999.

I don't know where to begin with all the graphics on this page. At the lower right is an idea for an animation where a 3D shape consisting of 7 cubes assembles itself. There is a heavy influence of positive and negative spaces here. Also, an interesting folded-paper symbol is drawn in three styles in the approximate center of the document.

A curious mention of "metaflash" - a product that never really came to fruition - along with a "capture your soul" tagline appears at lower left. A comparison to quantum-leap technology leads me to use a metaphor of taking polaroids of indigenous peoples. I have heard that they used to believe that photography could capture your soul.

You will also notice that most everything on the page is tilted about ten to fifteen degrees.

The reference to 3D Paint was because I was working on Detailer, the 3D Paint product.

Rule Number 2: Never Stop Creating

Wherever you are, you should still continue to create. Be able to capture your creative ideas on the move. Here is an example of that kind of constant creativity that worked famously.

John Derry and I were in Chicago for a Print Expo one year. We were sitting in an outdoor cafe, under the shade of some nice trees. I remember this day vividly because, on a table closer to the street corner some rude college-aged kids would make some reference to "corn muffins" to every pretty girl that walked by. I still don't quite know what they were talking about!

Anyway, John and I were eating lunch when suddenly John said. "Hey Mark. There's a tree. How can I paint with that?". He just wanted to paint and have a tree come out immediately. He pointed to some stucco texture on a neighboring building and asked the same question. Then he pointed to some bricks and repeated the question.

I began to formulate an idea of capturing snippets of reality and having them come out, individually masked, onto the canvas right out of the brush. I thought that each one could have a shadow, or could be individually lit with some kind of 3D model in mind. So I began to draw out a structure where these images could be stored and referenced with varying degrees of randomness and at varying scales and angles. I started thinking about how these scales, angles, light, and shadows could be controlled by the user's hand, by the parameters that come out of a stylus.

This technique, which became the Image Hose in Painter, was dreamed up entirely within this lunch in Chicago.