Interesting Persons, Part 1

Back in the 1980s I was fascinated by sound synthesis and analysis. The most well-known work I did was a little application called SoundCap (for Sound Capture) that was coupled with an Analog-to-Digital converter initially sold by Fractal Software, my partnership with Tom Hedges, and eventually sold by MacNifty. It is fortunate for many of the early Macintosh developers that this box hooked up to the back of a Mac through the serial port. Several sound-producing apps were produced with it, including Airborne! by San Diego's Silicon Beach Software.

Stephen St. Croix was a friend of mine. He contacted me at Fractal Design in the 1990s and wow'ed me with a few of his wondrous stories. We spoke at length on several occasions about digital sound synthesis, one of my many hobbies. I was surprised to learn that he was one of the inventors, at Marshall Electronics, of the Time Modulator, the box that introduced digital delay line flanging to more than a few famous musicians.

The most interesting story he told me was about the job he did with Lay's. Yes, the people who make the potato chips. It seems that their spokesman, Jack Klugman (of Quincy fame), had lost his voice as a result of throat cancer. This really made a problem for them because his commercials for Lay's potato chips were pulling quite well. After all, he was a very recognizable and a well-loved actor. His voice was distinctive. People listened to him.

Stephen informed me that they invented a new kind of voice synthesis device to recreate his voice. It used formant synthesis. Incredibly, they could exactly duplicate the distinctive gravelly sound of his voice in this manner! It seems that the very low-frequency warbling of his vocal cords, though inimitable by human voice impersonators, was entirely imitable by digital synthesis techniques.

At Marshall Electronics, they spent quite some time analyzing sound. They had room analyzers. And so they also had room simulators. But the least known cleverness involved voice analyzers. Imagine picking apart someone's voice, layer by layer. Figuring out the pitch-profiles and the syllabic inflections. Hand-tuning the cadence of the words. My mind was boggled constantly by Stephen's work.

I informed him of my work in music extraction. I had a special application called Do-Re-Mi that allowed you to whistle a tune that could be output using MIDI in key duration format, complete with amplitude and pitch profiles suitable for modulating a pitch wheel and a volume pedal. It could tell you how many cents (hundredths of a semitone) sharp or flat you were when you whistled. I used a clever correlation technique that involved a time-delta histogram for correlation, pitch-multiple disambiguation, Lagrange peak-finding, and other techniques for isolating the pitch accurately. This work was all done in the 1980s, before Fractal Design, as part of Fractal Software's work.

Tom Hedges, of course, was the hardware designer of the first Macintosh sound sampling box and my contribution was the software, much of it written in Motorola 68000 assembler. Our work with sound continued when we did a bit of work with Bogas Productions, involving Ed Bogas, Ty Roberts, Neil Cormia and others. I met them through a mutual acquaintance, Steve Capps, who was working on the Finder in 1984.

I wrote a sequencing application in 1984 and Tom was fascinated by it. He modified it so it could sequence samples and then proceeded to digitize his piano, note for note. This was in a day when samplers existed, but were quite crude and expensive. He encoded Rhapsody in Blue (he was so proud of playing it) and also a perennial favorite, Wasted on the Way (a thickly vocal-harmonic piece from Crosby, Stills, and Nash). We were both musically literate, but in different ways. I was a composer who played piano and I was fully familiar with sheet music (actually, I had to teach the rudiments of it to Tom before he could digitize the songs, which took a week or so to get it just right). Tom was a DJ with KZSU Stanford and an advanced audiophile. And he had a very wide understanding of music. His father played piano (which explained Tom's interest in Gershwin's Rhapsody in Blue).

So when I began speaking with Stephen St. Croix, I was very deep into audio analysis and synthesis. And the author of a very popular application for sound manipulation on the coolest new computer around, the Macintosh.

It wasn't a big surprise at all that we spent hours and hours talking about sound synthesis, analysis, music, and the recording business. Crazy times and a really good guy.

Seven Ways

There are seven ways that we best retain information, and five of these ways are tied to our natural innate skills as humans. These five are: typing, handwriting, speaking, seeing, and hearing. Two other ways help you complete the process of learning by semantic cross-tagging: mixing and anchoring.

Typing is a skill that we develop to codify something in symbolic notation: language. When we use keyboards for entry, this gives the language center of our brain a workout, which is concerned with coding and symbolication. But what are these codes and symbols? In language, we break our writing into chapters, chapters into paragraphs, paragraphs into sentences, sentences into words, and words into letters. These symbols, their organization, and their semantic meanings are inherent to symbolic processing. And, as humans, we definitely excel at this.

But there are more kinds of codes and symbols. When we use a musical instrument, we usually produce music in a coded symbolic representation: note for note. We break songs into sections, such as verses, refrains, and bridges. We break sections into chords. We layer melody on top of accompaniment, on top of bass. We accent with drums. We break melodies and chords into notes. We even break notes into tone, duration, and volume. Unlike text, music has quite a number of internal properties of continuity. Like staccato and slurring notes together. All of these are also kinds of language symbols that our brains use. Clearly we are using our brains' auditory centers when we make music.

I certainly didn't miss that writing is a bit like music, also. Because when we write creatively, we use plots like we use the interrelationship of melodies and leitmotivs. We make characters and develop alternate realities. We use metaphor and hyperbole. We season our writing with alliteration and onomatopoeia. A theme can pervade a novel. The resolution of a character's arc can stir us like a brilliant cadence. But writing struggles in the last chapter to compete with the finality and intense closure of the coda for a great piece of music.

Handwriting is a perfect way to match our muscle memory to our brains. We coordinate our hands and eyes to denote what we hear or what we think. Taking notes can be a compelling way to retain your thoughts. When we combine it with symbolic representations, we can end up with text, mathematical equations, musical notation, or even scribbles, doodles, and drawings. Let's face it, we think a bit more when we are handwriting than when we type, because a different part of our brain is required to do it.

In some ways, handwriting is utilizing the visual center of our brain. Typing does this also because we use our eyes to verify the text we enter. It seems like it is the connections between brain centers that reinforce our understanding of knowledge and help us to retain and memorize.

Speaking is our natural form of expression. We use our voices conversationally and this method of communication is highly generative, using our cognitive powers to express a thought, a concept, to deliver commands, to convince or inform. We use our language processing centers in a different way, and this is evident in the way we often speak very differently than we write: less formally. When we are in front of a group, we speak from memory, following a train of thought. Actors and presenters learn to do this and shade their performances with attitude and gesticulation, making the art of speaking a multi-dimensional task.

When we sing, we are expressing much more than just notes and words. We are using emotion. We link our generative capabilities to our voice when we sing. When we learn to play piano and sing, we are using much more of our brain than we usually might employ.

Seeing is much more than just looking at a photograph or diagram. It's also seeing in the mind's eye. Some people are very visual and can instantly see a concept in their head before they can express it. They can see the directions on a map in their head when they drive. Our eyes are the key to visualizing, certainly. But even blind people can see concepts. We have spatial reasoning to thank for this. When you have a visual memory, you get to see an object when it is described.

There is more to visualizing than just what is real, though. We can thank our imaginations for this fact. We can imagine impossible figures, for instance, and this concisely illustrates that our imaginations can transcend the real.

Perhaps for many people the spark of an idea comes visually. Perhaps concepts are symbolic for others. Perhaps concepts are neither visual nor symbolic for some: just floating in consciousness waiting to be expressed in some way.

Hearing is a natural way to capture and acquire information. But few of us actually hear a sentence and turn it into text in our head. Maybe a few of us turn it into visual information. But most likely hearing is its own domain. Somehow what we hear simply gets directly converted to knowledge. Still, often we must write something down to retain it, usually.

When I am composing or playing piano, I do not generally rely on my ear to remember the tune and the rhythm. Thankfully, I can record what I play. In other situations, I write down what I play (by hand), in common musical notation.

Even so, I can hear quite a bit of music in my head. It even seems like it is playing back. At 17 years old, I used to do this just before going to sleep, in that nebulous state in between waking and sleeping. I would consciously play a piece in my head. One that I was working on, or a familiar song. Or even a symphony. I guess I was practicing the ability to imagine polyphony. I was on the verge of being a composer at that age.

Mixing modes is the most powerful form of memorization. Sight-reading is a great way to commit a piece to memory. Playing a piece I'm composing, to cement the chord and melody structure is a good way to hone a piece. Record it and listen to it later, to take a step back and form new ideas for where the piece is going.

Listening and taking notes is a good mixture of modes for memorization and retention. But if you really want to cement it into your memory, type up your notes later. Draw diagrams. Learning, though, is much more than memorization. True retention requires application of a concept.

Anchoring is an essential endgame for learning a subject properly. I have a friend who says "I don't want to hire the people who can memorize terms and subjects, I want to hire people that can do something with what they've learned". Memorizing words in a foreign language is useful, but using those same words in sentences is much more powerful because then the words will forever be connected to concepts and subjects in your mind.

Piano Improv

Software development can cause frustration: large projects run over, thorny problems seem intractable, bugs can be hard to find, leading to hours of hard, painstaking logical analysis.

And that's why I like to turn to music to gain perspective.

You can check out my relationship with music in my blog post, Music. You can learn a little bit about how I write songs in my blog post, Writing Songs. And now I will share with you some of the ways I create music ad libitum.

Years of Piano Playing

I taught myself to play the piano in 1973 and this led to years of constant obsession with piano-playing. I went through the catalogs of the Beatles, Pink Floyd, the Grateful Dead, and many other artists. But most of all, I learned to improvise.

I once read that Ludwig Van Beethoven was a master improviser. He could sit at the Hammerklavier and spin tales of music directly from his touch. I always found his music to be inspiring. I also read that Johann Sebastian Bach was a genius at improvisation. And also Schubert, Mozart, Franz Liszt, and others.

In 1973 I wondered if I could ever learn to do that. Just touching the keys of the piano that my parents had bought for my sister Susan to learn piano, but which she had rejected after a few lessons, I saw a tremendous regularity. A devilish simplicity and yet a complexity that defied simple rigorous analysis.

So in that summer when I was left alone in the house in Sunnyvale while the rest of the family went to New Mexico, I had nothing but time to myself.

I have mentioned that the upright piano we had was painted puke green and was tuned a half-semitone flat. The strings were too old to make them any sharper, which would require making them tighter. So it got tuned more flat so all the strings could at least be in tune with each other.

And when you teach yourself notes on such a piano, you can forget about having perfect pitch.

After going to college at Caltech, I found a piano in the practice room in Page house. I played it incessantly. And my abilities improved immediately. The quality of the instrument makes a huge difference, I found.

When I went to UC Berkeley in late 1975, it was as a music major, as I have mentioned. But in between assignments, I improvised. Sometimes 8 hours a day. This is what I meant by obsession.

There were a few years when I had to work and push the piano-playing aside, but I always returned to it. There was always just too much value to it, emotionally and intellectually. It's probably good I did, too, given the frustration capacity for the profession I finally landed in.

I played piano wherever I could find one. I found that the Yamahas and the Kawais were the best pianos. And in that generation, this was true. I played a Bösendorfer once, but I found it to be stiff and unexpressive. I also played Steinways but found them to be unremarkable and sometimes quite muddy. In retrospect, it was probably just how well they were tuned.

In the early 1980s I was improvising quite a bit, with quite a varied style. A little modern, a little freestyle. You can check out the following improvisation to see what I was playing like in September 1981. often my improvs are titled based on the date they were recorded, in this case 09-22-81 #1 shows it was the first recording I did on the 22nd of September, 1981.


As the 80s wore on, I continued to play and write pieces. Here is another improvisation that I worked into a full-blown piano piece, called Black Widow. This piece is much more structured and rhythmically interesting. My style was progressing.


Eventually in the late 1980s I got the Yamaha 7-footer that I use today. I also had, for a while, a Yamaha 7-foot Diskclavier, but I didn't like the way it worked. Yes, I had two grand pianos for a while. I'm serious about pianos, and I probably always will be. But I also worked with synthesizers. Including a weighted-key synthesizer that still works great after so many years, the Roland RD-500. Supposedly it was a roadie's dream because it was so stable and bulletproof. As a digital piano, it also featured various kinds of electric piano styles. In 1990, I played the following improvisation. This was when I was about two months into the secret work on Painter, before another living soul saw what I was cooking up. It is interesting to see the sonorous moods I had in those periods. This improvisation does a lot to tell you the story behind my early work on Painter: intricate, creative, and spontaneous. It's called 11-10-90 #2. Note: sometimes I recorded more than one improv a day, if I was on a roll.


My improvisation recordings are entirely unedited, so if I make a mistake: it's all on me!

Also from this era is an improvisation that features more interesting rhythm but also a melodic inner section that has a quote from one of my songs from the era (which I no longer have a recording of) called keys to your heart. It's not bad. It's called e piano improv, a rather undescriptive title it's true.


When I moved out by the beach, for a while I had a house with room for two pianos. It was there that I did most of my modern production work with ProTools. But I continued to improvise. Even in a smaller, more manageable house I still have a 7-foot grand. And I play piano every day. When I'm writing songs, I will improvise on the theme of the song. Usually I will work up the song and record it onto my iPhone which at least gets it down for posterity. My worst problem is that I don't record the songs that I compose from day to day.

So, the nature of improvisation is that most of it goes unrecorded, which is a pity when I do a particularly brilliant one. So maybe one in twenty gets recorded.

After 39 years of piano-playing, my fingers and my brain are entirely in sync. Does this mean every note is perfect? No. But it does mean that I can maintain a rhythm while modulating from key to key. All the chords have familiar hand shapes, so I don't really look at the keys any more. Sometimes I play in the dark, but usually only when I am alone.

How I Improvise

So, how do I do it? What is going through my mind when I'm playing?

If you were to watch, you would just see me sit down and play.

With my hands on the keyboard, I focus first on the key I'm playing in. And whether or not I want to start on a chord that is moving or stable. Moving chords are like sevenths: minor sevenths, major sevenths, inversions (particularly those with the seventh in the bass). This is because sevenths typically resolve into more plain chords in a cadence. Here we have an A minor seventh chord resolving into a D major chord. These chords also have very natural hand shapes to me, and I favor them. But, of course, they can be expressed in any key and their hand shapes might be a bit different. Also, suspended fourths are unstable, and tend to resolve fairly quickly.

Playing in different keys is important. At one point, in in the mid-80s, I forced myself to improvise in a different key each time I sat down at the piano. I had a little tally sheet. J S Bach used to do this, I suspect, since his Well-Tempered Clavier has two books with a piece in every key. Fréderic Chopin also did this with his Preludes and Études.

Anyway, it's also important to try out a few rhythms with a chord sequence I work out. And also to try a few chord inversions, so the notes on the top can begin to form a cohesive melody.

When it comes to the bassline, I will find ways to work chromatic descending bass into my improvisation. This sounds very good, and often forms a focus point for a chord sequence. Sometimes I put the third in the bass. Sometimes the seventh goes into the bass.

Here the bass descends to the seventh and down to the third of the C chord, to finally rest on F, with an added 2nd (G) added on top. This kind of sequence works best when the bass is very low indeed, perhaps one to two octaves below where I've written it here.

After a number of years playing, I have tried out pretty much all the chord sequences there are. But of course that is wrong because there are plenty of chord sequences that sound terrible. And I naturally avoid those after perhaps making the mistake of trying them. Once.

Another thing to remember is that the melody doesn't have to fit the chord. You may find that the melody contains suspended notes, or even notes that are entirely off-chord. This can be to great effect.

Chord inversions are of interest, when you are playing. They can help add color to the chord. And sometimes inversions are totally necessary in creating the feel you want.

Here I have shown some inversions of the C ninth chord, with the root of the chord (C) taken out. It's only fair that someone should play only four notes at a time, to avoid too much of a handful of notes. The one I favor lately is the third in the sequence, with the G being played by the thumb.

Ninth chords are in general interesting because they produce a thicker chord. In C alone, there are at least four interesting ninth chords. I have shown some of them here.

They really only differ in whether the E and the B are natural or flatted.

The C major ninth chord (or CM9) is bright, a bit dissonant, and often leads into a C chord by having the top two notes raise diatonically up to C and E.

The C minor ninth chord (Cm9) is soft and brooding. There is less dissonance because there is no B natural. But now the D and the Eb form a direct dissonance.

The C ninth (C9) chord has no semitone dissonance, and has a lush, cheerful sound. It is descended directly from the C seventh chord, but has an added D.

The C minor major ninth (CmM9) chord combines the darkness of the major chord with the dissonance of a C major ninth chord, and can be a bit ghastly and tragic.

So you can alter the color of your music by using the right chord. This is particularly so in the first chord of your verse. For instance, Pink Floyd, in their song Breathe, starts their verse off with an Em9 chord, resolving to an A chord. It is the minor ninth chord, with its sonorous sound, yet tainted by the added 2nd, creates the ambience of the entire song. In fact, that particular song has some of the most perplexing and powerful chord sequences in the repertoire.

A Question and an Answer

It has always been interesting to me, since improvisation and music generation is so completely hardwired into my brain, if it might not also be possible to write a computer program to do something similar. I think harmony is pretty easy to encode into a computer program. Indeed, in 1975 I did such a thing.

But it is an entirely different thing to encode the structure of music into a computer program. I am, of course referring to the high-level structure of music. The ABA format, the romantic period sonata form with its introduction, exposition, development, recapitulation, and coda sections. If we look at modern songs, there is the intro/verse/refrain/bridge/outro structure and all the common rearrangements of them.

My question is this: even if all these forms were to be faithfully duplicated by a songwriting program, would the computer be able to replicate the angst, emotions, and desires of the human composer?

The answer is simple actually. Such a songwriting program would not be able to write the songs. That would be the human user's job. Why do I say this? Because with Painter, our job was to accurately collect the artist's expression, and faithfully reproduce it, allowing the artist's style to come through. Because Painter and the computer running it are enablers for human creativity. They allow you to do more than you could do before.

And this is precisely what a songwriting program would do for a composer: enable them to compose bigger and better pieces, and let their emotions and creativity come through like never before.

And What About Improvisation?

Well, after many years of work at the piano, my brain is that channeler of creativity. That enabler of emotional expression.

Maybe Fractal Design Composer is what's needed. Maybe I can take my improvisational skills and build a program around it.

Maybe that's where music and me come full circle.

Hmm.

Mess and Creativity

Organization is important. Without it we could never accomplish any task more complicated than tweeting. We all know intuitively when sloppiness works against our productivity. But at what point does organization work against our creative minds?

I hold that some disorganization, some mess, is required to get to a creative point. If a Rubik's cube couldn't be messed up, then it wouldn't be any fun to solve it. In fact, the messed-up Rubik's cube makes a nice symbol for intelligence in disarray.

While the cube provides some exercise for my mind in solving it, it also suggests that we need a mess to get into an analytical state as well. Or at least to practice our analytical side.

But the real value of mixing it up is to be presented with different things in different contexts. We night never put them together if it weren't for the mess in the first place. No, this is not about entropy and order. This is about the interconnectedness of thoughts. The complex relationships, similes, metaphors, and comparisons that our mind makes when presented with diverse options.

Operations on Ideas

One of my fields is imaging: pictures. When developing new imaging technologies, I come across many different techniques that apply to imaging. But the new techniques come from a specific set of operations on ideas that help me to cross disciplines and make something new.

The first operation is deconstruction. With this technique a problem or a subject is torn apart in different ways so we can see what it's made of. With images, this can be something like moving the image into the frequency domain. Or, it can be thinking of the image as a bunch of tiles, or a mosaic. Or re-representing an image as a Gaussian pyramid. You can think of this as the analytical pre-step in being creative. The more ways you can deconstruct something, the more likely you are to find something new.

A sculpture from Napoleon's Arc du Triomphe

Some years ago at Fractal Design I began getting interested in extracting directions from an image: another way to deconstruct an image. This was in order to create a Van Gogh effect, that represented an image using brush strokes that were aligned to the directions of an image.

This is as good a subject as any to introduce the next operation, random association. For something like directions, these associations are concepts like motion, velocity, vectors, maps, routes, paths, going the wrong way, a drunkard's path, alignment, perpendicular, turning the wheel, etc.

The image processed by representing it as directions

These ideas can lead me to look at things that aren't necessarily associated with images, like vector fields, tracing along a direction path, making directions random, canceling the randomness or uncertainty in directions, and other things that seem to be associated with directions.

I use this to help me in branching out, another operation in creative ideation. If I move from directions to vector fields, for instance, I can then look at various properties of vector fields, like vorticity. I can also look at operations that apply to vector fields, like div, grad, and curl that I might never think of when I'm focused solely on images.

Or, looking at direction as a velocity vector can lead me to realize that it has length and angle. Branching out from this can lead me to realize that a direction's angle can be changed. This has dramatic consequences for imaging, since it can lead to an altered reality.

The next creativity operation is experimentation: varying the parameters.

A picture of a tree can be quite picturesque, particularly one from the Kona coast. But when you take the directions in the image and rotate them all 40 degrees counterclockwise and mix them back into the image, you get a windswept alt-tree.

Very different from the original. By both branching out and experimenting, I have created a new effect and it has some profound visual consequences.

It so happens that this effect was accidentally discovered while working on something completely different (which I can't actually talk about)!

I will show few more pictures for the visual people among my readers.

To right you see a picture of me taken about a decade ago near the Statue of Liberty. Here I have applied maybe a fifteen-degree tilt to the directions.

There is a little bit of windswept effect from the direction alteration. This is just what I was going for, and it is fortuitous. With the Arc du Triomphe image, the directions weren't altered in the least. But there is some uncertainty to evaluating directions and, when this effect is applied at a certain scale, you can see that the smallest details aren't always preserved. If they were, it wouldn't be interesting in the least.

Apply this effect to clouds and make the directions perpendicular to their usual course, and you get a puffy, almost feathery, cloud.

It's a jittery, crazy kind of reconstruction of the cloud image.

This brings me to the last operation, which isn't really about creativity, but you can be creative about how you do it, of course: reconstruction. If you have figured out what something is made of, managed to jumble it up internally, then you can reconstruct it and hopefully not end up with Frankenstein.

Can we apply this to another field as well as images? Of course! In music, I deconstruct songs typically so I can reflect on how they are made. Then I can randomly associate the structure of music with, say, grammar. And I can branch out from the usual song grammar to use multiple linking sections, or to precede each verse with a little prelude (I did this in my song Baby, I). Or rearrange the chords backwards and see how that sounds. I did try recording a backwards guitar part in one of my songs, and the interesting process is chronicled in another blog post.

Or, I can deconstruct music into inter weavings of sound and silence. I can put moments of silence into a song, to create a full stop. Perhaps a fake ending, or just a de-textured rest. Yes, I used this in I Know You Know.

I can deconstruct sound into treble, midrange, and bass and create a wall-of-sound interpretation of music. Or deconstruct vocals from instruments. Randomly associating, I can have the vocals play the instrument parts, like the Beach Boys' Brian Wilson had his band do in Help Me, Rhonda.

One of my favorite creativity groups was the Traveling Wilburys. In Handle With Care, they changed the song structure several ways. First off, the song has 2 bridges. Then the solo comes in the fourth verse, and it's only a half-solo. The full solo comes at the end, after the repeat of the two bridges. After the fifth verse.

The Evils of Organization?

While organization is important to our everyday life, too much organization can also hamper us in ideation. The principle is this: if everything is in its own little box, then you simply can't imagine something that is out of the box. Is this even true?

As impossible as it seems, even obsessive-compulsive people can make great artists. This may be because OCD leads to various interesting styles, like horror vacui. Many indications of "madness" in art have drawn us to them in the past. VanGogh and Hieronymous Bosch stand out in my mind.

And there is a place for neat freak artists as well.

But somewhere in between neat freaks and slobs are all the rest of us just trying to be creative and solve problems in a new way. Bringing a fresh approach to something is what it's all about. And disorder (mixing it up) is just one tool to accomplish that goal. Analysis is just as valid, and it is actually necessary for deconstruction.

To properly deconstruct, you need to take a step back and look at your problem in a new way. For instance, looking at texture as geometry instead of pixels. Looking at stuff in different ways is really the true basis of creativity.

Look at this video of the Traveling Wilburys Inside Out. It's just the band playing, but you can see that they took song lyrics and looked at them in different ways. Thanks to the lyrical talents of Bob Dylan and Tom Petty. And the entire subject is inverted again in George Harrison's bridge "be careful where you're walking".

Check out my blog post on Where Do Ideas Come From to see some of these principles in practice, and the value of operating in a creative group, as the Wilburys did.

Writing Songs

Creativity serves us well, as a source of ideas. Sometimes it comes from the cross-pollenation of ideas between disciplines. Sometimes it seemingly comes from the blue when our minds work when we are asleep or when we are idle, mentally. And sometimes creativity is a visceral thing, springing forth from situations of trauma or intense joy, as our souls try to heal themselves or simply work to process what they have gone through.

This last source is so often the one that makes us sing.

When I get overwhelmed or troubled, I usually turn to the piano to work out my angst. And heaven help me if I don't have something to record it. I have captured these improvisations from time to time, and some days there are two or three.

I like to write songs, because I'm somebody who feels things. They are a great outlet. You can find some of my songs on Soundcloud. Check out Not Enough Time for instance.

Starting Songs

How do I write songs? I start three ways. The first way, I have something to say. If it needs to get out, melody and words will occur to me simultaneously. The second way, I think up a melody and I then need to put words to it. The third way, I write a song instrumentally, and I record it and play it back, writing words and framing phrases as I sing to it. But these three methods are only how it all starts.

Once it's started, then it needs to be completed.

Songs are typically made up of sections. You probably know these: verse, refrain, bridge, intro, outro, hook, etc.

Verse

Verses are the body of the song, where most of the matter of a song occurs. But what's in a verse?

A song can tell a story. Then the verses do the telling, scene by scene. A good example of this is Rocky Raccoon, by the Beatles. No doubt that's what is going on in that song!

A verse can tell a point of view; a later verse can alter that point of view. One verse can have my point of view, and the second can have your point of view. A third verse can apply to a group.

Verses can be similar in form and structure. For instance, Joni Mitchell's Clouds has three verses with similar structure. The first is about clouds, the second is about life, the third is about love. This is a common pattern, because, as listeners, we want to see where the songwriter is going with it. The similarity forms a comfort zone, a familiarity that helps us absorb each verse. Also, a principle of songwriting is: if you repeat something, do something different with it. So, for instance, you might bring in the drums and the bass on the second verse, for example. This is called buildup.

Sometimes having a similar structure can simply mean repeating the same words at the beginning of the verse (like Dear Prudence).

There is plenty of creativity in verses.

Verse patterns can also be interesting. Countless songs have two verses of vocals and the third verse is taken up by the solo. Then, the fourth verse often returns to vocals.

Refrain

Usually a hit song will have a refrain, or chorus. A refrain is most commonly a headline phrase, like Here Comes the Sun. A songwriter will save the most catchy melody or chord change for the refrain. It is generally the most memorable part of the song, and this is by design.

Sometimes the refrain is a round, or a four or eight bar phrase that gets repeated. Think Hey Jude. Even when the refrain is not a round, it can get repeated, so be prepared to rework the end of the refrain for the purpose of dovetailing with the beginning of the next repeat.

Usually you won't repeat the refrain the first time you introduce it. Repeats of the refrain are simply more common at the end of the song, because you want to leave the listener with the best and most catchy part in their mind when the song is done.

Bridge

The bridge, or middle eight, can link the verse to the refrain, or just be a nice contrast with the repetitive nature of the song. Sometimes the bridge is the jewel of the song. In my song Tagman, the bridge is a flash of virtuosity.

Often, in the bridge, I modulate to another key, for instance the relative minor. Then the trick is to modulate back into the main key of the song.

I like the texture of the bridge to be different from the rest of the song. This can mean a different rhythm (say, triplets), a more edgy drum beat, or even a totally different treatment like a classical version or a blues version.

Intro

The intro of the song is the part that gets the listener's attention. This is usually something novel. In Wish U Were Here, Pink Floyd begins their song with an AM radio-compressed guitar solo that another guitar, seemingly more live, begins to jam with. Now that's novel.

Some songs don't have an intro, like Ruby Tuesday, by the Rolling Stones. It just begins with the first verse directly. Other songs use the hook as their intro, which is quite common. Think Day Tripper, Satisfaction, and I Want To Tell You.

One particularly useful kind of intro is the rhythm intro, where the drums begin the rhythm of the song and it gets amplified with each repeat. Here, songs like Steve Winwood's Higher Love and Rob Thomas' Her Diamonds are good examples.

Some songs have two intros, like Queen's Somebody To Love. You get an a cappella section followed by a short piano intro.

Outro

Many songs use a modified intro as their outro. This provides a sense of closure, and is often poetic.

I think an outro is essential, because I tire of songs that just repeat a round at the end. Well, Hey Jude did it and others followed ad nauseam. But it seems like the sign of a song that isn't finished, when it just fades out at the end. But even epic, classic songs like Procol Harum's Whiter Shade of Pale do this.

The traditional musical term for outro is actually coda.

Hook

A hook is a riff that catches the ear directly because it has a novel rhythm and melody. Rock music is so full of good hooks. The opening bars of Tom Petty's Mary Jane's Last Dance certainly suffice as a simple example: a guitar riff that helps to build the rhythm of the song while catching the ear. The Beatles' Day Tripper also has the quintessential prime example.

But a hook must be integral with the song. If it is stand-alone then the song can seem manufactured, or formula.

I traditional music, the hook is often called the motiv.

An Example

Here I will take apart one of my songs. It's a bit like the reverse of what I did to put it together, and it's also a bit like how I put it together in the first place. But you can take apart any song, really. I think it makes me a little odd: instead of listening to a song in an unbiased way, I take it apart. "Oh, listen to that bridge!" "Wow - third verse a cappella, very interesting!" That sort of stuff. I probably drive people crazy with this.

The song Not Enough Time shows I have some organization when I write my songs. To the right you see page 1 of the lyric sheet. These were not just written the way you see them here. They were changed, listened to, endlessly edited, tried out, and rearranged.

First, I usually just get a verse idea in my head. Then it's just a problem of making sure its a good lead-in verse. For this one, the really big problem was getting the second two lines of the first verse right.

Then one day, it just occurred to me that dictionary could be pronounced the British way, and it all fit together. I was trying to get the cadence of accents of the syllables to fit into my meter.

One day, probably a year after I wrote the basic verse melody (which also fits the round) I figured out a nice bridge format. Then I sat down at the piano and put some chords down. Pretty soon I could play it reasonably well. This is the first step in becoming comfortable with a new section in a song.

The second lyric sheet has very few lyrics, but it highlights another problem with Not Enough Time.

How do you end a piece?

I referenced a concept before called an outro, which is a modern term for coda, the end of a composition. At the end there might be some kind of definitive statement. The Beatles used to sometimes end their songs on an unusual chord, like the fourth of the scale. This is used to great effect in And Your Bird Can Sing.

Interspersed in the lyrics is a reference to a round. This serves as the refrain in this piece.

You see, the more I played with the verse, the more I wanted to sculpt it into a 4-chord round.

Modern commercials are the home of the 4-chord round these days. And it does kind of make me a little ill. In the case of this song, it took probably three years before I had it right enough to produce.

Here you see the score sheet for the round. The piano part is very foursquare. That's to contrast with the rest of it, which tends to be syncopated.

Vocals 1 to 3 are harmony vocals that define the chord structure of the round: D - A - Gadd2 - A.

The 3 flutes are additional orchestration that adds color to the round.

The bass is highly idiosyncratic and natural. It bounces at first and then proceeds in small scales. Note that the bass has a different ending for the second repeat of the round.

The drums add a fast beat to the round, giving it energy. The kick and snare are flammed: in other words doubled.

The other two vocals actually wrap around the round, stretching into the next repeat. This is a nice trick if you can carry it off, and it makes the round seem much more natural.

Here we have the structure of the song. I always write a structure sheet out so I can remember the chords. If you forget the chords, all is lost!

Also notice the exact timing of the song is given, in mm (stands for Maelzel Metronome) and it is the number of beats per minute. In this case, it is the number of quarter notes per minute.

Here I give the round as D - A - G - A twice in a row. This also serves as the chord sequence of the verse. Except that the back end of the verse has a quite different set of chords.

The bridge is notable, with a chromatic bassline in the tail end.

The secret chords of the coda are revealed here, and they are quite a parallel harmony indeed.

For the back end of the verse, I did some harmonization using multi tracked vocals. This required some planning beforehand, to get the chords right, particularly when the G is compressed into a G2 chord, since the third of the scale (B) becomes replaced with the second (A) in the chord.
Here you can see that I did three vocals. My trick is to double-track each of the backing vocals to give a bit of chorusing to the sound. It makes the backing vocal fatter and slightly more ethereal.

Other vocals were multi tracked, and required a bit of preparation as well.

When you look at the structure, you see that the round serves as the intro, and the second round backs up the solo.

If you listen to the song, you might notice the backwards guitar solo in the second round, just before verse 3. I reversed the entire section, recorded three solos to it, and chose the one that sounded the best when I re-reversed the section (with the solos) back to forwards.

It is a bit disorienting to listen and practice to music that has been reversed. You should try it sometime. It really has a spooky feel to it.

This serves to show a bit about how I write songs. Most of it is hard work and meticulous planning. And the real ideas come after quite a while of letting the piece percolate into you brain.

I have a confession: this song is not typical. Most songs I can crank out in a couple of days. But songs always sound better when you sit back and listen to them after leaving them alone for a while. And new ideas are easier to come up with too.

When I write songs now, I usually start by singing them into an iPhone while driving. I use VoiceNotes for this, because it is simple and very handy for capturing ideas. Then, when I get home, and I have a moment of relative silence, I create a piano backing for the song. This I record using an iPad or some other handy tool, just so I can comp up the song in GarageBand. Quick and dirty.

I have used tools like ProTools in the past, for multitracking. I find that modern hardware has more than enough horsepower to do the same task without expensive outboard DSP cards.

I do love living in the future.

Where Do Ideas Come From?

Sometimes I have too many ideas. The problem becomes: which ideas might pan out, and how do they interrelate? But for most, it's more a question of: where do ideas come from? Let's talk about the various places.

Spontaneous Head Combustion

Sometimes an idea just pops into your head. I think this is because brains are always processing in the background. I had a calculus teacher in high school, Mr. Pearson, who said that the solution to a particular differential equation he had been working on would sometimes come to him while shaving.

Though that was a fairly nerdy thing to say, he wasn't far off my point.

The process of spontaneous idea generation is related to why we dream: the brain always wants to exercise in periods of disuse. When shaving or driving, your brain is not fully engaged. This means there are cycles that can be put to use. We will return to this later. But some people dream in ways that are connected to stuff they are working on. During my most productive periods when I get a cool idea and then rush to implement it (especially a very complex one that will probably take several days of constant work) I am told that I sometimes talk in my sleep: I'm mumbling variables and data structures. This definitely happened to me when I was working on Mosaics in Painter.

Hmm. Maybe I work too much!

But a simple way to use this is to think about a problem before you go to sleep, and your mind may come up with a solution while you sleep. In fact, I used to listen to music in my head right when I was going to sleep. I liked to listen to Beethoven's 9th Symphony in this state. Not really listening, per se. Actually just hearing it in my mind, every note and phrase. It was great practice for musical composition.

Sketch It Up

I like to sketch things out, and get ideas that way too. I guess it's because I'm a visual person. But, for me, lots of graphical ideas work themselves out on paper. A doodle can quickly become something larger.

I was working on borders and corners one day when I realized, as many people have, that you could represent the corners as four image pieces and the edges as another four image pieces. And the edges could be stretched or repeated. So I drew one of my favorite things, and I have reproduced it at left. I suppose it's a good example of three-dimensional thinking.

I draw in guide lines that showed the corner pieces and the edge pieces succinctly. Then I realized that these corners and edges could be filled with pretty much any image.

So i drew cylinders with pipes interconnecting them, like you see at right here. I realized that even shadows could be part of these clipped images. Note: this 3 x 3 partitioning is used to create buttons and other UI elements, also. In fact, that was what I was using it for at the time.

But I didn't want the UI to be constrained to be a rectangle. No matter! These pieces will still work, as long as all the corners are squarish. For instance:

At right, I have shown something that can be made out of the usual eight pieces, just rearranged in a different fashion from the typical arrangement I have shown so far.

This goes to show that it doesn't matter how you design it. If you take a little care, that design can service a much larger class of objects.

This design really hinged on two things: the corners and the edges. So I concentrated on the corners. What could the corner possibly be? A number of three-dimensional things popped into my head, and I sketched them up.

Here are a number of possibilities that occurred to me, drawn as corners. To left: a gabled approach that exhibits shading and has pyramidal corners. To right: that same approach, but with different edges. The edges can have their own articulations as well.  There are plenty of three-dimensional things that can grace the corner of a design.

Here, I explore the use of cubes and balls. To right I considered using a stake at each corner and a taut piece of rope in between. And knots on the rope in between the corners.

Of course, there are a number of articulations on the edge that can be used in addition to knots and spikes. Really, any repeating line pattern can be employed along the edge.

Here are some line patterns: wavy lines, beads, and twisted rope. Of course braids and yarn motifs are useful as well. But plain patterns, like stripes and diagonal "candy" stripes are also useful.






Exercising Your Brain

When I am otherwise not mentally engaged, I use that time for practice. Sometimes I do something tedious like factoring five-digit numbers in my head. You know, I like prime numbers. For instance, today I passed a house on San Jose-Soquel Road that had a number of 24,769. (I look on the right side as I'm driving home and there are often odd numbers like this one, which means they won't be divisible by 2.) I'll cut to the chase and tell you that it's 17 x 31 x 47. It took me about 4 minutes to determine that, performing long division in my head. Then, when I got home, I checked it using Siri.

It may be tedious, but it does help to keep my brain sharp when it comes to numbers. Another activity I do while driving is writing songs. I compose the melody and lyrics simultaneously. When I get an idea, I record it into Voice Notes, an iPhone app. I will go over the lyrics a few times and record a verse or a refrain.

This process keeps my creativity working while I would otherwise be bored. Exercising the right side of my brain. I used to turn on the radio while driving. I don't do that any more. I let my mind fill the silence.

I also got a great idea for a novel while driving. So I used Voice Notes to get it all down.

And, by the way, it might be a good idea to pull over, if you get too involved in an idea while driving!

Know Your Subject?

When I am wracking my brain for ideas on a particular thing, I can easily come to a standstill. It is at moments such as this that I take a step back. Specifically, this is called meta-level thinking. This process can actually be impeded by knowing too much about a subject. You can get into the mindset of thinking that it's all been said and done.

When writing a novel, it can be as simple as taking a step back and considering a character's arc. When writing music, this can manifest itself as rearranging the structure. Or adding in a full break or another device that so often makes songs interesting to our ears.

The Beatles practiced this. In We Can Work It Out, for instance, there is a section that breaks into triplets in an otherwise foursquare piece. On the back side of Abbey Road, they ran all the songs together into a single homogenous piece, a structural device.  In Getting Better, one of the verses becomes one long melody instead of several broken up melodies. On All You Need Is Love, the verses contain an extra half-measure per phrase, which helps to offset the otherwise foursquare melody (the Beatles called it the Dirge), the device of changing the time signature.  Or just having two authors can create a cognitive dissonance that can make a piece remarkable. On I've Got A Feeling, arguably the last composition that Lennon and McCartney wrote together (unless you consider the back side of Abbey Road to be a single piece), their contributions are quite different, and improve the song immeasurably. It goes on and on.

When programming, it helps to look at the problem in a different way. This can be a meta-level approach when you look at multiple ways of solving the problem, or you find a way to combine what would otherwise be different methods. This is particularly good when there is something you are missing.

Sometimes it's just a matter of employing multiple modes of thought at the same time.

Ideas Travel In Groups

When finding another person that is likewise creative, it is often possible to have a huge idea session. This has happened to me countless times with various people, including Tom Hedges and John Derry, the co-authors of early Painter.

A single session can generate hundreds of good ideas, I have found. When you get onto an interesting or fruitful subject, the result can be a veritable treasure trove of ideas.

This may be the Lennon and McCartney effect: the influence of two brains on creation is synergistically better than if the two created separately.

I have experienced this effect with John Derry on several occasions. Together we thought up Painter's image hose, for instance.

Combining Expertise

When you become an expert on one subject, it can help to give you the ability to solve problems in that subject. But is thorough research a ticket to stagnation?

I have found that it helps to be an expert on several subjects and then to combine the knowledge between the subjects. I'm a pretty good programmer, for instance, and I know how to draw. Combining them produced Painter.

Anyway, it's hard to ignore the profound influence of general thinking: knowing a reasonable amount about several subjects and thinking about how they relate.

Grist For the Mill

I'm not sure why it is, but I am equally sure that this is true: when we have emotional experiences, we are driven to create. This is typified by Lord Byron clutching his forehead at the top of the stair: the tortured artist. Or by Michelangelo undergoing great pains to complete the Sistene Chapel's frescoes: the agony and the ecstasy. Is it a cliché that only through pain can we create?

No, I don't think that it is. After hitting a patch of black ice and crashing my car in Denmark on the night of December 28, 1995, I sat down and wrote 7 really good poems. Sure, I write song lyrics, but poems? I was in a rare mood that evening after some wonderful "Good Samaritan" Danes stopped to pick me up and kindly delivered me to my Hotel in Copenhagen. Note that this near-death experience is not chronicled in my post Handling Serious Events. It seems I have more than my share of such events.

But, seriously, how many love songs are there? How many my baby done left me songs are there? I tell you: there's something to this theory of emotional distress and creativity. It's grist for the mill.

Seven Songs

This post has seven of my songs embedded in it, hosted by soundcloud.

By the way, mysterytrain is my soundcloud handle in case you wanted to scan for future postings or follow my songs. I'll be putting some more recent things up too, and some not-necessarily-finished stuff. Just so you can get an idea what I'm doing musically these days.

Hope you like 'em!

Tagman is a song with a catchy rhythm about a guy that flies a spaceship and stakes claims on mineral-rich asteroids. It's about someone who's the best. It has a nice descending-bass refrain and a virtuosic bridge. It was written in March 2001.

Tagman by mystery train

The song I Know You Know is kind of an elegy to spymasters, with a heavy dose of sarcasm. It was written in December 2000. It features a lush piano part and sections with very heavy vocal overdubs. One thing I was trying out were actual tiny silences interspersed with the music, interrupting its continuity. It seems like it works to great effect.

I Know You Know by mystery train

Give and Take was written in October 2001. It is about a relationship that has descended into spy vs. spy but should be a love relationship. Like I said in the notes on soundcloud, I always wanted to write a song with a kick like this one.

Give And Take by mystery train

Forever is a love song that talks about unending love. It has kind of a haunting guitar part and it was written in April 2002.

Forever by mystery train

Wonder Why was written in January 2002 and is more of a jazzy kind of a song. It has an interlude where piano, bass, drums, and guitar do a jam. Although all 4 parts were actually recorded separately by me. It has one of the more complicated rhythms of all my songs.

Wonder Why by mystery train

Baby, I was written in November 2001. It is an experiment that has a pop song, interspersed with small classical interludes. And it has a few ironic lyrics to it as well. The chords in the refrain are classic rock chords all the way.

Baby, I by mystery train

Not Enough Time is quite approachable and poppy. The lyrics and basic melody line was written in May 1995 and produced up in 2000. I think you'll like this one.

Not Enough Time by mystery train

Music

Music is about the direct application of creativity. And, since one of the main subjects of this blog is creativity, I will write for a bit about where it comes from, and then write about one of my favorite subjects: music.

Now, I have several areas of specialization, and I think most of you do as well. This capacity is what allows us to observe things better: more areas of expertise give us a better perspective on the world, a different perspective. But there is more to it than just observation. The more areas you know, the more you can apply what you know to other areas as well. The cross-application of modeling, technique, and problem-solving is crucial to creativity.

Music is also about expression: when you have something to say, it just comes out. When you don't, then it's hard to write a note. Emotional experiences can be grist for the mill.

Listening, Memorizing, Dreaming

So, when I talk about music, it is because it is one of my areas of expertise. My mother first introduced me to music when I was very young. We would practice singing songs when I was seven or so and this went on for quite a while. By the time I was 9 or so, I had built an intuitive notion of harmony. And I could use my voice, so that was my first instrument, I guess.

My sister Sue decided she was going to learn piano, and so my parents bought an upright piano. It was painted puke green, I remember. Also, it was tuned a semitone flat because its strings were too old and tight to be properly tuned. This wasn't a good sign, but it did work. Sue gave up her lessons because she simply didn't have the patience and stick-to-itiveness to continue for long. And the upright piano sat unused.

We had a neighbor across the street nicknamed Muffy that was a bit of a bohemian, and whose parents never seemed to be around. She was an acquaintance of my brother Jim. I remember crossing the street to Muffy's house to listen to her new Beatles albums when they came out. Particularly Rubber Soul and Revolver. I liked the Beatles and I knew every song by heart. One of my brothers got Rubber Soul for Christmas one year.

When I was 15 or so, my very liberal parents took me to see Stanley Kubrick's A Clockwork Orange. Once I saw that movie, I fell in love with the sound of Switched-On Bach, and later Walter Carlos' purely electronic arrangements of Bach, Scarlatti, Purcell, and especially Beethoven. Once I heard the ninth symphony, my life was changed forever.

I became a Bach and Beethoven freak, while still listening to and appreciating the pop music of the day. Particularly, along with the Beatles, I often listened to the Rolling Stones, the Who, the Beach Boys, and later, of course, Jimi Hendrix, Janis Joplin, Crosby, Stills, Nash and Young, and Joe Cocker. After Woodstock, there was plenty to listen to.

But I also absorbed almost all of Beethoven and Bach I could find. I started picking up and photocopying scores of Beethoven symphonies and Bach concerti. When I was 16, I went to UC Berkeley on a National Science Foundation math seminar for young adults, I located a real sheet music store on Durant and I got scores for both books of the Well-Tempered Clavier and the Brandenburg Concertos.

When I walked home from my friend's house at night, I got into the rhythm of the walking and began to hear music in my head. I found this endlessly fascinating. In 1972, I began being able to hear entire pieces of music in my head, usually just before I went to sleep. I knew Beethoven's 9th symphony so well that I could play it back in my head.

Self-Teaching, Improvising

So, when my parents took the rest of the family to Las Cruces in the summer of 1973, I was alone in the house with some musical scores and a working piano. And I had a stereo. So I cued up Brandenburg 5 on my record player and set the pocket score (the first page of the pocket score is shown to the right) on the piano and worked out what the notes were. By ear. Which wasn't easy because the piano was tuned flat by a semitone. But I still was able to work it out.

Yes, I taught myself piano that summer.

It took some time to figure out common music notation. Particularly the rules about accidentals, key signatures, time signatures, and rests. I copied the Berlioz Treatise on Orchestration so I had a good reference work to look at. It helped me to understand what instruments had what range and how an instrument such as a Cor Anglais might be used in practice.

I copied most of my scores from the San Jose Municipal Library, and from the San Jose State libraries. I often rode my bike from my home in Sunnyvale to those libraries. My mom had long-since grown tired of being a shuttle service.

It was only a matter of time before I began to write music as well. I had a notion to write a string quartet and that was fun for a while.

Early on, most of my time on the piano was spent improvising. And playing the piano part from Beethoven's Choral Fantasy Op. 80 (again and again), which drove my parents totally crazy.

I had heard that Beethoven was a superb improvisation-master.

Transcribing

To really write the music I wanted, I reasoned that I could just play it on the piano, record it, and then take it down. I could certainly figure out where the notes went, but that rhythm thing was a bit harder. I took down two of my improvisations, struggling through at first. Then I got better. It turned out I had quite an ear for it. Then I turned to the Beatles.

So, I transcribed Strawberry Fields Forever, note for note. The time I spent transcribing things and consequently attuning my ears to intervals and rhythm turned out to be quite useful to me.

Here are scans of my transcription, at least the first two staves of it. As I transcribed it, I was surprised at the number of times John Lennon would change time signatures. He probably thought of them as short measures.

At Caltech for a year in 1974-1975, I became familiar with the Grateful Dead (via the Wake of the Flood and Mars Hotel albums) and also Pink Floyd. I also introduced some Page House members to Beethoven's 9th Symphony. But I remember quite well I wasn't allowed to play Wagner's Ride of the Valkyries. A crew of 5 seniors would pick you up and throw you in the showers for that! Apparently there was some history concerning that piece at Caltech.

Also at Caltech, I audited some sessions with a dedicated music interpreter, James Boyk. He worked at Caltech for 30 years. I was lucky enough to catch up with him in his first year there and go through Chopin's Fantasie in F note by note. It's still one of my favorite pieces. I also took a course on music appreciation, in which we learned a few things about music that I already knew.

Later, after a year at Caltech, when I got into UC Berkeley for my college work, I had found that the school quota for Computer Science majors was full. So, determined to get in, I declared a music major and took the entrance exams. The first test was to take down a four-part chorale that a professor played twice. There were at least twenty students taking the test, in a classroom with a piano at the front. I found that I was able to transcribe it before anyone else because of my extensive practice listening and re-listening to a cassette tape, with my pen to paper. And I got accepted as a music major. The first year was hard for me, with second year harmony and first year musicianship. Music 27 was second year harmony, and I had to write several short compositions. One is shown to the right. It shows that I had a grasp of Chopin-like chromatic harmony even in 1975. The hardest thing we had to do was to harmonize a melody that the professor had composed. I always got an A at harmony.

The musicianship course was where I learned to play the piano and sing at the same time. Some days I spent 8 hours practicing and improvising on the Durant house piano. For my final, I had to sing and play Ich Grolle Nicht, from Robert Schumann's Dichterliebe. You can hear plenty of renditions of this song on YouTube.

Then I redeclared my major to Computer Science and the rest is history. But I never gave up playing the piano and singing. It's how I compose songs to this day.

Composing

My first amateurish compositions were done in 1973, before college. They were little fantasy pieces and preludes for the piano. In 1975, when I was taking a quarter off in between my Freshman year at Caltech and my Sophomore year at Berkeley, I took a course in orchestration at DeAnza college, from a cool professor, conductor, and cellist named Nelson Tandoc. He introduced me to the music of Sibelius. And part of my assignment was to write a piece to be performed, and so I wrote the Fantasia in D minor (the first page of the score is shown to right). Really it was more of a Toccata. I had it performed that year by an expert pianist, a woman whose name I can no longer remember.

Eventually I got off my classical composing bent. And moved on to more pop-oriented stuff: songs.

There were years when I couldn't spend a lot of time composing. I usually kept myself busy doing improvisation or by playing Beatles and Grateful Dead songs. My housemate, Jeff Bulf, was a devout Grateful Dead fan, and I went along to several Dead shows in Oakland, the Fillmore, and other venues. Of course, I had my own musical tastes at the time. I was into Pink Floyd, Yes, and eventually, Genesis.

Composing sticks with me to this day. I still write songs and play them for enjoyment. Or to express myself.

Recording

I have always been interested in recording as long as I can remember. When I got my first cassette recorder, I used it to record my piano work.

But I wanted more than anything to multi-track so I could record my voice. I knew I was good at harmony and all I needed was a multi-track recorder to prove it. In the late 70s I actually got a TEAC 4-track reel-to-reel recorder and did my first work. But it was hard to do, and wouldn't sync properly.

I gave up on recording for a decade. Later, I picked up an Alesis ADAT and started making 8-track recordings. And I produced my first song, with 4 vocal harmony tracks and a piano backing, which I laid down first. I finally realized just how hard it was to sing on key when I listened to my first recordings. I also realized the value of the diaphragm in creating sound.

There is nothing quite as humbling as hearing yourself on tape.

But I kept at it. Pretty soon, when I got enough money to afford a nice house in Aptos (after ImageStudio became successful) I started buying synthesizers and recording equipment. Eventually I put together a very good ProTools system with several signal processing boards. That system was capable of recording dozens of simultaneous tracks.

And I set about producing song after song for my own pleasure. Each one seemed better than the last. And this all happened as Painter was on the rise. Sometimes I would work with John Derry on a song or two. He was quite the guitarist.

Fractal Design had a band, with Bob Olander, Mary Mathis-Meltzer (now Mary Zimmer), Laurel Hemnes, John Derry, Tim Thomas, myself, and others that neither Mary nor I can recollect at the moment. I'm sure John Derry will send me a note, and I can add the others. I can remember banging out Lady Madonna, Midnight Hour, Mustang Sally, and I Want To Hold Your Hand on my road-tested Roland RD-500 mil-spec 88-key weighted digital piano. We played for the company at a summer event, and those were halcyon days indeed.

Here is one of my songs, Not Enough Time.

Not Enough Time by mystery train

When I finally sold off Painter to Corel and finished my consulting with them, it became a time to move into a smaller house and by that time I had three kids and less time, space, and silence to devote to recording. I'm sure I'll get back to it someday.

Music and Computers

There were periods when I spent time working on a crossover between mathematics and music. My first efforts in this area started at UC Berkeley in 1976, where a professor in Etcheverry Hall challenged me to write a program that could generate 4-part harmony. This was the first time I analyzed the 12-tone scale, created code (in Fortran, sadly) that handled accidentals and could represent chords and their inversions. I built a small expert system and an AI that could modulate from key to key and assign voices according to predefined preferences.

My next sojourn into computers and music began when I got my first Mac in 1984. Having met Owen Densmore, the printing person on the original Mac programming team, Tom Hedges and I managed to get ahold of an assembler, and a debugger. We started turning the Macintosh into a development platform. My interest in music surfaced once again and I made my first effort a music composition program. I could use the mouse to place notes and I could play back my composition. Of course, a much more interesting and lucrative pursuit was found by Tom Hedges when he designed an audio digitizer. I wrote the software, called SoundCap. And here again I brought my expertise in sound to play.

SoundCap got lots of users, including Andy Hertzfeld, Bill Atkinson, and Steve Capps from the original Macintosh team. But one of our customers was Bogas Productions. Now, Ed Bogas well-known as the composer of Linus and Lucy. And in those days he had some programmers, notably Ty Roberts and Neil Cormea working for him. As time went on, Tom and I consulted for them and worked on the digitization of piano tones. We bought a Mac-based music sequencer and Tom entered the entire Rhapsody in Blue into it (his father had been quite the piano player, it appeared, and Tom had the score in his possession). Whatever format it generated, we were able to read in and produce a real piano sound playing it, using our samples. This capability never really turned into a program we sold, however.

Tom was quite interested in scanning the sheet music and writing a program to recognize sheet music and convert it to some standard common music notation format. But, as it turned out, there was no such format at the time.

In 1988, as ImageStudio became quite a moneymaker for us, I started a little project on the side (I had a habit of doing this), called MIDI '88. This was a program that was a recording studio for MIDI music. This program was updated in when I renamed it MIDI '90. It had multi-track recording and playback of MIDI data. It could display what you played as a piano roll, unrolled sideways, so time moved left to right.

I remember demonstrating it to journeyman musician Todd Rundgren, and his first comment was "why doesn't it run top to bottom like the scroll in a player-piano?". Hmm.

With this software, I could play an improvisation, fix finger-faults in it, adjust the timeline so it fit a nice beat, and try it out on a host of different patches and synths. It was using this software that I taught myself proper drum-lines and bass-lines.

You see, when I listen to music, I deconstruct it in my head, listening to each part separately. If some piece has a great bass-line, then I can tell immediately. When it modulates to another key for the bridge, I can tell you what kind of modulation just occurred.

I still have lots of MIDI files sitting around: nearly all of my old improvisation since 1988. And my MIDI program is now called MIDI 2008, has migrated onto Mac OS X, and sports a UI that makes it quite useful in constructing song organizations out of chord sequences. A small screen-shot is shown at right.

Practice Makes Perfect

So I learned every Beatles song I could, and practiced them on the piano. Many of them I can play and sing on sight. The rest I can just play.

And I turned to The Dark Side of the Moon, by Pink Floyd. I have practiced and played all of them as well.  Most songs I hear on the radio, if I hear them often enough, I can play.

A couple of years ago, I learned to play and sing Daniel Powter's song Bad Day. Not a bad tune, that.

Lately

In the last two years I have been writing again, maybe ten songs. There is always something to sing about now, good or bad.

I have learned the craft of songwriting. I can write a mean intro and a catchy outro. I know my way around verse, bridge, and refrain. I can multi-track the harmonies when I want a particular sound. It has served me well over the years.

PostScript: I want to post some of my songs, but it seems to be quite difficult to do so with the blogging tools I have at hand. Still working on it!