Follow by Email

Wednesday, March 28, 2012

Thinking Backwards

Most of us naturally think of time as an arrow that always points from the past to the future. This is why it might be so difficult for us to think backwards. But, you might ask, why do it? There are reasons, and some of them are not only compelling, but also fundamentally necessary to our way of life.

When I write code, bugs inevitably happen. Bugs are unforeseen problems that result in a crash or some other detectable error. And there I am, in the debugger, looking at the error. So I have to think backwards: given that this happened, what could have caused it? There are only so many possibilities. As we rule them out, whatever is left, however improbable, is the answer. This allows us to catch the problem before it happens, and then we can trace backwards from that problem using the same sequence of deduction. This proceeds backwards in causality until we find the source of the problem: generally the small wrong thing that snowballed into the error or the crash.

Imagine a homicide detective, presented with a dead body. Or a National Transportation Safety Board inspector at a plane crash site. What begins innocuously as debugging becomes a troubleshooting process that can apply to a much more dramatic and even fatal series of events. Events which must be traced back to their root cause so we can figure out how to prevent them from happening again and continue to believe that the world is safe.

But there are creative reasons to think backwards as well. And these are also more than simply exercises for our minds.

Distortion Filters

The filters in PhotoBooth and other Core Image applications often allow us to distort images in interesting ways, like a fun-house mirror.

Once you have learned analytic geometry and also exercise a bit of creativity, it is relatively easy to develop a function that can, for instance, create a bulge in an image.

A circular bulge like the one that makes my nose way too big is a piecewise mathematical function that conceptually takes a point in the source and transforms it into a point in the destination.

To do this, we envision a circle or radius R about the center O of the distortion. Point P may lay inside the circle, and we create a vector V from O to P (which is simply P - O). Note that the vector actually has two components, an x and a y component.

With this in mind, we can build up the distortion as a function based on these things. For instance, we can compute the length of V, and this we will call D. A function that goes from 0 at the origin O to 1 at radius R is simply evaluated by f = D/R. Actually we want f to be 1 outside the circle, so f = min(D/R, 1). And we want f to be a smooth function as well, so we might want f = 3*f*f - 2*f*f*f (in other words three f squared minus two f cubed, which is the smoothstep function).

To create a distortion, we must alter the scale of the image. To increase the size in the center, we want the scale to be greater than 1, and to make the distortion mesh with the undistorted image at the edge, we want the scale to be exactly 1 at radius R. If we call the bulge factor at the center B (B is 1.5 or so at the center of the distorted image above), then we can define the scale of the distortion to be s = B + (1-B)*f. This arranges the scale to be B at the center O and it will smoothly transition to 1 at radius R from O.

And the distortion itself becomes P' = O + s*V. In other words we are scaling the vector displacement from O by the transitioning scale factor s. Since s becomes 1 at radius R and beyond, the image is left undistorted outside the circle of effect.

But wait, we have made a huge error! It turns out that we are evaluating each point of the destination and we need to map back to the source! So we need to think backwards to get to the source point from a point in the destination.

Well, we don't actually need to do that, it turns out. We can fake it by simply making the scale less than one at the center to make the bulge get bigger, and correspondingly make the scale greater than one at the center to make the bulge a pinch instead.

You see, all distortion filters work this way: they must work backwards from a point in the destination to a point in the source. This makes it an exercise in thinking backwards.

Backwards Guitar

When writing the song Not Enough Time (listen to it on, I decided to make the second round solo an exercise in backwards guitar (which is at 1:29 in the song). To construct this, I had to do several things backwards. First, I had to create a time-reversed version of the song. Then I had to listen to it backwards and jam to the appropriate section of it until I had a basic idea of what I wanted the solo to sound like. This had to mesh with the forwards version, and so I constrained the start and the end notes to what I wanted to hear in the forwards version.

Sounds easy so far, right? Well, continuing, I mapped out the chords of the round, which is repeated twice, and reversed them. I have to say that the chords do sound eerie backwards because cadences are not what we expect them to be.

In fact, listening to and jamming to a backwards song is very unsettling. You feel like you are in a different world while you are doing it.

I recorded three solos in all, and then I reversed them again back into forwards time. Each one was then played with the song in its normal form. One sounded best, so I kept it.

Whew! Thinking backwards takes a lot of preparation in this case, and it's not an easy thing to do.

This all goes to show that thinking backwards is actually more useful than it appears at first thought. And our world is immeasurably better because we can do it.


  1. In math and programming theory, we have the duality of concepts algebra and coalgebra, or inductive and coinductive respectively.

    In type theory (or sets), we have logical conjunction (union) and disjunction (intersection), or greatest lower and least upper bounds respectively.

    However, I have posited that w.r.t. to the World state, time is irreversible, because we can't enumerate the World state:

    (this brings us back to the discussion in the Future part 1 blog about infinity and 2nd law of thermo that possibilities/disorder is always increasing for the universe as a whole. My theory was the degrees-of-freedom is conserved on the universal scale, i.e. Coase's theorem that any local order that decreases the degrees-of-freedom will ultimately fail or be routed around. That doesn't mean that local orders and their reversibility is useless, and it is even possible some cases increase the degrees-of-freedom on other considerations such as creativity and thus be long-lived paradigms).

    1. It is because of the Second Law of Thermo that it takes so much work to trace a chain of causality backwards. Often the clues are eradicated by the noise of possible consequences.

      When programming, we are keeping a pact with our code, at least we are attempting to establish one. When we make mistakes and the pact is no longer kept, then things break, even degrees of freedom.

      There is lots of work that considers time-reversibility. Let me just say that brainpower can look backwards into time with uncertainty which is exponential in the number of steps we wish to probe. We can use concepts of likelihood (Occam's razor and Bayesian methods) to increase our certainty and bring past events into greater focus.

      When recording sound, time-reversibility is a curious capability, since we would have very little understanding if it weren't for our ability to record stuff and play it backwards. This can open our minds to the concept of backwards flow, that really does not exist in any reference frame in the real universe.

      Singing back to a time-reversed sequence is spooky, almost as spooky as the action at a distance of quantum entanglement. Some things are not natural, but we can get used to them because we have an incredible tool that can even make sense of things that do not make sense: our minds.

    2. Some rambling thoughts...

      Recording a sound discards infinite amounts of information, e.g. we didn't have a microphone at every possible axis, distance, polar sensitivity variant, etc.. So we are replying an aliasing version of history.

      Our perception of the present is also aliased for the same reason, that our sampling rate is not infinite. Two observers can perceive a different simultaneous reality. A transducer (or transformation) between realities (perceptions) can turn mutual noise into mutual signal.

      The uncertainty is introduced when we want coherence (resonance) between mutual observation. Degrees-of-freedom (potential energy) is relative to the mutual perspectives in play.

      If I remember correctly, an inductive type or algebra is constructed from an initial condition in the past with the future open-ended, and a coalgebra or coinductive type is constructed towards a final condition in the future with the past open-ended. An example of a coinductive type is a stream, where the known future final condition is when the stream is deconstructed (closed).

      If travel backwards in stream time, we have to worry about coherence with other events in the past.

      If you actually spoke the words of a song backwards, the physics of the environmental would probably be altered so that just playing one possible recording backwards is incoherent with what could be modeled by multiple observers of a real backwards event.

      The reversibility of time is thus an instance of a limited mutual coherence, which is really a forward event in time. But forward reality is also a limited mutual coherence. Thus the perception of the direction of time is a limited mutual coherence. Ditto degrees-of-freedom.

      So maybe dark matter is the infinite alternative realities (perceptions) that are only coherent to our current shared perception of the universe, at the gravitational event horizon. What I mean for example, is that I can't perceive everything that you do every instant of the mutual notion of forward time we share, so that information (a form of matter) is only visible to me at the gravitational event horizon, where you also perceive mine.

    3. See also my comment in your Energy 1 blog.

    4. Sound is really very one-dimensional, compared to video. So it has a very simple forwards-backwards aspect. And it's cool how pitch is preserved.

      For me, backwards recording is really a creative outlet, because it sounds cool. So I don't really stop to think much about the causality of it. Nor do I think much about the loss of information, except when considering the imperfections of the recording such as room echo etc.

      When I record against a backwards audio clip, I realize that reversed time is a creepy place. I don't think I would want to live there. I think it's the only experience I will ever have with reversed time, really. I have reversed videos and I also get a strange experience. It's totally outside our frame of reference.

      But that doesn't mean we can't think about what it means.

      I have often thought of 1-to-1 image functions as being time reversible and thus a possible useful crypto system. Perhaps this relates to coinduction.

    5. "Sound is really very one-dimensional"

      I don't get this. Any sample of sound is dependent on the environmental variables (chamber acoustics, transducer Q, etc), which are infinitely variable.

      "imperfections of the recording such as room echo"

      What is the perfect reference point?

      Isn't all relative to the observer's preference.

      "reversed time is a creepy place"

      Agreed, it is very challenging for us to adapt outside of our established comfortable perceptions.

      Single level undo is not so creepy. But unlimited undo can be incoherent in some scenarios, and especially selective ancient undo.

      Perhaps it is the illusion of control we don't want to give up. The expansion of possibilities is something we don't face when we choose how much we deviate from the things we think will be reproducible. But this is a delusion, c.f. Handling Serious Events blog and my latest comment in the Transparency blog about the illusion of housing net worth.

      Your blogs Mess and Creativity and Different Modes of Thought relates in that we embrace random exploration to find new frames of reference. I definitely lean to the random thought process exploration (deconstruction), but I do try to come back and solidify with some ordered quantification (reconstruction).

  2. I should have mentioned DeMorgan's Law and also for other readers, much of that comes from a course in Discrete Math, as I am sure you know.

    1. Although it is a tiny bit convoluted, DeMorgan's law does make sense to me. And I figured it out before I took that course on Discrete Math. This happened one evening at Calma over a logic simplification problem.

      I suppose this is one of the reasons I program.