The Classical Schrödinger Equation

There is a common view, perhaps, that discovery is straightforward and the people involved know what is going on. In the interest of encouraging independent thought, and experimentation, I would like to present some apposite opposite evidence.

In my view, the course of history rarely runs straight, and it is often more confusing to those directly involved. You may think that you are doing one thing, but soon events and knowledge overtake you and you realize that you are actually doing something different. That has been my stark experience several times in scientific research.

I would like to present that experience in summary because I hope it will encourage young researchers to stick at their investigations.

Do not be dissuaded from your course of action by uncertainty, peer resistance or non-comprehension – whether your own, or that of ofter people. We are all human and the wonder of the new is that it can be most perplexing. Nature is a Chinese Puzzle. She has her way to be — your task is to find that out. The true Scientist is compelled to mold his or her mind to Nature and not Public Superstition.

Follow Nature where the evidence leads you, even against the Conventional Wisdom.

So it was for me in early 1991. I had just finished my first publication dealing with my doctoral work on Quantum Inference. This was one of the early works on Quantum Information Theory. It showed that the Heisenberg Uncertainty Principle was more subtle than people first supposed. When you take account of the statistical Law of Large Numbers, then you discover that quantum states can be precisely measured. However, you would need an infinite supply of identically prepared systems to be able to do that. I was looking for applications of Quantum Inference and so thought of experimental tests of the quantum transformation theory.

I reasoned that if you could measure states precisely, which you can given enough of them, then you might make a sensitive test of the so-called Wigner theorem. I started out as an experimentalist in Chemical Physics, so I knew all about ultra-cold atomic and molecular beams. I had done my Bachelor of Science thesis in the Research School at ANU on Vibrational Pre-Dissociation Spectroscopy. Using an atomic-beam setup, to shoot cold atoms through a cavity, you should be able to detect non-linearity via a failure of Wigner’s theorem. This is the simple thought that got me started.

So I started looking at possible generalizations of the Schrödinger equation. I knew there had been some work and that the subject was of considerable mathematical difficulty. Hence I resolved to test my understanding with a simpler problem. Since quantum measurement involves the coupling of a formerly well-isolated quantum system to a classical-level measuring device I wondered if the classical limit might involve some non-linearity. Perhaps when a whole bunch of particles got together, there would be a stronger non-linearity. It made sense to me, given that the Schrödinger Cat Paradox only applies to linear theories.

To test this idea I framed a simple mathematical question:

Can one find a wave-equation which propagates waves so that their position and momentum expectation values follow classical trajectories?

Of course, I knew about the Ehrenfest Theorem, but I also knew it was approximate. The goal was to look for an exact equation. Obviously, that could not possibly be the ordinary linear Schrödinger equation. It took me a whole year of constant reflection. Then the answer suddenly struck me like a bolt from the blue:

Of course you could! Moreover, the answer would be unique.

It all followed from a beautifully simple piece of group theory due to Herman Weyl.

I could see the entire proof laid out before my eyes. It was, in essence, a very simple geometrical construction. It was so simple I could not believe it. However, it was also perplexing because I knew I had done something, but could not figure out what.

It can be that way in mathematics sometimes. People like to pretend they know what they are doing when they do mathematics. This is a fiction. If you know what you are doing it is certainly old mathematics. I knew I had new mathematics. Then things fell into place. I could prove the equation was necessarily non-linear, and was unique.

Aha! I said. This is the nonlinear representation theory of dynamical systems.

Reasoning about my own mathematics, I recalled a paper by Weinberg which I had glanced at when finishing my PhD thesis. It was called Testing Quantum Mechanics. In a frenzy, I searched for his paper and was immediately gratified that the equations looked different. However, I am a good enough mathematician to know that simple appearances can be deceptive. Sure enough, I figured out how to generalize his scheme to wave-functions and find an equivalent result.

The Classical Schrödinger Equation was for real, both ways!

I was elated for a simple reason. Many people might assume that it is discouraging to find prior work on a similar track. This is not so. I was elated because I had found something peculiar in my mathematics.

Operators were secondary to wave-functions.

Try as I might, I could not banish this feature. Once I found the result in the equivalent scheme of Weinberg I could see why. I had discovered something. Schrödinger mechanics was more general than Heisenberg mechanics.

Operators were an artifact.

Furthermore, the Classical Schrödinger Equation remained exact for any value of the Planck constant. This was a great surprise. I checked and re-checked everything but it always came out the same way. Greatly puzzled, I figured out why. It involved a Taylor series style expansion that got you back the ordinary Schrödinger Equation.

Excitedly, I prepared a summary of my result and sent it to Weinberg. He was kind and very generous with his time and I presented a seminar at the University of Texas, at Austin on Jan 30th 1992.

I was very nervous and prepared and rehearsed my talk extensively. At that time, I had very little seminar experience. When you first start out it is difficult to get much practice. I worried greatly that I would make a slip in my math somewhere. This is natural for young people. However, important work is never that way.

If I can offer any advice to young researchers it would be this:

Frame and motivate your question clearly.

So, I gave my talk. The result was silence. I was baffled and perturbed. Weinberg was kind enough to ask me later what I thought my talk was really about. I explained that the scheme of mathematics he had proposed for testing quantum mechanics actually contained a complete and exact copy of classical mechanics.

He looked at me intently and said: Oh!

In that moment, I knew I had a problem. As the saying goes:

You broke it, you fix it!

Later I completed the mathematical proof that Copenhagen Quantum Mechanics is Incomplete. I was grateful nobody noticed.

It is a bad feeling to break a perfectly good theory when you don’t know how to fix it.

However, where there is a will there is a way…