Ptolemy Cosmographia
Ptolemy Cosmographia of Black Sea States (1467)

One of the key reasons why I abandoned Academic Physics was the extraordinarily narrow-minded approach of the mainstream to scientific thinking. It was not always so, and certainly it did not feel that way in high school. However, once I got into the post-doctoral game it became clear that the operating ethos was political.

People did not appear to be responding to research results with dispassionate scientific interest. Rather the game had taken on some typical aspects of political economy. Pay attention to the existing structures of power, work out where the dominant economic interest lay and then dutifully do or say anything to support that clique.

This is not a bad way to live and probably quite sensible to build a career and to feed a family. However, it does not seem much like the Scientific Attitude which attracted me to study science as a child. Quite the opposite, it felt exactly like the religious environment of faith that I grew up with as a child. I had rejected Religion at the age of three, and did not want a physics career badly enough to concede defeat at thirty three.

So you proved mathematically that the Copenhagen Interpretation was incomplete? Who cares? We will still think that way because otherwise String Theory would not make any physical sense. We would prefer not to listen and will simply ignore you.

Did we ever get any evidence for String Theory? No! Who cares? You can get public funding to do String Theory so it must be right. In short, the game has become an economically driven contest for survival and not knowledge.

That is okay by the way. It simply means that Academic Science will cease to advance. It becomes a turgid backwater not unlike the Priesthood in Medieval times. In those days, the Monastery was a great storehouse of learning, but it did not really advance knowledge. It simply perpetuated the established and inherited views of classical Greece. The authority of Aristotle dominated and could not be challenged.

I am sure that the Medieval Scriptorium probably had a career structure with something like an H-index. The more you copied, the higher your reputation!

For some odd reason, which I do not comprehend, Western Society seems to have stepped backwards towards this old kind of close-minded perpetuation of established truths and verities. They even had a name for it. When you did an experiment and got exactly what you expected they called it a: Brilliant Confirmation!.

This situation is very odd. Indeed, the contemporary scientific literature is positively bulging with speculative innovations that purport to be new. However, what seems to unite this innovation thrust in physics is an unspoken agreement not to ever try and place any new idea in its proper historical context. Further, the Golden Rule of New Stuff is: don’t be stupid enough to make it testable.

It is okay to plant 10^1000 Angels on sundry pins in 10^500 Universes! However, thou shalt not explore anything testable which contradicts the Copenhagen Canon!

For instance, you find many thinking physicists today happily espousing the Everett theory of Many-Worlds. However, they generally fail to acknowledge that: 1) this is considerably less economical as a theory than the realist view of Erwin Schrödinger; 2) is not testable with any new predictions; and 3) is not robust to any failure of the super-position principle. Put simply: folks are inherently signing up to a world view (a Cosmographia) of positively Byzantine complexity which is:

1) at odds with direct experience (so must be sold as received wisdom)
2) not robust to changing a single assumption (such as relaxing quantum linearity)
3) computationally useless (provides no point of difference or expanded scope)
4) untestable for deviations (makes a real nice Faith but not a good Science)

This type of situation seems symptomatic of our time.

In the unrelenting thirst for Novelty at All Costs we seem to have forgotten the roots of science. As an antidote to this Scientific Rot I composed my own Cosmographical Glass. This diagram is my attempt to sum up the Conundrum of Modern Physics.

Is the Tree of Field Theory Forked?
Is the Tree of Field Theory Forked?

In this picture, I display a tree with a forked trunk. On one side, the views handed us by Bohr. On the other side, the views handed us by Schrödinger.

As my Newtonian Quantum Gravity paper showed, these are testably different theories.

The tree metaphor simply records that knowledge builds upon itself. It accretes over time, but must build on solid roots and a firm foundation. However, any student of history should know that the path of history is crooked. The tree is forked because Physics took a turn at the 1927 Solvay conference to socially endorse one true path (that of Bohr). As a result, interest in alternatives fell away and is now treated as a philosophical pursuit.

At one time we believe one thing; at another our view takes a turn on some nuance. The journalistic view of modern times is quite false in its assessments. We are supposed to believe that only the last three minutes matters and that all which came before is usurped in a day. Of course that is nonsense.

In the above diagram, I clearly display a real dichotomy of thought from the last one hundred years. The dichotomy stems from two separate (and ancient) contending views of Nature. Each was present in the Ancient times of Greece. On one view, Nature is discrete and composed of separated and separable pieces, particles, lumps… whatever. In the other view, Nature is continuous and composed of one holistic unity, wave, fluid… whatever. These two views contend today, as they have for 2000 years.

The scientific training of our time is exceedingly poor in that it fails to even acknowledge that such a story can be legitimately told about our understanding of quantum nature. Far worse, the collective progress of the last 87 years since the original papers of Schrödinger seems to be passively ignored, if not actively suppressed.

It is a situation akin to that wonderful poem by Robert Frost: The Road Not Taken. The mainstream has gone one way (Bohr), and I the other (Schrödinger).

Personally, I do not think this state of affairs is malicious. I simply think that humanity went through a major psychological disruption during the last century (World Wars, Atomic Weapons, Biological Weapons etc). This is now coming to a head with the present mass denial of Climate Change Science. It is (as if) the human race considers this moment to be always and perpetually disconnected and special, as though it has no real historical context. Hey, it is all Going up in Smoke so Who the Hell Cares?

Of course, this is a false viewpoint. Every social trend that happens today has occurred before: widespread fraud; banking crises; public distrust of leaders; the disconnect of plutocracy beside poverty; high learning and low conceit. There is nothing new under the sun. The forces of change forever contend with the forces of denial.

However, the Internet is as a force as Big as Gutenberg. That I firmly believe.

A little over a year ago, I dusted off some old scientific work to resurrect a few ideas in blog form. Now, through the power of the internet, I have discovered a community of like minds and a vast repository of search accessible historical literature. This is fantastic for the development of human knowledge.

While the Academy may struggle with the issues of the Medieval Monastery we have Freemen Now! The feudal days of Paying Obeisance to some Liege for Tenure are over forever. The world of Privateer Science is now alive and active with potentially enormous trans-formative economic power.

In the Cosmographic frame, let me fill in the above picture with one closing remark.

The left hand side of the tree is Copenhagen Physics.

The right hand side of the tree is Mongol Physics.

What is Mongol Physics? It is the systematic exploration and development of the un-named, un-explored, un-acknowledged Other Physics that Bohr, Heisenberg, Dirac and others refused to explore. It is what Schrödinger, Einstein, de Broglie, Barut and others thought feasible. It is a continuum field theory based upon a realist interpretation.

It is the knowledge that the Academy refuses to collate, develop, classify and codify.

It is Mongol Physics:

Taking over Physics, one equation at a time!

Saddle up and ride, Nomads. There is a Wide World of Physics to Conquer.

Beltrami Fields

Ingenio Eugenio (Beltrami)
Ingenio Eugenio (Beltrami)

“It’s not a fragrant world”
Raymond Chandler

Recently, our tribal leader has expressed his will for a trip into the depths of the remote past. There lies a hidden treasure of secrets covered under tons of malodorous Danish Blue otherwise known as the Copenhagen Interpretation.

Having had the same feeling for years, I asked a million loyal Mongol bots to complete their “Seek and Loot” mission among old and dusty library shelves, forgotten drawers and even locked chests. The loot is in, and all prisoners have been interrogated. Just as I expected, their testimony reveals that we have perhaps come full circle.

Readers may recall the disarray caused to the old guard of 19th century electrodynamics when they realized the conundrum caused by a circulating electron. Why did it not radiate and fall in upon the nucleus? This gave rise to the famous Atomic Stability problem.

Even today, many still think that this problem has no solution in classical electrodynamics. However, people of the “Old Kingdom” were unaware of the richness of certain classical solutions of Maxwell equations. The key is a “Radiation Cancellation” condition, wherein the field is dynamic but not dissipative.

The first example involved an homogeneous, isotropic sphere of charge that is unable to radiate when in purely radial oscillation. Subsequently, Langevin examined the case of a wall of charge in motion. Progress remained slow until 1948 when Bohm and Weinstein published a general study of Non-Radiating Distributions in the Physical Review.

The first to notice the importance of such Radiation-less Conditions for Quantum Mechanics was Goedecke with his landmark 1964 paper:

Classically Radiationless Motions and Possible Implications for Quantum Theory

where we seem important connections made with quantum theory.

Finally, in 1973, the seminal paper of Devaney and Wolf entitled:

Radiating and Nonradiating Classical Current Distributions and the Fields They Generate

saw the light of day. Here a magnificent truth is finally revealed in all its glory: an important general formula with which every radiating current must abide.

It is not surprising that Devaney and Wolf reached their conclusion in an effort to solve a practical problem unrelated to difficulties with QM. They focused on the task of recognizing and accurately identifying a radiation source. This requires the solution of an inverse problem of great significance to radar research and other such applications.

Decades later, two more pioneers, Marengo and Ziolkowsky continued this early work and recovered the full condition for a non-radiating current. It is quite suggestive to take a look at the specific formula appearing in p.3346 as eq.(4) of this paper which describes a condition on the current.

Perhaps the most important, the most significant, characteristic of this formula is a category of current flows that simply fail to obey the given condition. Such currents give rise to a host of possible Non-Radiating solutions!

The origin of this neglected possibility is to be found in 19th Century Hydrodynamics, but does not appear widely known, as judged by the contemporary Wiki consensus: see Non-Radiation Condition. The missing link consists of the Maren-Ziolkowsky work combined with the concept of a Force-Free Magnetic Field

It is then sufficient to take the formula for the eigenfield of the rotation operator, the “curl”, and instead of applying it to the magnetic field, apply it directly to the current sources.  All sorts of strange things will then start coming out of the hat. One will be forced to conclude that, under such circumstances, there will be a myriad of strange, tangled current flows giving rise to non-radiating, non-stationary charge volumes when the rotation eigenvalue is not a constant number.

What we are facing here is clearly a case of general conditions being reduced and studied only as special cases. Such oversights in science can happen. However, it is perhaps embarrassing that no mention, and no credit, is given to the father of the “force-free” condition, although he was well known in his own time. One has to dig under tons of old literature to rediscover the work of Eugenio Beltrami in the general classification of the different types of hydrodynamic flows.

It was Beltrami who explored eigen-fields of the rotation operator, which first appeared in his 1889 “Considerationi Hidrodynamiche”, Rend. Reale Ist. Lombardo 22. Some translations of extracts of these works are listed here:

The Importance of Eugenio Beltrami’s Hydroelectrodynamics

On the Mathematical Theory of Electrodynamic Solenoids

Considerations on Hydrodynamics

In our view, such vector fields might properly be called Beltrami Fields. They constitute eigen-fields of the rotation operator with a non-constant eigenvalue. As such, they can never satisfy the radiation condition of Devaney-Marengo-Ziolkowsky!

Evidently, it may be worth reexamining certain assumptions. Especially when one knows of other works in Solar Heliodynamics and Magneto-Hydrodynamics (MHD) where such general solutions of Beltrami equations lead to “Compartmentalised” solutions.

These are conceptually quite close to the apparent “quantisation” conditions only with some new ingredients that may perhaps serve as a model for an extended non-linear theory including the essential ingredient of “mass” as a self-interaction term as already proposed by our tribal leader.

In the end, I would not be surprised to find some similarity between such controversial proposals as “Spherical Lightning” and protons and electrons.

At least to this, I have had a minor contribution:

A Transmission Line Model for the Spherical Beltrami Problem

As for the rest, perhaps we must wait for the Shamans to fly away from their present obsession with branes, strings and other sundry instruments of mind haze.

[Editor’s note: An interesting and heartfelt post. Theo is encouraging us all to re-examine the classical works of Eugenio Beltrami and perhaps widen our field of vision. I would add one thought to amplify what he is perhaps alluding to at the conclusion. Locating self-consistent solutions to the Maxwell-Dirac and Einstein-Maxwell-Dirac equations has, so far, resisted all attempts by a most capable and committed group of mathematicians. However, we do know that static solutions with a net charge are not possible. Since an electron is charged and has a permanent magnetic moment the commentary above does seem pertinent. We are seeking a current configuration that is dynamic but non-radiating. In this respect, Theo has pointed us toward a very interesting and neglected avenue of inquiry.]

Subversive Epiphany

I have just gotten back from an extended business trip to the United States.

This was a lot of fun. I went to the SIAM Boston meeting on Computers in Science & Engineering. Later in the trip I went to PyCon2013 and PyData2013 to catch up with where things are at in the Pythonic universe.

It is probably close to fifteen years since I have been to a scientific conference. I very much enjoyed the developments in computational science since they involved many practical problems.

Regular readers of this blog will appreciate my love-hate relationship with mainstream physics. I tend to get very annoyed when I read physics journals, but I do enjoy the computer and applied mathematics literature.

However, I may yet come to some rapprochement with physics-at-large.

The epiphany for me was the sudden realization that most physicists today do not seem to believe in the Copenhagen interpretation but they do believe in point-like particles.

I don’t know exactly when that dawned on me, but it was somewhere in the middle of listening to some talk on numerical solutions of the Schroedinger equation in the semi-classical regime. None of the correspondents in that session seemed to know anything about my work in this area so I kept quiet.

I did not want to spoil the fun of searching for the elusive classical limit.

Reflecting upon what I already knew to be a grand folly I realized that physicists in general seem very sure that they know what rests within the Schroedinger equation. Look at it this way. They know three things:

1) the Schroedinger equation is right
2) the Classical world seems real enough and sensible
3) therefore, it must fall out of the mathematics somehow

All very sensible, but sadly untrue. The relevant mathematics does exist but is not what they are looking for!

Now to the epiphany. The physics community, in general, believe that we know particles are real. However, they are confused about what a wavefunction might be or mean since a wave is clearly not a particle!

What the community has yet to appreciate is that in a certain limit, that of the Classical Schroedinger Equation, the wavefunctions do bounce around, scatter and behave exactly like particles. However, that is a limit where the wave aspect of the physics is frozen out. You get to the idealised particle from the wave description.

However, that regime of behavior is very special and only ever approximate.

Let us now turn things around. Suppose there are no particles at all. Suppose that is merely an illusion and matter is pure wave. Even further, suppose that the phase of that wave is the gauge field (e.g. the photon). Then there is no light separate from its sources. They are part and parcel of the same phenomenon.

In that case, we have things perfectly upside down. We are using a micro-level mental picture of particles and then wondering why the classical level continuum physics is not easily recovered. Turn that around and employ a micro-level picture of pure waves. How then do you recover familiar billiard ball behavior?

That is easy, it is the decorrelated approximation of mean-field theory for which the Classical Schroedinger Equation is just the simplest example in a family (the next one up is the Hartree-Fock approximation).

What then is the reality? Well, in the alternative picture it is a matter wave in configuration space.

What about measurement theory, you ask? How come we only see fields in 3+1 dimensions not (3+1)N dimensions? How can you possibly shrink a beast in configuration space to a manageable quasi-classical field?

Easy. The very same rule invented by Max Born still works!

If you calculate the one-body density of the matter wave you get the classical level of experience. The mapping from (3+1)N dimensions to (3+1) is trivial and we already know the empirically supported rule.

Once we abandon thinking about particles, then there is no measurement theory at all. There is nothing separate, just a whole. There is no particle being measured by something, and no observer either. You can just junk the lot.

There is only one single wave and a continuum foundation to everything. Since the wave equation does admit eigenfunctions, that wave is quantized. Think of it as Universal Jelly with Natural Modes of Vibration.

Furthermore, the classical limit is merely an idealization. The wave never breaks up into independent particles in the sense that quantum field theory describes. Entanglement is ubiquitous and permanent (eat your qubit socks quantum computing zealots).

Wilder still, if you analyze the causality structure of such a theory it is not determinable.

While the underlying wave evolution may well be deterministic the one-body quantities are not sufficient as initial data. You could know them exactly and still not be able to predict what happens.

There is no need for God to Play Dice. You can have a non-local fully deterministic hidden wave theory. This you cannot do for particles, since local hidden variable theories are excluded!

The epiphany which struck me in Boston was the realization that hidden variable theories are alive, but only when constructed in the wave formulation.

If you insist that the wave describes a point-like particle then you cannot conceive of a hidden variable theory. This is because particles are local entities. Waves, on the other hand, are non-local entities. If you only think of particles you will miss this possibility.

The totality of the wave in configuration space is a non-local hidden variable.

I think it is possible to build a new theory now. However, this journey is not for the faint hearted. Quantum Field Theory simply cannot survive such a revision of physical interpretation in its present form.

QFT is necessarily a theory of particles not waves.

If we do get new physics it will be New Physics. Interpretation changes everything.

Getting the particle back as an idealized concept is easy. That is the Classical Schroedinger Equation.

Sionara QED I. Enter QED II, the sequel with added Gravitation.

The question is what to call this hidden layer of reality we are positing. Since the idea of junking QFT in its entirety is so utterly subversive I favor the term subverse.

To be really perverse, let us call the perceivable layer the PERVERSE and the imperceivable hidden layer the SUBVERSE. As Confucius himself might opine:

Contemplation of perverse perceivables guides us to subversive imperceivables.


SIAM meetings are a lot of fun.

I’ll be back.

The Long and Short of Physics

Physics is a very peculiar subject. When it changes, it does so radically in concept. However, the difference in results is barely perceptible.

Quantum mechanics is close to a hundred years old now. However, the classical theories of point and continuum mechanics are fine for most applications. How can the conceptual structure of a theory be so different and the results so similar?

For instance, under quantum theory we are supposed to believe in wave-particle duality. Bohr introduced the idea of complementarity to support his assertion that things are just too weird down there to understand. However, I wonder if this was just a ruse.

In 1926 there were two contending pathways forward: the continuum wave-mechanics of Schrödinger; and the discrete matrix mechanics and transition theory of Heisenberg. Of course, Dirac was able to demonstrate how they might be viewed as equivalent.

The philosophy of wave-particle duality has potentially obscured an important reality. While the two theories may be mathematically equivalent, due to the Dirac transformation theory, they are not physically equivalent as pictures. They suggest different ideas.

The wave-mechanics of Schrödinger leads the mind in different directions than does the matrix mechanics of Heisenberg. What we think of as natural in one picture can seem unnatural in the other picture.

For example, particle physics conceives of all particles as point-like and subject to creation and destruction via quantum transitions.

That picture is rather unnatural in the Schrödinger scheme of things. The wave equation has no inherent jumpiness to it at all. It is completely deterministic. Moreover, when one thinks this way the usual matrix elements for atomic transitions become rather obvious derivates of classical charge densities. It is a just a small variation on the classical theories of the dielectric due to H.A. Lorentz.

In the Heisenberg scheme, the transition element is just plucked from thin air. It has no obvious connection to the previous theory of classical dielectrics. Indeed, students are mercilessly beaten if they should dare to even suggest such a thought!

The lesson for me in all this can be stated rather briefly.

Suppose Schrödinger and de Broglie were right and we should treat matter waves as real stuff and not probability amplitudes?

Where would that take us? What would become of wave-particle duality? What is logical?

One thing seems very clear to me… There would be no particles.

In this picture, everything is a wave, albeit one which can be dynamically localized. The electron, as a wave, has an extent, and this is determined by the prevailing interactions. These can naturally be of two kinds:

1) mutual interactions

2) self-interactions

In the detailed development of quantum chemistry we can say a lot, in detail, about the first class of interactions. They are entangling, and so ensure that the waves in question must exist in configuration space. In short, entanglement is the normal state of affairs.

However, about the second kind of interaction we do not actually know very much. What we do know is that if you break wave-particle duality in favor of particles then you need to introduce a vacuum, along with particle creation and destruction. This is the basis for Quantum Field Theory and the reason why most particle physicists habitually refer to a wave-function of a single coordinate as a classical field.

Within this way of thinking, it is necessary to turn this classical field into an operator and then speak the language of particles.

Of course, with that goes another problem. One needs to figure out how to obtain finite answers for the electron self-energy and the related vacuum polarization effects. One has also bought into an infinite zero-point energy and a host of other technical difficulties.

On the other had, if we break wave-particle duality in favor of waves things look different. In that case, it is easy to get a finite self-energy term and there is no vacuum, nor any pesky zero-point energy. However, there are still meaningful problems. The main one is to get a stable bound structure for the electron since the static Coulomb field is repulsive. This would seem to involve the need to introduce an auxiliary field.

The purpose of this all too brief survey is to highlight one simple fact. Waves and particles are most definitely not equivalent as ideas for theory construction. When you attempt to build a theory based upon waves in configuration space then many things look different:

1) particles are no longer point-like and must therefore be dynamic entities

2) quantum fields are simply entangled wave-functions in configuration space

3) the reconciliation of mutual-interactions with self-interactions is incompletely understood

It is the last item which presents the major difficulty. Quantum Field Theory does not go over into this new regime in unmodified form. However, there is a new freedom gained to treat certain questions differently. For instance, pair processes and indeed all higher energy resonances, as seen in particle accelerator experiments, would need to be differently interpreted as resonant wave phenomena.

If I had to make a bet for the next hundred years in physics I would take these positions:

Waves: Long
Particles: Short

de Sitter Gravity: Long
String Theory: Short

Schrödinger: Long
Heisenberg: Short

Non-Linear Field Theories: Long
Linear Field Theories: Short

Einstein on Determinism: Long
Bohr on In-determinism: Short

It seems to me that these are all in fact the exact same bet.

When you change a theory, you need to make it self-consistent as well as empirically accurate. Strings go away because the reason for them is gone: the particle is really a wavy quasi-particle. The other bets relate to the physical consistency of real matter waves.

Of course, these are all quoted at very long odds right now.

However, they are my personal bets.

In a world obsessed with the very latest: everything old is new again.

The Classical Schrödinger Equation

There is a common view, perhaps, that discovery is straightforward and the people involved know what is going on. In the interest of encouraging independent thought, and experimentation, I would like to present some apposite opposite evidence.

In my view, the course of history rarely runs straight, and it is often more confusing to those directly involved. You may think that you are doing one thing, but soon events and knowledge overtake you and you realize that you are actually doing something different. That has been my stark experience several times in scientific research.

I would like to present that experience in summary because I hope it will encourage young researchers to stick at their investigations.

Do not be dissuaded from your course of action by uncertainty, peer resistance or non-comprehension – whether your own, or that of ofter people. We are all human and the wonder of the new is that it can be most perplexing. Nature is a Chinese Puzzle. She has her way to be — your task is to find that out. The true Scientist is compelled to mold his or her mind to Nature and not Public Superstition.

Follow Nature where the evidence leads you, even against the Conventional Wisdom.

So it was for me in early 1991. I had just finished my first publication dealing with my doctoral work on Quantum Inference. This was one of the early works on Quantum Information Theory. It showed that the Heisenberg Uncertainty Principle was more subtle than people first supposed. When you take account of the statistical Law of Large Numbers, then you discover that quantum states can be precisely measured. However, you would need an infinite supply of identically prepared systems to be able to do that. I was looking for applications of Quantum Inference and so thought of experimental tests of the quantum transformation theory.

I reasoned that if you could measure states precisely, which you can given enough of them, then you might make a sensitive test of the so-called Wigner theorem. I started out as an experimentalist in Chemical Physics, so I knew all about ultra-cold atomic and molecular beams. I had done my Bachelor of Science thesis in the Research School at ANU on Vibrational Pre-Dissociation Spectroscopy. Using an atomic-beam setup, to shoot cold atoms through a cavity, you should be able to detect non-linearity via a failure of Wigner’s theorem. This is the simple thought that got me started.

So I started looking at possible generalizations of the Schrödinger equation. I knew there had been some work and that the subject was of considerable mathematical difficulty. Hence I resolved to test my understanding with a simpler problem. Since quantum measurement involves the coupling of a formerly well-isolated quantum system to a classical-level measuring device I wondered if the classical limit might involve some non-linearity. Perhaps when a whole bunch of particles got together, there would be a stronger non-linearity. It made sense to me, given that the Schrödinger Cat Paradox only applies to linear theories.

To test this idea I framed a simple mathematical question:

Can one find a wave-equation which propagates waves so that their position and momentum expectation values follow classical trajectories?

Of course, I knew about the Ehrenfest Theorem, but I also knew it was approximate. The goal was to look for an exact equation. Obviously, that could not possibly be the ordinary linear Schrödinger equation. It took me a whole year of constant reflection. Then the answer suddenly struck me like a bolt from the blue:

Of course you could! Moreover, the answer would be unique.

It all followed from a beautifully simple piece of group theory due to Herman Weyl.

I could see the entire proof laid out before my eyes. It was, in essence, a very simple geometrical construction. It was so simple I could not believe it. However, it was also perplexing because I knew I had done something, but could not figure out what.

It can be that way in mathematics sometimes. People like to pretend they know what they are doing when they do mathematics. This is a fiction. If you know what you are doing it is certainly old mathematics. I knew I had new mathematics. Then things fell into place. I could prove the equation was necessarily non-linear, and was unique.

Aha! I said. This is the nonlinear representation theory of dynamical systems.

Reasoning about my own mathematics, I recalled a paper by Weinberg which I had glanced at when finishing my PhD thesis. It was called Testing Quantum Mechanics. In a frenzy, I searched for his paper and was immediately gratified that the equations looked different. However, I am a good enough mathematician to know that simple appearances can be deceptive. Sure enough, I figured out how to generalize his scheme to wave-functions and find an equivalent result.

The Classical Schrödinger Equation was for real, both ways!

I was elated for a simple reason. Many people might assume that it is discouraging to find prior work on a similar track. This is not so. I was elated because I had found something peculiar in my mathematics.

Operators were secondary to wave-functions.

Try as I might, I could not banish this feature. Once I found the result in the equivalent scheme of Weinberg I could see why. I had discovered something. Schrödinger mechanics was more general than Heisenberg mechanics.

Operators were an artifact.

Furthermore, the Classical Schrödinger Equation remained exact for any value of the Planck constant. This was a great surprise. I checked and re-checked everything but it always came out the same way. Greatly puzzled, I figured out why. It involved a Taylor series style expansion that got you back the ordinary Schrödinger Equation.

Excitedly, I prepared a summary of my result and sent it to Weinberg. He was kind and very generous with his time and I presented a seminar at the University of Texas, at Austin on Jan 30th 1992.

I was very nervous and prepared and rehearsed my talk extensively. At that time, I had very little seminar experience. When you first start out it is difficult to get much practice. I worried greatly that I would make a slip in my math somewhere. This is natural for young people. However, important work is never that way.

If I can offer any advice to young researchers it would be this:

Frame and motivate your question clearly.

So, I gave my talk. The result was silence. I was baffled and perturbed. Weinberg was kind enough to ask me later what I thought my talk was really about. I explained that the scheme of mathematics he had proposed for testing quantum mechanics actually contained a complete and exact copy of classical mechanics.

He looked at me intently and said: Oh!

In that moment, I knew I had a problem. As the saying goes:

You broke it, you fix it!

Later I completed the mathematical proof that Copenhagen Quantum Mechanics is Incomplete. I was grateful nobody noticed.

It is a bad feeling to break a perfectly good theory when you don’t know how to fix it.

However, where there is a will there is a way…

Lost Papers #1 and #2

In a previous post entitled The Lost Papers of Highgate Hill I described my project to restore some old and long lost works. I have just put up two of these on arXiv:

UM-P-91/45: The Classical Schroedinger Equation

UM-P-91/47: On Quantization, the Generalized Schrödinger Equation and Classical Mechanics

At this time, the Internet was young and arXiv was a dream. The preprint system was largely paper based. This means that the fossil record has a few gaps which will certainly challenge future historians of science.

In my case, the two published articles:

Classical Mechanics as an Example of Generalized Quantum Mechanics

General method for deforming quantum dynamics into classical dynamics while keeping hbar fixed

were based on more detailed work done at the University of Melbourne in 1991. I just uploaded these to arXiv after restoration from floppy disk.

If you look at the references in the first paper from PRD you will find two references to preprints: UM-P-91/45 and UM-P-91/47.

Each of these was sent out for peer review and received hostile referee reports. The first I withdrew for a rewrite pending a clearer physical interpretation and the second was rejected and later rewritten to appear as the second paper above, in PRA. However, the rewrite of UM-P-91/45 never happened.

I did come up with the physical interpretation in a PRA article entitled The Exclusion of Intrinsically Classical Domains and the Problem of Quasiclassical Emergence, but The Classical Schroedinger Equation never made it to print.

Over time, I had so many rejected works in this area that I actually assumed that UM-P-91/45 had been rejected also.

In fact, as I now know from my records, I withdrew it from the Annals of Physics after the initial hostile review. All of this, including the original LaTeX source, got lost and forgotten for twenty years as I moved house, left physics and misplaced my notes.

The notes and files got boxed up at my late parent’s house in Queensland and then followed them around as they moved all over. Only now is everything back in the one place after my parents passed away and I found the lost material in their garage.

It has taken a little time to sort out the old electronic storage media and obsolete file formats but I can now bring the material back to life.

I suspect there are many people out there with the same problem. When you get through the slog it is worth it.

In my case, I has long assumed that The Classical Schroedinger Equation was rejected from the Annals of Physics.

It wasn’t. I withdrew it, lost it and then forgot what I intended to do with the manuscript.

Life is funny sometimes.

Towards Agile Physics

Something struck me the other day.

Why is it so damn difficult to progress Physics?

Some would argue: That is because the ways of Nature are Subtle and Difficult!

With that I would agree, but I think there is more to it than that.

I think perhaps the Physical Sciences could learn something from Software Development.

In the software industry, new ideas are constantly floated and rigorously pursued to failure and exhaustion. It seems to me that Software Developers very well understand that the Nature of Computing is Subtle and Difficult. Further, they are over feeling bad when a project did not work. They simply junk the source code and move on. There is no shame there, just a thing learned and a new brick in the foundations of the discipline.

What then of Physics?

Things do not work this way in the Physical Sciences. There is Precious Reputation to be defended. There are Ivory Towers to be built, and Gilt Ramparts to defend. Mostly there is Hierarchy and there is Orthodoxy. There are Popes and Priests and Cardinals. There are fancy cloaks, titles and prizes. Perhaps even funny hats and Ermine Collars.

In short, there is a bunch of Ceremony and Puff-Penguin Nonsense there for the Crowd and not Progress. It is Ritual Blather to dissuade anyone from Perspicacious Inquiry.

You see, Physics is a difficult enough Science that we manage to Canonize each faltering step forward. The moment one pushes a toe forward we have to canonize that and halt all further progress for a century or two.

It is the most agonizingly hide-bound Science on the Planet.

Think about it.

Right now we have grown Men and Women running around declaring that Many-Worlds is Gospel and Everything Happens at Once. They deny the very existence of the individual events on which accumulated evidence the theory is founded.

This is a Divine Comedy worthy of the Middle Ages.

Yet there are few voices raised against such nonsense.

It is Canon.

It is Gospel.

It is Law.

However, in my world it is Pure Bunkum.

Let us move forth in Agile Fashion.

Fork the Physics Kernel. Try new things. See what works.

The Mind Projection Fallacy

What got me really fired up with my last post was some of the recent literature I have read announcing ever more elaborate philosophical interpretations of the wave-function. It is my opinion that these attempts (too many to cite here) are afflicted with what E.T. Jaynes called the Mind Projection Fallacy.

The essence of this idea is the common affliction to mistake models for reality. There is a popular and equivalent concept of distinguishing carefully between Map and Territory. For those who like big words, the difference between Ontology (what is real) and Epistemology (what is known). It is my contention that contemporary physics is in a deep bind precisely because it has ignored this distinction.

The Shut up and Calculate crowd are complacent about what they think they know, while the philosophers obsessively change the words without changing the content of the theory. Each of these extremes leads to trouble.

So let me now make a bold assertion. I now firmly believe that important sections of the new quantum theory have already been published. Yes, you heard me right. Accepted for publication and in print.

This statement ought to shock people.

If there is a new theory of quantum mechanics already out there, then: How come nobody has ever heard of it? Surely with all these hyper-smart people running around in physics they could not be blind to the existence of a new quantum theory right under their noses?

In the journals they know and read? Surely not!

Let me explain why I think this to be the case and exactly why nobody has noticed.

Firstly, if the assertion is correct then how could people not notice?

Answer: Jaynes was absolutely correct. Physicists as a community suffer from the Mind Projection Fallacy. They fervently believe that their theory is correct and therefore they have stopped noticing difficulties. Now we are in the terminal phase of this illness. People not only do not notice difficulties, they do not notice solutions. How can you notice a solution to a difficulty you do not believe exists? Of course, you won’t.

It all started with a few small issues being swept under the rug. However, as time goes on little problems build up. I will state only two of direct relevance to this post:

1) the ongoing perplexing problems of infinities in treatment of self-energy; and

2) the apparent existence of a non-zero cosmological constant.

Of course, there are answers to both conundrums. The first is dealt with in the Standard Model through the program of renormalization, while the second was catered for early on by Einstein’s “great blunder” – his modification of general relativity to achieve a stationary universe by including a cosmological constant.

What if these two pat answers hide something deeper? Perhaps renormalization is simply a stop-gap. Perhaps the presence of the cosmological constant signals something else completely like a self-energy of geometry?

Secondly, if the assertion is correct, then: Where is the evidence of a new theory?

The best example I know is the Self-Field Quantum Electrodynamics of the late Asim Barut and co-workers. There is a lot more than that, but we are simply establishing prime-facie evidence for the logical possibility of my thesis.

Thirdly, if there already is a new theory: Why don’t people pay attention to it?

Here is an old conundrum: Why does mankind oftentimes fail to take heed of warnings?

A host of very distinguished physicists: Einstein, de Broglie, Schrödinger, Bohm and Bell, foremost amongst many, have been critical of the Copenhagen Interpretation of quantum mechanics. They thought it was missing something: that is was incomplete.

Niels Bohr persuaded physicists that these problems either: 1) had deeply philosophical answers; OR 2) were not to be considered part of the practical program of calculation.

In other words, Bohr persuaded folks that it was okay to blur ontology and epistemology. When one did not work, simply slip effortlessly into the other mode until your audience is mentally exhausted and simply gives up.

Whenever something did not make sense you should either: change the words employed, or try not to think about calculating anything. With this device you could happily sweep any problem under the rug.

However, over time this ruse has caught up with physics. There are now so many different words used to say what a wavefunction means that nobody knows anymore. Further, the different tricks and devices have turned fundamental physics into one giant hairball. It is next to impossible to know what to say, or what to do.

I believe this situation is now self-evident. However, the very many professional academic physicists out there will doubtless dismiss this appraisal as betraying a lack of familiarity with the wondrous advances of recent years.

I say: Hooey to that!

Let me close with a simple observation.

Some years ago I conjectured something I called the hypothesis of restricted observables. For now, I will give the one sentence, just go with the flow, formulation:

The hypothesis of restricted observables posits that matter waves are real, but that reality, as we perceive it and experience it, constitutes only the reduced one-body fields projected from the total entangled multi-body matter wave in configuration space.

In short, Schrödinger was right, with one small proviso of restriction, and Bohr and the rest of the team were dead wrong. And no, Dorothy, this ain’t philosophy. You missed your stop in Kansas. This is physics with a big P. The surf is up and the waves are huge today.

As a corollary to that statement, I would go further and state:

If I am correct then 99.999% of the Academic Physics community are dead wrong and we need a new theory. Since they do not believe a new theory is needed, and supposing I am right, they will simply read about it one day. They are not even in the race.

Let us close with a simple general challenge to the Academic physics community.

Here is a “picture” of the Electron Localization Function (charge density) as used daily by Chemical Physicists. The image is form a recent Nanotechnology article on experimental imaging of the electron density. That is (essentially) a one-body density.

I started out in experimental Chemical Physics so I have awesome respect for these people. They solve truly hard problems and do not have Particle Physics Big Head Syndrome. In short, they are good old-fashioned scientists.

Here is my challenge to academic physicists:

According to the hypothesis of restricted observables the above picture is a description of laboratory reality. Yeah, you read that right. That is it. Done and dusted.

The challenge is to prove this statement incorrect through experiment.

In return, I accept my own challenge – one of theory.

Can I build a self-consistent field theory based on the hypothesis of restricted observables?

I believe yes. The physics community (I warrant) believes no. One challenge speaks to ontology (decisive experiment) the other to epistemology (constructive theory).

People who do not understand why this challenge is real are welcome to simply have their heads explode on the spot. There is no shame in saying:

I won’t play, this physics game is just too perverse for words!

First you say: particle; then wave; then both; then what, exactly?

As a final icing on this cake, let us reprise the de la Vega Confusión de Confusiones theme and an old advertisement from the South Sea Bubble days. In that time, there were many prospectus offerings to raise money for dodgy ventures.

I am a financial guy, so here is the prospectus for my part of this here challenge:

…a challenge for carrying out an undertaking of great advantage, but nobody to know if it has been done already

Let the fun begin!

The Lost Papers of Highgate Hill

Today I am happy to report that I have resurrected a good number of my “lost works” on Nonlinear Quantum Theory and its physical interpretation.

Twenty years ago, I was a post-doctoral researcher at University of Queensland. This was the early 1990s, and I was very active at that time in developing some new lines of research in nonlinear dynamics and quantum measurement.

The goal was to rebuild the present quantum theory from an alternative base. The different base was to use the Schroedinger interpretation of wavefunctions and not the Copenhagen interpretation.

That may seem like a small thing but it is a change with big consequences.

In particular, the Schroedinger interpretation is compatible with nonlinear field theories.

Nonlinear field theories occur naturally if you treat self-interaction effects directly rather than via a second-quantized quantum field theory.

There are many issues to overcome in rebuilding quantum theory along these lines. However, back in the early 1990’s I could see how to move forward.

Unfortunately, getting grant funding is a more difficult task than that of making progress in fundamental physics. The majority of physicists I interacted with at that time considered this line of research to be “too risky”, perhaps even “crazy”.

So it is with the sociology of science. It is pretty close to that of religion.

The majority of my “lost works” were submitted to various peer-reviewed scientific journals and rejected. The standard reason for rejection was: “We do not think this is important enough to warrant publication”. An all too common story with discovery.

However, one paper I actually withdrew from publication. This was the first paper I wrote on the Classical Schroedinger Equation. At the time, I felt this had become obsolete. However, even today, twenty one years later, it seems the results are unknown.

In that period, from 1991-1995, I was making progress so rapidly that my understanding of the field of nonlinear field theories changed extremely quickly. It was, so to say, opening out before me in real-time and there were few other active researchers. Indeed, the majority of physicists simply thought that to even try to build an alternative theory of quantum mechanics was a “crazy thing to do”.

So be it. People are quite rightly “crazy” until they succeed, I said. Then, in a single instant, everybody else is “crazy” for never having tried.

In such circumstances, the traditional model of refereed publications does not function very well. The referee cannot know about other works that are already in the pipe, but which cannot be cited since they are not published.

Pretty soon, things fall apart with the scientific process.

I had moved to a different place than most workers in physics and they knew next to nothing about why. I may has well have been speaking Greek.

It is a funny circumstance in hindsight, but was rather serious at the time.

The upshot was that I had to “find a new career”. I boxed up my notes and moved on.

I quit physics in 1996 and sought to make a living in financial markets. It was always a vague plan to return to my grand project one day. However, I needed some financial independence, liberty and freedom from chasing government funding. Aside: The sociology of herding is the biggest drag on human progress IMHO.

Today I am happy to relate two pieces of progess: 1) I am financially independent of the government; 2) I managed to restore my lost papers from obsolete storage formats.

I have found The Lost Papers of Highgate Hill.

Very Indiana Jones. Perhaps the Mayan prediction was right…

Many Worlds do end in 2012!