January 2013 – Annals of Post-Medieval Thought

Privateer Science

The ongoing multicore parallel computing revolution seems to have gone up a gear or two in the last few years. As I have written before, this is the Age of Amdahl’s Law: what matters is the serial bottleneck.

Work comes in two basic kinds: serial and parallel. The first cannot be sped up by adding new resources but the second can.

In my view, this is the least understood of all economic laws and the one with the greatest consequences for business and social innovation.

Hierarchical decision-making structures are replete with bottlenecks. Think of any bureaucracy. To get a form you stand in line. The first line is to get a number, so you can wait. You wait so you can get in line. The second line is to get the form. However, that form is the wrong form so you go back to stand in line. Then you get a number…

You get the picture. In any society that does this habitually very little gets done.

Of course, this problem is not limited to Government. Business suffers greatly from this problem. So too does Academic Research.

Call me a cynic, but my experience of academic research in physics taught me that before you were allowed to discover something you had to get in line.

The first line was about doing stuff that made you an Academic. Usually that process was so long and drawn out that you wound up forgetting what the line was for.

It turns out you could just go right ahead and discover a whole bunch of things right of the bat. However, then you were Not in Line. Get back in line!

This is a perfect example of a serial bottleneck.

However, once you understand the power of Amdhal’s law then things look different.

Here are the game changers:

1) all physics is an algorithm
2) algorithms can be turned into software
3) software can be protected and is valuable

Finally, in this new world of Google Compute Engine you can spin up a cluster for rent. Hence the capital cost barriers to radical innovation in algorithms is low.

So what are the real barriers?

The social bottleneck of Academic “publish-or-perish” screw-you syndrome pretty much ensures nobody with any good idea will get anywhere.

Chalk that up as a good thing.

The negative network economy of Academic Rivalry pretty much stalls all real advances for at least one generation until all opponents have finally died out.

The Academic flotilla may have all the Pomp and Pretty Pennants of the Spanish Main, but they are: Dead in the Water.

So how does one exploit this?

I think the answer is Privateer Science.

Academics are not crash-hot at business.

For decades, they enriched publishers by madly competing to publish in journals that nobody read. Now they are madly competing to pay top dollar to be published in journals that the public can read for free.

Okay, so first barrier down.

You can read what they did for free while they fight each-other for scraps.

What does a modern Privateer do in this circumstance?

Focus on the profitable bottleneck.

In quantum mechanics, that is many-body theory with strongly correlated fields.

I know, I know, every wanna-be genius in Physics will duke it out on quantum computers.

Okay, cool… but where is the market?

The market for strongly correlated fields is Chemistry, Superconducting Electronics, Photovoltaic Devices and Nano-structures among other things.

These are Big markets, here today, and served by existing Multi-Physics software companies with real product.

And it is all about parallel algorithms

That is what any budding Tesla or Edison of today should focus upon. The Big Score.

Work out a better QED that is simpler for strong self-interactions and you are done.

Better yet, if you find one – keep it a secret. Exit the rat-race of Publish-and-Perish.

Raise the Jolly Roger and load those cannons with grape shot…

All is Ripe for Plunder on the High Seas.

A Blinding of Naked Emperors

There seems to be a global special on Naked Emperors right now. They are everywhere, like a New Year Clearance Sale.

Get your Naked Emperors now… by the brace, by the pound, by the dozen.

Personally, I blame Advertising. Why not? It is ubiquitous and rots the mind.

The plethora of positive messages about product has corrupted the normal channels of curmudgeonly defenestration.

Truth Talking is Trash Talking in a Sound-Bite World of Blinding Gloss.

No longer can we just throw our Naked Emperors out the window and wait for the satisfying splat as they hit the dung heap.

This leads me to ponder an obvious deficiency of language.

Where is the Collective Noun for Naked Emperors?

We have a skein of geese, a coffle of asses, a parliament of baboons and a bellowing of bullfinches. With this many Naked Emperors one must have the proper word…

Enter an obvious Thought Experiment.

Suppose we were to place a number of Naked Emperors in a darkened room behind one-way glass. What would happen?

A whole bunch of Naked Cavorting in the Dark, I would guess.

That whole pirouetting Naked Emperor thing!

There is an old Danish proverb which speaks to this situation:

Necessity teaches the naked woman to spin.

Naked Emperors are no different, only the rate of spin is much higher. Indeed, in a sufficiently dense medium – such as Physical Review Letters – we can often witness Naked Emperors spinning much faster than the phase velocity of light in such a medium. The result is the brilliant blue light known as Eminence Radiation.

This is positively blinding to the General Public. Hence our collective noun:

A Blinding of Naked Emperors.

Your most Scintillating Spinning Eminence… I am Blind before your Nakedness.

All Popes and No God

I grew up in a religious family but am not myself a believer. This upbringing shaped my attitudes to life, belief, certainty and respect for contrary views. When members of your family harbor beliefs that you do not share then it is necessary to come to some accommodation. You cannot simply discount matters of the human spirit.

In my view, that which moves the human spirit is ultimately some form of faith. It may be faith in an External God, for the Spiritually Minded. It may be faith in the Scientific Method, for those who believe in a Rational World. Or, something else entirely.

In my case, I developed Faith in Creativity. To me the God was innate in the ability of us all to create. We may not be able to create in the Biblical sense, but we sure can be creative in our lives, thoughts, actions and artifacts. This idea has proven to be a meaningful motor for my spirit. It gets me out of bed each morning.

However, many other Scientists seem contemptuous towards matters of faith. There grew up in the 1990s a form of Scientific Triumphalism which irritates me immensely. On the one hand, you had scientists lambasting faith-based religion as lacking any sound basis. However, they then trumpeted ideas such as The Theory of Everything, Many Worlds, Parallel Universes and a host of other bits and pieces of histrionic nonsense. None of these had evidence to support them – pure mind fluff.

All of this happened in the context of a quantum theory which has never once seemed self-consistent. It makes no direct statement of how the objects of the theory relate to observable reality. It is based on a panoply of rules without coherent foundation and a set of recipes for throwing away any results that should prove embarrassing.

Infinite predictions? Don’t worry, just settle on a cut-off and re-normalize away.

Infinite zero-point energy? Don’t worry, just ignore it.

Paradoxes of measurement and self-reference? Don’t worry, just kick the cat.

It is not bad to be pragmatic. It is sensible to press on and see what can be done in face of these difficulties. However, there also seems to be a stark double standard.

On the one hand, Scientists lambast faith as an arcane and primitive deficiency of the intellect. They trumpet rationalism in place of this and then laud Science as its most perfect exemplar. On the other hand, they hound and pillory those within their ranks who would fix the cracked foundations of their own glorious theories.

You simply cannot have it both ways.

If Scientists want the Public to be humble before their own God of Rationalism then they must be humble before the Flaws in Their Own Theories.

However, this is not the spirit of our age…

Much as a messianic leader of old, there is an attitude among Scientists, especially Physicists, that questions are unwelcome for the answers have been given.

I do believe in Rationalism, but I also believe in what the philosopher-investor George Soros described as Rational Fallibility. No theory is perfect. There is no Final Theory.

When you think you know it all, a surprise is surely in store.

In this connection, Weinberg once said that:

Science has Heroes but no Prophets.

Perhaps that is true, but who then will show the way to new theories? Is not prophecy the art of foreseeing that which the multitude will not see?

No rational mind would attribute divine qualities to a Hedge Fund manager. However, what equipped George Soros with the gift of financial prophecy was an understanding of Human Fallibility, both his own, and that of other market players.

The Prophet sees what could happen which the multitude do not want to happen.

Hence I am led to a small prophecy about Physics.

I think physics has become very authoritarian. What is done and what is praised no longer seems to bear a strong relationship with scientific method. Instead, it seems to have become a game to garner social favor. Ideas are presented and then canonized seemingly without regard to their level of empirical support.

Once we develop a social caste of High Priests then the destination is clear:

Physics will have Popes but no God.

Once a community stops asking the searching questions, the rational basis of Science is lost. Not for ever, but until the boil is lanced.

Until that day, there is next to no difference between Science and Religion.

It will be High Priests versus the Heathen Mongols.

Age and Invention

There is a common view that invention is the province of the young. For an interesting discussion of this see Age and Great Invention, where some contrary evidence is also presented. Reflecting on observation of my peers and my own experience:

1) I believe that the dissemination of any invention is much harder today

2) However, the capacity for meaningful social innovation grows with age

The reasons, I think, are largely social.

The big change over the last fifty years or so has been the huge expansion of global research activity. This is both a good thing and a bad thing. Of course, it is good to have more research. However, it is also more difficult for ideas to become widely adopted. That may seem paradoxical, but lies at the heart of my observations.

If you reflect on the scale of research today, in terms of the sheer number of persons involved, then it is clear where the social friction comes from. There is a tendency in social systems towards a Winner–Take–All mentality. In the field of ideas, that is very problematic. Crafting a Win–Win Outcome is hard — perhaps impossible.

Perhaps the best analogy I can think of is the network law due to Bob Metcalfe:

The economic value of any network of N members goes as the square of N.

Allow me to introduce Jones’s First Law of Academic Fractiousness:

The bitterness and intensity of Academic disputes and in–fighting goes as the square of the number of Academics involved.

This does not make any individual Academic a bad person. They are not. It is just that the Laws of Economics predict an Almighty Bun-Fight at the End of Time.

Folks better hope that nobody ever finds a Final Theory or the resulting priority dispute may spin the Earth clean off its axis! It would be Academic Ragnarök.

Returning to points 1) and 2) my summation is simple:

Innovate while you are young then spend the rest of your life explaining what on Earth it was you did. Embrace the second part since it will make you a better person.

That seems to be the general rule and pattern of life nowadays.

Perhaps this seems cynical, but then part two — learning how to introduce something useful to society — is good for the soul. It is all about framing: “What’s in it for me?” on a genuine basis of mutual engagement. This happens to be great training for starting any business and so is very useful indeed.

To paraphrase Jessica Rabbit: Academics are not bad, they just behave that way.

Stringy Ectoplasm

It is a commonplace for folks to imagine that lasting innovation in physics is down to raw intelligence. I think it may be more complicated than that. Physics is most definitely not mathematics. Pure intelligence is an unqualified asset in mathematics, but, in physics this seems doubtful. After all:

It is about Nature, Not about You.

A good example is Richard P. Feynman. Undoubtedly a great genius, Feynman was not your regular smart guy. A most appealing attribute of this “half-genius half-buffoon” was his proud reflection of a moderate IQ, 127 he claimed. However, you never know with Feynman. Ever the showman, he also aced the USA Putnam math exams.

This leads me to reflect on the high profile of String Theorists. I find this puzzling given the tenuous links to observational data. String theory is everywhere in the Mind and appears nowhere in Nature. How did these folks hoodwink everybody?

It is hard to puncture this eternally inflating bubble of irrelevance. String theorists seem impervious to criticism. If you were feeling cruel you might observe:

That’s some Pretty Pulchritudinous Phenomenology you’re Packing my P-Braned Pal!

However, string theorists do not get out much. They probably don’t spend much time in bars and would doubtless consider this an insult.

In short, I have to admire them for getting away with it thus far.

There has grown up around this community a popular view that they are the smartest of the smart and therefore must be on the right track. About this I have mixed feelings. I have zero doubt that Edward Witten is an order of magnitude more intelligent than me. However, I have grave doubts that string theory is on the right track in physics.

The reason has to do with something that really was exceptional about Feynman. He knew in his bones when he was fooling himself and backed away.

It is about Nature, Not about You.

Clever may be a pleasant detour, but Right is the correct destination.

Sometimes being right means appearing to be dumb.

Let me illustrate. String theory is popular because the notion of an “extended object” seems like the “right” way to move beyond point particles. The physical reasons for wanting to do that have to do with infinities in quantum field theory.

These are especially acute for quantum gravity which happens not to fit the pattern of other theories. String theory is a neat piece of mathematics which intuitively seemed to fit the idea of how you might propose other degrees of freedom and then integrate them out to make the theory finite. People who are excellent mathematicians warm to this idea. They are drawn to it like moths to a flame: it seems “obviously right”.

However, this is where it pays to be less smart but more in tune with Nature. It may shock folks, but we actually have little evidence that particles are point-like.

The best example I know of is Carver Mead with his Collective Electrodynamics. Mead develops a cogent argument that we have misunderstood electrodynamics. He points out that coherent quantum behavior represents the real lesson of “how things are”.

This is important, because the popular view is that Quantum Mechanics is intrinsically indeterminate. Carver Mead provides a straightforward path of reasoning to a different conclusion. He argues, successfully I believe, that the indeterminacy is a function of a particular observational regime. Instead of supposing that we know how things are at the subatomic level, we should recognize how things are at our level.

Carver Mead may well lay claim to be operating in the tradition of Newton: Hypotheses non fingo. I make no hypothesis. I invoke what is necessary to explain the data and no more. I do not presume to know why it is that way. I make no hypothesis. The task of the Scientist is to mold his or her mind to the Ways of Nature.

Another fine example of this trend is John G. Cramer and his Transactional Quantum Mechanics. This can be best understood as a development of the Wheeler-Feynman Source–Absorber theory. The idea is to consider the “photon” and all other “exchange quanta” as resonance phenomena between the source and the absorber. The result is a transactional view of exchange quanta. This provides a different explanation of photons, and indeed all other such quanta as artifacts of resonant interaction.

I find these ideas interesting because they conform to a long tradition of conceptual physics. This approach involves deep thinking about physical models. It places at a lower rung the sophistication of the mathematics. A wonderful historical example is the dielectric model of H.A. Lorentz in his landmark: Theory of Electrons. This may have been classical, but it is still useful.

A contemporary example is Tony Seigman’s book Lasers which manages to develop most every aspect of laser operation using only classical physics.

People schooled in the finer details of Quantum Electrodynamics might find such works quaint. However, the goal of physics is to understand not to impress.

In contrast, String theory seems to come at things backwards. It proposes that more sophisticated mathematics is necessary to understand the structure of matter.

When this fails, the String Theorists claim: If theory does not work, let us invent new Hypothetical Universes where it might.

I am not a believer.

The Classical Schrödinger Equation

There is a common view, perhaps, that discovery is straightforward and the people involved know what is going on. In the interest of encouraging independent thought, and experimentation, I would like to present some apposite opposite evidence.

In my view, the course of history rarely runs straight, and it is often more confusing to those directly involved. You may think that you are doing one thing, but soon events and knowledge overtake you and you realize that you are actually doing something different. That has been my stark experience several times in scientific research.

I would like to present that experience in summary because I hope it will encourage young researchers to stick at their investigations.

Do not be dissuaded from your course of action by uncertainty, peer resistance or non-comprehension – whether your own, or that of ofter people. We are all human and the wonder of the new is that it can be most perplexing. Nature is a Chinese Puzzle. She has her way to be — your task is to find that out. The true Scientist is compelled to mold his or her mind to Nature and not Public Superstition.

Follow Nature where the evidence leads you, even against the Conventional Wisdom.

So it was for me in early 1991. I had just finished my first publication dealing with my doctoral work on Quantum Inference. This was one of the early works on Quantum Information Theory. It showed that the Heisenberg Uncertainty Principle was more subtle than people first supposed. When you take account of the statistical Law of Large Numbers, then you discover that quantum states can be precisely measured. However, you would need an infinite supply of identically prepared systems to be able to do that. I was looking for applications of Quantum Inference and so thought of experimental tests of the quantum transformation theory.

I reasoned that if you could measure states precisely, which you can given enough of them, then you might make a sensitive test of the so-called Wigner theorem. I started out as an experimentalist in Chemical Physics, so I knew all about ultra-cold atomic and molecular beams. I had done my Bachelor of Science thesis in the Research School at ANU on Vibrational Pre-Dissociation Spectroscopy. Using an atomic-beam setup, to shoot cold atoms through a cavity, you should be able to detect non-linearity via a failure of Wigner’s theorem. This is the simple thought that got me started.

So I started looking at possible generalizations of the Schrödinger equation. I knew there had been some work and that the subject was of considerable mathematical difficulty. Hence I resolved to test my understanding with a simpler problem. Since quantum measurement involves the coupling of a formerly well-isolated quantum system to a classical-level measuring device I wondered if the classical limit might involve some non-linearity. Perhaps when a whole bunch of particles got together, there would be a stronger non-linearity. It made sense to me, given that the Schrödinger Cat Paradox only applies to linear theories.

To test this idea I framed a simple mathematical question:

Can one find a wave-equation which propagates waves so that their position and momentum expectation values follow classical trajectories?

Of course, I knew about the Ehrenfest Theorem, but I also knew it was approximate. The goal was to look for an exact equation. Obviously, that could not possibly be the ordinary linear Schrödinger equation. It took me a whole year of constant reflection. Then the answer suddenly struck me like a bolt from the blue:

Of course you could! Moreover, the answer would be unique.

It all followed from a beautifully simple piece of group theory due to Herman Weyl.

I could see the entire proof laid out before my eyes. It was, in essence, a very simple geometrical construction. It was so simple I could not believe it. However, it was also perplexing because I knew I had done something, but could not figure out what.

It can be that way in mathematics sometimes. People like to pretend they know what they are doing when they do mathematics. This is a fiction. If you know what you are doing it is certainly old mathematics. I knew I had new mathematics. Then things fell into place. I could prove the equation was necessarily non-linear, and was unique.

Aha! I said. This is the nonlinear representation theory of dynamical systems.

Reasoning about my own mathematics, I recalled a paper by Weinberg which I had glanced at when finishing my PhD thesis. It was called Testing Quantum Mechanics. In a frenzy, I searched for his paper and was immediately gratified that the equations looked different. However, I am a good enough mathematician to know that simple appearances can be deceptive. Sure enough, I figured out how to generalize his scheme to wave-functions and find an equivalent result.

The Classical Schrödinger Equation was for real, both ways!

I was elated for a simple reason. Many people might assume that it is discouraging to find prior work on a similar track. This is not so. I was elated because I had found something peculiar in my mathematics.

Operators were secondary to wave-functions.

Try as I might, I could not banish this feature. Once I found the result in the equivalent scheme of Weinberg I could see why. I had discovered something. Schrödinger mechanics was more general than Heisenberg mechanics.

Operators were an artifact.

Furthermore, the Classical Schrödinger Equation remained exact for any value of the Planck constant. This was a great surprise. I checked and re-checked everything but it always came out the same way. Greatly puzzled, I figured out why. It involved a Taylor series style expansion that got you back the ordinary Schrödinger Equation.

Excitedly, I prepared a summary of my result and sent it to Weinberg. He was kind and very generous with his time and I presented a seminar at the University of Texas, at Austin on Jan 30th 1992.

I was very nervous and prepared and rehearsed my talk extensively. At that time, I had very little seminar experience. When you first start out it is difficult to get much practice. I worried greatly that I would make a slip in my math somewhere. This is natural for young people. However, important work is never that way.

If I can offer any advice to young researchers it would be this:

Frame and motivate your question clearly.

So, I gave my talk. The result was silence. I was baffled and perturbed. Weinberg was kind enough to ask me later what I thought my talk was really about. I explained that the scheme of mathematics he had proposed for testing quantum mechanics actually contained a complete and exact copy of classical mechanics.

He looked at me intently and said: Oh!

In that moment, I knew I had a problem. As the saying goes:

You broke it, you fix it!

Later I completed the mathematical proof that Copenhagen Quantum Mechanics is Incomplete. I was grateful nobody noticed.

It is a bad feeling to break a perfectly good theory when you don’t know how to fix it.

However, where there is a will there is a way…

Incoming…

The older I get the more I realize I made the right decision to branch out. Academic Physics was too much like Church for my taste. All Ceremony and Fake Wine.

Of course, I do appreciate and enjoy study of the Physical Sciences. However, I am far more moved by what happens in the Real World: hence my employment in Defense and Finance. These pursuits are intellectual but harbor a combative element. I do not by nature pick fights, but I cannot stand when the effort made elicits no reaction.

At least in military or financial affairs you can be certain of a reaction when you pull the trigger. That beats the sometimes deafening silence of politely disengaged academic discourse. Each to their own. We are made differently after all.

I realize now I was always cut out to be a Heathen Mongol and never a High Priest.

Quiet monastic omphaloskepsis was never for me.

Rather than lead the quiet life ensconced in The Citadel I would rather: Sack It!

300px-Bagdad1258
Hulagu’s army conducting a siege on Baghdad walls.

Lock up your Reliquaries, Pilgrims!

I am a self-confessed Heathen Mongol with a Ghenghis Kahn complex. Give me a band of Mongols and a Moonless Night. We could ride down from the Steppes, Sack the Citadel and make off with the Horses, the Gold and the Women, in that order.

It is when the World moves, that I feel Moved. When nothing happens at all, life gets very dull. What moves the mind moves the world – that is my motto.

No wonder I never fitted into the traditional academic world with that attitude!

Yep, a Hedge Fund and a private Helicopter Gun-Ship for me…

The best kind of scientific paper is like a sea-skimming Exocet missile.

It should come in low and fast, without warning, and take out the Flagship…BLAM!

Nobody, but nobody, should ever see it coming.

Then you simply Crash-Dive and lay low for a while.

Incoming…

Lost Papers #1 and #2

In a previous post entitled The Lost Papers of Highgate Hill I described my project to restore some old and long lost works. I have just put up two of these on arXiv:

UM-P-91/45: The Classical Schroedinger Equation

UM-P-91/47: On Quantization, the Generalized Schrödinger Equation and Classical Mechanics

At this time, the Internet was young and arXiv was a dream. The preprint system was largely paper based. This means that the fossil record has a few gaps which will certainly challenge future historians of science.

In my case, the two published articles:

Classical Mechanics as an Example of Generalized Quantum Mechanics

General method for deforming quantum dynamics into classical dynamics while keeping hbar fixed

were based on more detailed work done at the University of Melbourne in 1991. I just uploaded these to arXiv after restoration from floppy disk.

If you look at the references in the first paper from PRD you will find two references to preprints: UM-P-91/45 and UM-P-91/47.

Each of these was sent out for peer review and received hostile referee reports. The first I withdrew for a rewrite pending a clearer physical interpretation and the second was rejected and later rewritten to appear as the second paper above, in PRA. However, the rewrite of UM-P-91/45 never happened.

I did come up with the physical interpretation in a PRA article entitled The Exclusion of Intrinsically Classical Domains and the Problem of Quasiclassical Emergence, but The Classical Schroedinger Equation never made it to print.

Over time, I had so many rejected works in this area that I actually assumed that UM-P-91/45 had been rejected also.

In fact, as I now know from my records, I withdrew it from the Annals of Physics after the initial hostile review. All of this, including the original LaTeX source, got lost and forgotten for twenty years as I moved house, left physics and misplaced my notes.

The notes and files got boxed up at my late parent’s house in Queensland and then followed them around as they moved all over. Only now is everything back in the one place after my parents passed away and I found the lost material in their garage.

It has taken a little time to sort out the old electronic storage media and obsolete file formats but I can now bring the material back to life.

I suspect there are many people out there with the same problem. When you get through the slog it is worth it.

In my case, I has long assumed that The Classical Schroedinger Equation was rejected from the Annals of Physics.

It wasn’t. I withdrew it, lost it and then forgot what I intended to do with the manuscript.

Life is funny sometimes.