1 Space-Time is Not the Perfect Arena for Quantum Theory
Space-time is absolute, in conventional classical physics. Its geometry is determined dynamically by the distribution of classical objects in the universe. However, the underlying space-time manifold is an absolute given, providing the arena in which material bodies and fields exist. The same classical space-time arena is carried over to quantum mechanics and to quantum field theory, and it works beautifully there too. Well, almost! But not quite. In this essay, we propose the thesis that the troubles of quantum theory arise because of the illegal carry-over of this classical arena. By troubles we mean the quantum measurement problem, the spooky action at a distance and the peculiar nature of quantum non-locality, the so-called problem of time in quantum theory, the extreme dependence of the theory on its classical limit, and the physical meaning of the wave function . We elaborate on these in the next section. Then, in Section 3, we propose that the correct arena for quantum theory is a non-commutative space-time: here there are no troubles. Classical space-time emerges as an approximation, as a consequence of the dynamical collapse of the wave function. When quantum theory is viewed on the non-commutative space-time, there is no notion of distance, no spooky action at a distance, and no mysterious probabilities.
2 Quantum Theory is the Truth But Not the Whole Truth
Let us accept as given the universe with its terrestrial physics laboratories, planets, stars, galaxies, and the classical absolute background of space and time. When we treat the microscopic world according to the laws of quantum mechanics, against this given background, we have a theory that agrees marvellously with every experiment that has been done so far to test it. Even in the macroscopic world, the theory exhibits many collective phenomena such as superconductivity, superfluidity, and Bose–Einstein condensation. However, as is well known, despite its great success, there are some features of the theory that many physicists find tentative, and/or unsatisfactory. We recall these briefly in the next few paragraphs.
2.1 The Theory Has Not Yet Been Experimentally Tested in All Parts of the Parameter Space
The principle of linear superposition is a central tenet of quantum theory, and is famously shown to hold in the double-slit experiment with photons and with electrons. It has been experimentally shown to hold also for heavier particles such as neutrons as well as atoms and molecules. But what about heavier composite systems such as macromolecules or even larger objects, say nano-particles, or silicon clusters? Technology poses great challenge to matter-wave interferometry experiments, or any other kind of experiments that aims to test the superposition of position states for mesoscopic objects. The largest macromolecule for which superposition in an interference experiment has been shown to hold has a mass of about 10,000 atomic mass units . On the other hand, in the macroscopic classical world, position superposition does not seem to hold: a chair is never seen here and there at the same time. The smallest objects for which such classical behaviour is known to hold for sure have a mass of about a microgram, which is about 1018 atomic mass units. Now, it could well be that this absence of macroscopic superposition is caused by environmental decoherence, in a many-worlds scenario. Or it could be explained by Bohmian mechanics or some other interpretation/reformulation of quantum theory. But how can we be sure that this is not a result of new physics: the possibility that linear superposition is not an exact but an approximate principle? With superposition lasting for enormously long times for small particles such as electrons and atoms, but becoming progressively short-lived for heavier objects, until it becomes extremely short-lived for macroscopic objects such as chairs and tables. The only way to settle whether or not there is new physics is to do experiments in a decoherence-free environment. These experiments are extremely challenging, and are being pursued in many laboratories that exploit frontline technology . Between 10,000 and 1018 a.m.u., there is an experimentally untested desert spanning 14 orders of magnitude, where there is a question mark on the validity of the superposition principle. (Note that collective internal states such as in superconductors and Bose–Einstein condensates (BECs) do not qualify as experimental tests of superposition in the present context, because these ‘one-particle-type’ states are not the Schrödinger cat states one is looking to create.)
2.2 The Quantum Measurement Problem
When a quantum system which is a superposition of eigenstates of an observable interacts with a classical measuring apparatus, why does it end up in being just one of those eigenstates? What causes this breakdown of superposition and the so-called collapse of the wave function, in apparent violation of the Schrödinger equation? And what systems qualify to be called a classical measuring apparatus? Quantitatively speaking, how large must a quantum system be before it can be called classical? And why is the collapse random, even though the Schrödinger equation is deterministic? Where do probabilities come from, in a deterministic theory, where initial conditions are perfectly well defined? Standard quantum theory has no answers. Do these unanswered questions call for new physics? Maybe! .
2.3 The Peculiar Nature of Quantum Non-Locality
Consider a pair of quantum particles A and B, in an entangled state, which start off close by and fly off in opposite directions to get very, very far from each other. Perhaps so far that they travel for billions of light years while remaining in an entangled state. If a measurement is made, say on A, causing its state to collapse, it instantly and acausally influences the state of B as if A and B were together a physical structure like a rigid body. This is the non-local quantum correlation, the infamous spooky action at a distance . And it has been confirmed by experiments , , . The measurement on A influences the state of B, even though B is outside the light-cone of A (space-like separation). Of course, the measurement on A cannot be used to transfer information to B faster than light (no superluminal signalling) . This is the microcausality condition: quantum commutators vanish outside the light-cone. Is non-locality a problem? For many physicists, it is not a problem because special relativity is not violated. This is how quantum physics is, they say. For other physicists, which includes the present author, quantum non-locality is one of the greatest mysteries of physics. It challenges cherished and long-held notions of locality and separability. How does B ‘know’ that a measurement has been made on A? Undoubtedly, the collapse aspect of quantum theory is oblivious to physical distance .
2.4 The Problem of Time in Quantum Theory
The time that appears in quantum theory is part of a classical space-time, whose geometry is determined by macroscopic classical bodies, according to the laws of general relativity. But these classical bodies are a limiting case of quantum theory. In their absence (for instance, in the very early universe, soon after the Big Bang), there would be no classical space-time geometry. If there are only quantum matter fields in the entire universe, the gravitational field they produce would also possess quantum fluctuations. These fluctuations in turn destroy the underlying space-time manifold, rendering it physically meaningless. This is the essence of the Einstein hole argument: a classical metric field must overlie the space-time manifold, in order to allow a physical interpretation of the point structure of the manifold . How then does one describe quantum dynamics of quantum objects if the universe is not dominated by macroscopic objects, and does not possess a classical space-time manifold? As a matter of principle, one ought to be able to describe the dynamics under such circumstances. It therefore follows that there must exist an equivalent reformulation of quantum theory that does not depend on classical time .
2.5 The Extreme Dependence of the Theory on Its Classical Limit
Classical mechanics is a limiting case of quantum mechanics. Yet, in order to arrive at the canonical quantum theory, one must first know the classical theory. One must know the classical configuration and momentum degrees of freedom, its Lagrangian, its Hamiltonian, and its Poisson brackets. Then, employing an ad hoc (albeit highly successful) recipe, the Poisson brackets are replaced by the canonical commutation relations of quantum theory. This dependence of the quantum theory on its classical limit is unsatisfactory. It is as if we have a set of quantum recipes, rather than a theory constructed from first principles. There ought to exist a construction/derivation of quantum theory that does not depend on its own classical limit.
2.6 The Physical Meaning of the Wave Function
In order to illustrate the difficulty, let us consider the double-slit experiment with electrons. An electron, supposedly behaving like a localised particle before it leaves the electron gun, spreads like a wave after leaving the gun. The wave passes through both the slits, the two secondary wavelets interfere with each other, until the electron reaches the screen and randomly collapses to a point on the screen. But wait a moment! What is this wave a wave of? In order to describe the interference pattern, one must consider the ‘wave’ to be the wave function (i.e. the so-called probability amplitude) and superpose the complex wave functions representing the two wavelets. But how could the complex wave function live on real physical space, and pass through the slits, and behave as if it were a wave? The wave function in fact lives in Hilbert space, not in physical space. To save the phenomenon, the wave is sometimes said to be a probability wave. But, of course, we know that probabilities do not interfere, probability amplitudes do. And if it is a probability wave, what physical meaning can we ascribe to the wave function? This bizarre state of affairs led Feynman to famously assert that nobody understands quantum mechanics. We have an extraordinarily successful theory, but we have a rather poor idea what actually is going on out there.
To understand what is going on, we take the holistic view that all these problems described above must have a common resolution. Quantum theory is not the whole truth. It is an approximation to a deeper theory. A resolution to its problems does exist, and when we push the proposed resolution to its logical conclusion, we are confronted with a new conception of space and time. Space and time, as we know them, are emergent, and result from the dynamical collapse of the wave function. Fundamentally, space-time is non-commutative, and its geometry is a non-commutative geometry.
3 The Whole Truth is that Quantum Theory is Approximate
3.1 A Possible Resolution of the Quantum Measurement Problem
There have, of course, been various proposed solutions for the measurement problem, which result from a re-interpretation or mathematical reformulation of quantum theory. These include the many-worlds interpretation, Bohmian mechanics, and consistent histories, among others. Since they make the same experimental predictions as quantum theory, it really becomes a matter of taste as to which one is preferred over the other .
On the other hand, consider the possibility that quantum theory actually breaks down in the domain of measurement, and is not the right theory to describe the interaction of a quantum system with a measuring apparatus. Maybe there is a more general universal dynamics, which includes a stochastic element responsible for the randomness, and which reduces to quantum mechanics for microscopic systems and to classical mechanics for macroscopic systems. In the micro-limit, the stochastic aspect is extremely negligible, and in the macro-limit it is so significant that the dynamics looks like classical dynamics.
Any attempt to generalise the Schrödinger equation must obey severe constraints if it has to solve the measurement problem, without contradicting the outstanding experimental success of the Schrödinger equation. The new dynamics must be non-linear so that it causes breakdown of superposition during a measurement. The non-linear aspect must be random in nature, so that outcomes of measurement are random, and also because deterministic non-linearities lead to superluminal signalling, which we wish to avoid. The new dynamics has to be non-unitary, in order that the system is driven to one outcome; yet the non-unitary evolution must be norm-preserving, so that probability is conserved. The universal dynamics must also possess an amplification mechanism so that the non-linearity is negligible for small masses and becomes progressively more significant for larger masses.
Remarkably enough, such a dynamics does exist, and was proposed many years ago by Ghirardi, Rimini, Weber, and Pearle , , , , . It is known as continuous spontaneous localisation (CSL). It proposes that the wave function of every elementary particle (let us assume it to be the nucleon) undergoes a spontaneous collapse at the rate λ, and gets localised in physical space to a region of size rC. Between every two collapses, the wave function follows the usual Schrödinger evolution. λ and rC are two new constants of nature, which in the CSL model are assumed to take the values
The wave function of a composite object consisting of N nucleons collapses at the rate Nλ: this is the desired amplification mechanism. The CSL effect is mathematically described by a stochastic non-linear modification of the Schrödinger equation which satisfies the constraints mentioned above. CSL provides a solution of the measurement problem, explains the classical nature of macroscopic objects, and explains the Born probability rule, by converting the linear superposition into an approximate principle. The theory makes predictions for experiments, which differ from the predictions of quantum theory, in the currently untested mesoscopic domain. Ongoing laboratory experiments are at present in an exciting and crucial phase, and could verify or rule out CSL in the next few years , , , , .
A serious limitation of the CSL model is that it is non-relativistic, and dedicated efforts to make a relativistic version face serious difficulties. And perhaps for good reason. We will argue below that collapse and relativity are incompatible, and the non-locality induced by collapse can be meaningfully understood only in a non-commutative space-time obeying a generalised Lorentz invariance.
3.2 Collapse Models are Phenomenological: What is their Fundamental Origin?
Another limitation of dynamical collapse models, of which CSL is the most advanced, is that they are purely phenomenological, having been designed for the express purpose of solving the quantum measurement problem. A more fundamental theory would derive the CSL model from some underlying first principles. A beautiful attempt in this direction has been made in the theory of trace dynamics (TD) developed by Adler and collaborators , , . TD is a Poincaré-invariant classical dynamics theory in which the fundamental canonical degrees of freedom are matrices whose elements are complex-valued Grassmann variables. One constructs the Lagrangian and Hamiltonian dynamics of this classical theory in the usual way, and the configuration and momentum variables qi and pi, which are matrices (equivalently operators), all obey arbitrary commutation relations with each other. Nonetheless, as a result of a global unitary invariance, the theory possesses the remarkable conserved charge
3.3 A Possible Resolution of the Problem of Time
TD takes an external classical space-time as given. However, as we have argued above, there ought to exist an equivalent reformulation of quantum theory that does not refer to an external classical time. A generalisation of TD enables us to arrive at such a reformulation. The starting point for this generalisation is the proposal that, at a more fundamental level, physical laws should be invariant under general coordinate transformations of non-commuting coordinates. Amazingly, this generalisation of Einstein’s general covariance principle provides a path from classical dynamics to quantum dynamics. We first treat the simpler case of a ‘Minkowski’ non-commuting space-time, with coordinates which are now operators (t̂,
with the operators (t̂, x̂, ŷ, ẑ) satisfying arbitrary commutation relations, in the spirit of TD. Matter and space-time degrees of freedom are now being treated at par. Tr stands for the ordinary matrix trace, and the proper time s, which we call trace time, plays a very important role in what follows. In this space-time, because it is non-commutative, there is no point structure or light-cone structure, nor a notion of spatial distance or temporal interval. Nonetheless, this metric is invariant under Lorentz transformations of the non-commuting operator coordinates, and one can construct a Poincaré-invariant classical dynamics for matter degrees of freedom, which generalises special relativity and which we call non-commutative special relativity . The canonical degrees of freedom now include, apart from the position-momentum pair (q̂, p̂), also the energy-time pair (t̂, Ê), and the generalised Adler-Millard charge includes also the commutator [t̂i, Êi] for every degree of freedom. Evolution is described with respect to the trace time s. Following TD, one constructs the equilibrium statistical thermodynamics of the underlying non-commutative relativity, and obtains the canonical quantum commutation relations, along with the commutation relation [t̂, Ê] = iℏ. It is important to note that at this emergent equilibrium level, the configuration variables, including the space-time degrees of freedom (t̂, x̂, ŷ, ẑ), all commute with each other, although they continue to be operators. The generalised quantum dynamics (GQD) that we obtain satisfies the generalised Schrödinger equation
Evolution is given by trace time s, and the configuration variables include the operator time t̂ as well. This is the sought-for reformulation of quantum theory that does not refer to classical time . There is no background classical space-time, with the line element still being given by (2).
To show that this reformulation is equivalent to quantum theory, we must first recover the classical universe with its classical space-time and classical distribution of macroscopic bodies. To do this, we bring on the scene the statistical fluctuations of the Adler-Millard charge, which results in a stochastic non-linear modification of the GQD given by (3). Furthermore, now the space-time degrees of freedom (t̂, x̂, ŷ, ẑ) no longer commute with each other. Consider the very early universe, where there are present tiny energy-density fluctuations in the matter degrees of freedom, as well as fluctuations in the non-commutative space-time geometry. As these fluctuations grow, the non-linearity in the GQD becomes significant, resulting in a CSL-like effect: matter clumps into localised classical configurations. This is accompanied by, again because of the non-linearity, the localisation of the space-time degrees of freedom (t̂, x̂, ŷ, ẑ), which can hence be mapped to c-numbers, and to a classical space-time (t, x, y, z) . The collapse of the wave function is responsible for the emergence of classical macroscopic bodies, and also for the emergence of classical space and time. Because the original line-element (2) is Lorentz-invariant, the emergent line-element is the standard Minkowski space-time. It is rather intuitive and convincing that spontaneous collapse of the wave function that localises macroscopic objects such as stars and galaxies on a grand scale across the entire universe also gives rise to space: space is that which is between collapsed objects. No collapse, no space. Know collapse, know space! Not only does matter curve space-time, the spontaneous localisation of matter gives rise to space-time in the first place.
Given this classical background, one can show that the reformulation as GQD is equivalent to quantum theory. One can consider a micro-system either according to the generalised TD with (2) and arrive at (3), or one can consider it according to standard TD on a classical space-time and arrive at the Schrödinger equation. The two are equivalent because the operators (t̂, x̂, ŷ, ẑ) in the GQD commute at the emergent level, and can be mapped to the ordinary (t, x, y, z). And trace time can be mapped to ordinary proper time .
3.4 A Possible Resolution of the Non-Locality Puzzle
Given the machinery at hand, we can resolve the puzzle of non-locality described earlier. When a measurement is made on A to collapse its wave function, the statistical fluctuations of the Adler-Millard charge kick on, and the GQD becomes stochastic and non-linear. The space-time operators (t̂, x̂, ŷ, ẑ) associated with the quantum system (AB) no longer commute, and cannot be mapped to the (t, x, y, z) of the background space-time. Thus, the non-linear GQD, although it is invariant under Lorentz transformations of the non-commutative line element (2), is not invariant under the Lorentz transformations of standard Minkowski space. Thus, collapse and special relativity are incompatible with each other. GQD can be mapped to ordinary quantum theory and is compatible with special relativity, but when the collapse-inducing statistical fluctuations are added on to GQD, it is not Lorentz-invariant and hence there is no relativistic theory of collapse. (In the non-relativistic limit, there is an absolute time, and no need to worry about Lorentz invariance, so the CSL effect can be modelled using a random stochastic field.) We now understand why quantum theory gels so nicely with special relativity but does not gel well when collapse is included: we are confronted with the spooky action at a distance. The non-local quantum correlation is induced by collapse, and hence can be correctly described only in the underlying non-commutative space-time (which is Lorentz invariant). In this description, since there is no notion of spatial distance between A and B, there is simply no action at a distance. Nor is there any question of an influence travelling from A to B: travelling requires distance, and here there is no distance in the first place. Micro-causality and quantum non-locality can co-exist: the former belongs to standard quantum theory, the latter belongs to the stochastic non-linear modification of standard quantum theory. This latter modification is not consistent with special relativity, but is consistent with non-commutative special relativity .
3.5 What is the Physical Meaning of the Wave Function?
Now we have an answer to Feynman’s remark that no one understands quantum mechanics. No one understands quantum mechanics because space-time is not the best arena for describing quantum mechanics. Non-commutative space-time is. Let us revisit the double slit experiments with electrons. We have trouble, understandably, in thinking of the complex-valued wave function acting as a wave, interfering with itself in physical space: this makes no sense at all. However, the truth is that, fundamentally, the wave function lives in the associated non-commutative line-element (2) of the electron. The complex-valued state of the electron belongs to the Hilbert space of the generalised (matrix) TD, which includes a non-commutative space-time. It is in no need of classical space-time for its description: the classical space-time is produced by collapsed classical objects, and is approximate. It creates problems when we try to understand quantum mechanics. But the underlying Hilbert space encompasses matter states as well as states of space-time. The troublesome distinction between space and the Hilbert space of quantum theory has been removed. But via the trace time s, we do have a very useful notion of time at hand. It is as if time is more fundamental than space. In the classical limit, we can attach classical observers to inertial frames, so the coordinate time t plays a useful role as time. But there is no such thing as a quantum observer, so the operator time t̂ is of no use to observers.
4 Quantum Theory and Space-Time are Emergent Phenomena
There has been much talk lately , of space, and perhaps space-time, as emergent from quantum entanglement. But it would seem that this is hard to achieve within the confines of standard quantum theory. Quantum entanglement must be lost to arrive at classical space. But to lose entanglement, we need collapse. And collapse perhaps requires us to go beyond quantum theory, and modify it.
When we go to that ‘beyond’, we find that there is a new world out there. Both quantum theory and space-time are emergent phenomena. They emerge from the generalised TD. In the new world, nothing commutes with anything. Nothing is local, neither in space, nor in time. Locality and separability are approximations of our present-day universe. Spontaneous collapse is omnipotent and at play, making the universe look like it does. However, appearances can be deceptive. Deep down, if we look very carefully, everything is everywhere all the time, in a manner of speaking. Wandering is a property of the approximate universe. It is an illusion.
5 Technical Endnotes
Given a system of N particles, with a wave function belonging to the Hilbert space, its dynamics satisfies the following properties. During its evolution, the wave function undergoes repeated spontaneous collapse at random times, mathematically described as
Here, Ln( x) is the so-called jump operator, which is defined as
which localises the nth particle to the spatial location x in a region of size rC, with qn being the position operator of the nth particle. The probability for this jump to position x by the nth particle is given by
The jumps are assumed to occur according to a Poisson process, with a frequency λ. Thus λ and rC are the two new parameters of the model. Between any two jumps, the wave function undergoes normal Schrödinger evolution. The mass density of the nth particle in physical space is defined as
These then are the axioms of the model of spontaneous collapse. It is a universal dynamics, to which quantum mechanics and classical mechanics are limiting approximations. There is no need to refer to any classical measuring apparatus, or environment, or measurement, or macroscopic world. Measurement is just a special case of spontaneous collapse.
The beauty of spontaneous collapse is the natural manner in which the amplification mechanism comes about. In a bound system of N particles, any one particle undergoing collapse causes the entire system to collapse. Thus the effective collapse rate for the system is Nλ, which is an enormous amplification. If the individual particle is a nucleon, then the collapse rate can become enormously high for a macroscopic system, because N is very large. Thus macroscopic objects stay effectively localised in space, and this explains their classical behaviour and solves the measurement problem.
Mathematically, spontaneous collapse can be described as a continuous process, through a stochastic non-linear modification of the Schrödinger equation:
Here, the first term describes the usual Schrödinger evolution, with H being the quantum Hamiltonian. The second and third terms are the new terms that cause dynamical collapse. The new terms are non-unitary, yet they maintain the norm-preserving nature of the evolution. m0 is a reference mass, conventionally chosen to be the mass of the nucleon. M( x) is the mass density operator
The collapse inducing stochasticity in the CSL model is described by Wt( x), which is an ensemble of independent Wiener processes, one for each point in space. The constant γ is related to the rate parameter λ as
S. Weinberg, The Trouble with Quantum Mechanics, New York Review of Books, New York 2017.
N. Gisin, arXiv:1701.08300 [quant-ph] (2017).
G. Musser, Spooky Action at a Distance, Farrar, Straus and Giroux, NY, USA 2015.
L. K. Shalm, E. Meyer-Scott, B. G. Christensen, P. Bierhorst, M. A. Wayne, et al., Phys. Rev. Lett. 115, 250402 (2015).
T. P. Singh, Bulg. J. Phys. 33, 217 (2006). arXiv:gr-qc/0510042.
S. L. Adler, Quantum Theory as an Emergent Phenomenon, Cambridge University Press, Cambridge 2004, p. xii+225.
T. P. Singh, in: Re-thinking Time at the Interface of Physics and Philosophy (Eds. T. Filk and A. von Muller), Springer, Berlin-Heidelberg 2015. arXiv:1210.81110.
T. P. Singh, arXiv:1701.09132 [quant-ph] (2017).