Your assumptions are your windows on the world. Scrub them off every once in a while, or the light won’t come in.
― Isaac Asimov
In conducting science, the importance of models is central to practice. Modeling is paired with observation as Man’s abstraction for understanding the World around us. Models need to be descriptive and tractable for examining nature. These two aspects can be in direct conflict with each other. Observation under natural or controlled circumstances provides the core of scientific knowledge. Observation becomes science when we provide a systematic explanation for what we see. More often than not this explanation has a mathematical character as the mechanism we use. Among our mathematical devices differential equations are among our most powerful tools. In the most basic form these equations are rate of change laws for some observable in the World. Most crudely these rate equations can be empirical vehicles for taking observations into a form useful for prediction, design and optimization. A more basic form is partial differential equations (PDEs) that describe the basic physics in a more expansive form. It is important to consider the consequences of the model forms we use. Several important categories of models are intrinsically unphysical in aspects thus highlighting the George Box aphorism that “essentially, all models are wrong”!
Assumptions are the most damaging enemies of our mind’s equilibrium…An assumption is an imaginary truth.
― A.A. Alebraheem
Partial differential equations come in three basic flavors, hyperbolic, parabolic and elliptic. These flavors describe some of the basic character of the equations and have fundamental differences in how they are solved, understood as objects and more importantly physical context. The core of this essay is going to be physical in nature and to the point only hyperbolic equations are primal in physics. This is to say that at the basic level everything that we might describe as a physics law is hyperbolic in character. This is for a simple and very good reason, the principle of causality. Cause and effect, the flow of time and the presence of a cosmic speed limit. If we adhere to these maxims, the conclusion is utterly obvious. Other forms of PDEs produce instantaneous global effects that violate this principle. This in no way implies that parabolic or elliptic models are not incredibly useful, they are. Their utility and other properties exceed the issues with causality violations.
More on that point soon, but first a bit of digression on the other forms of PDEs. The classical elliptic equations is Laplace’s equation, . Elliptic equations are the simplest form and often describe physics where spatial terms are in equilibrium and there is no temporal, rate terms. Elliptic equations can include time terms, but usually implying something so deeply unphysical as to be utterly outlawed. If time is elliptic, the past is determined by the future, and since we know that time flows in one direction, this is deeply and fundamentally unphysical. In other uses, the elliptic PDEs are found through ignoring temporal terms. This is a philosophical violation of the second law of thermodynamics, which can be used to establish the arrow of time. In this sense we find that elliptic equations are an asymptotic simplification of more fundamental laws. Another implication of ellipticity of PDEs is infinite speed of information, or more correctly an absence of time. If elliptic equations are found within a set of equations, we can be absolutely sure that some physics has been chosen to be ignored. In many cases these ignored physics are not important and some benefit is achieved through the simplification. On the other hand, we shouldn’t lose sight of what has been done and its potential for mischief. At some point this mischief will become relevant and disqualifying.
Assumptions aren’t facts; they’re opportunities for research and testing.
― Laurie Buchanan
Next along the way we have parabolic equations and we can repeat the above discussion. Most classically the equation of heat transfer is parabolic (along with other diffusion processes). The classical form is the heat equations, . We often learn that these diffusion processes are fundamental, leading to the second law of thermodynamics. This comes with a deep problem that we should acknowledge. The parabolic equations imply an infinite propagation speed. Physically the process of diffusion is quite discrete associated with collisionality of the particles that make up materials, or discrete effects of solids (where electrons are particles that move, exchange and interact). This physical effect is utterly bound by finite speeds of propagation.
With elliptic equations the strength of the signal is unabated in time, but with parabolic equations, the signal diminishes in time. As such the sin of causality violation isn’t quite so profound, but it is a sin nonetheless. As before we get parabolic equations by ignoring physics. Usually this is a valid thing to do based on the time and length scales of interest. We need to remember that at some point this ignorance will damage the ability to model. We are making simplifications that are not always justified. This point is lost quite often. People are allowed to think the elliptic or parabolic equations are fundamental when they are not.
We now get to the third category of PDEs, the hyperbolic kind. The simplest form is a wave equation, . This can be written as a system of equations first-order PDEs, and . We can derive the simple wave equation by differentiating the first equation in time and the second in space then substituting to eliminate . The propriety of these steps depends on the variable being continuously differentiable, i.e., smooth. The second, first-order form is the entry point for the beautiful mathematics of hyperbolic conservation laws. As we will show, the elliptic and parabolic equations are simplifications of the hyperbolic equations made upon applying some assumptions.
I suppose it is tempting, if the only tool you have is a hammer, to treat everything as if it were a nail.
― Abraham H. Maslow
One key example of an equation where the presumption of the basic laws of physics is generally wrong are diffusion processes. One might consider Fourier’s law to be a fundamental law of physics as applied to heat condition in a parabolic form, . Instead this is a simplification of a more broadly valid law where heat flows using a hyperbolic equation. This requires a simple modification of the Fourier law to . For most applications heat flow can be modeled in the parabolic form as the hyperbolic form is important over very short time and space scales. Still the more fundamental law is the hyperbolic form, and the classical parabolic form is derived by assuming that certain aspects of the dynamics can be ignored. We must always remember that the standard modeling of diffusion processes has an unphysical aspect baked into the equations.
The importance of the hyperbolic character of heat conduction may be more important. It is related to the property of material called second sound. This property is measured and known to be significant under cryogenic conditions where quantum effects are significant. It is also very hard to measure. The leading and rather compelling fact is its relation to the sound speed; the second sound is slower than the sound speed. This would mean that its effective time scale is also longer than the acoustic time. If this aspect is generally true, then the time scale can’t be ignored under many conditions. The deeper question is how all of this plays with thermodynamics. So much of the grounding of thermodynamics is in an equilibrium setting, and this phenomenon adds a natural and potentially important relaxation time scale.
There is an issue with hyperbolic diffusion that we should acknowledge. This form of the equations can violate the second law of thermodynamics, which underpins the macroscopic dynamics of the universe and the arrow of time. By the same token the imposition of the second law through a physical process in continuum physics is invariably tied to diffusion. As such we have formed a veritable technical Mobius strip. A question is whether a fundamentally different equation can actually violate a less physical law that it is based upon. This might call the violations of the second law by hyperbolic diffusion into question rather directly! In other words, what would change about the second law of thermodynamics; if the diffusion process itself were hyperbolic. Perhaps this is a specific inroad to discussions of non-equilibrium thermodynamics. This may provide a necessary and distinct framing of a deeper discussion. Clearly infinite speeds of propagation for information is unphysical, functionally the second law could be recovered to account for temporal effects.
There is nothing so expensive, really, as a big, well-developed, full-bodied preconception.
― E.B. White
The incompressible Navier-Stokes equations are second primal example where hyperbolic equations are replaced by both parabolic-elliptic equations. A starting point would be the compressible equations that are purely hyperbolic without viscosity. Of course, the viscosity could be replaced with hyperbolic equations to make the compressible flow totally hyperbolic. These equations are the following, . Previously, we discussed the replacement of hyperbolic diffusion by the parabolic terms. For the incompressibility we remove sound waves analytically. The key to doing this is remove any connection between pressure and density with the divergence free constraint, . This also turns mass into a passively advected scalar. This is a useful model for low speed flows, but the diffusion and suppressed sound waves both produce infinite speeds of propagation. This violates the principle of causality where there is cause and effect. Instead everywhere is impacted by everything immediately.
As noted repeatedly these infinite speeds are definitely and unabashedly unphysical and signs that the equations are intrinsically limited in modeling scope. These issues are almost routinely ignored by most scientists and engineers. The reason is that the assumptions associated with parabolic or elliptic equations are valid for use. This will not always be true. It should be in the back of their mind. The message is clear, the equations will become invalid under some conditions, some length or time scale will unveil this invalidity. The question is what are these scales, and have we stumbled upon them yet? More generally the use of parabolic or elliptic equations produce these unphysical effects as a matter of course. This implies that the model equation will lose utility at some point under some conditions. We simply need to guard ourselves to this potential and keep this firmly in the back of our mind. The issue in this regard is the lack of capability to solve these non-standard models and make a complete assessment of model validity. By the same token, the non-standard models are harder to solve and may have deleterious side-effects if the full physics is retained.
A very good example of these side effects occurs with compressible flows when the Mach number is small. Solving low-Mach number flows with compressible codes is terribly inefficient and prone to significant approximation errors. This has a great deal to do with the separation of scales. As a result, the solutions often do not adhere to expectations. The consequence is there are many “fixes” to compressible flow solvers to remove the difficulties. The odd thing about this issue is the definitely greater physical reality associated with the compressible flow equations as compared to the incompressible equations. This might imply that the conditioning of the equations is the greatest problem. In addition, modern shock capturing methods have an implied discontinuity with their construction. It would seem that a continuous approximation might alleviate the problems. Conditioning issues with the separation of scales remains.
For modeling and numerical work, the selection of the less physical parabolic and elliptic equations provides better conditioning. The conditioning provides a better numerical and analytical basis for the solutions. The recognition that the equations are less physical is not commonly appreciated. A broader and common appreciation may provide impetus for identifying when these differences are significant. Holding models that are unphysical as sacrosanct is always dangerous. It is important to recognize the limitations of models and allow ourselves to question them regularly. Even models that are fully hyperbolic are wrong themselves, this is the very nature of models. By using hyperbolic models, we remove an obviously unphysical aspect of a given model. Models are abstractions of reality, not the operating system of the universe. We must never lose sight of this.
Everything must be made as simple as possible. But not simpler.
― Albert Einstein
Courant, Richard, and David Hilbert. Methods of mathematical physics [Methoden der mathematischen Physik, engl.] 1. CUP Archive, 1965.
Lax, Peter D. Hyperbolic partial differential equations. Vol. 14. American Mathematical Soc., 2006.
Körner, C., and H. W. Bergmann. “The physical defects of the hyperbolic heat conduction equation.” Applied Physics A 67, no. 4 (1998): 397-401.
Chester, Marvin. “Second sound in solids.” Physical Review131, no. 5 (1963): 2013.
Christov, C. I., and P. M. Jordan. “Heat conduction paradox involving second-sound propagation in moving media.” Physical review letters 94, no. 15 (2005): 154301.
Fefferman, Charles L. “Existence and smoothness of the Navier-Stokes equation.” The millennium prize problems 57 (2006): 67.
Doering, Charles R. “The 3D Navier-Stokes problem.” Annual Review of Fluid Mechanics 41 (2009): 109-128.