Von Neumann told Shannon to call his measure entropy, since “no one knows what entropy is, so in a debate you will always have the advantage.

― Jeremy Campbell

imagesToo often in seeing discourse about numerical methods, one gets the impression that dissipation is something to be avoided at all costs. Calculations are constantly under attack for being too dissipative. Rarely does one ever hear about calculations that are not dissipative enough. A reason for this is the tendency for too little dissipation to cause outright instability contrasted with too much dissipation with low-order methods. In between too little dissipation and instability are a wealth of unphysical solutions, oscillations and terrible computational results. These results may be all too common because of people’s standard disposition toward dissipation. The problem is that too few among the computational cognoscenti recognize that too little dissipation is as poisonous to results as too much (maybe more).

Why might I say that it is more problematic than too much dissipation? A big part of the reason is the nature physical realizability of solutions. A solution with too much dissipation is utterly physical in the sense that it can be found in nature. The solutions with too little dissipation more often than not are not found in nature. This is not because those solutions are unstable, but rather solutions that are stable, and have some dissipation; however, they simply aren’t dissipative enough to match natural law. What many do not recognize is that natural systems actually produce a large amount of dissipation without regard to the size of the mechanisms for explicit dissipative physics. This is both a profound physical truth, and the result of acute nonlinear focusing. It is important for numerical methods to recognize this necessity. Furthermore, this fact of nature reflects an uncomfortable coming together of modelling and numerical methods that many simply choose to ignore as an unpleasant reality.

In this house, we obey the laws of thermodynamics!

– Homer Simpson

Entropy stability is an increasingly important concept in the design of robust, accurate and convergent methods for solving systems defined by nonlinear conservation laws (see Tadmor 2016) The schemes are designed to automatically satisfy an entropy inequality that comes from the second law of thermodynamics, d S/d t \le 0. Implicit in the thinking about the satisfaction of the entropy inequality is a view that approaching the limit of $latex d S / d t = 0$ as viscosity becomes negligible (i.e., inviscid) is desirable. This isSupersonic-bullet-shadowgram-Settles.tif a grave error in thinking about the physical laws of direct interest, as the solution of conservation laws does not satisfy this limit when flows are inviscid. Instead the solutions of interest (i.e., weak solutions with discontinuities) in the inviscid limit approach a solution where the entropy production is proportional to variation in the large scale solution cubed, d S / d t \le C \left(\Delta u\right)^3. This scaling appears over and over in the solution of conservation laws including Burgers’ equation, the equations of compressible flow, MHD, and incompressible turbulence (Margolin & Rider, 2001). The seeming universality of these relations and their implications for numerical methods are discussed below in more detail, but follow the profound implications turbulence modelling are explored in detail for implicit LES modelling (our book edited by Grinstein, Margolin & Rider, 2007). Valid solutions will invariably produce the inequality, but the route to achievement varies greatly.

The satisfaction of the entropy inequality can be achieved in a number of ways and the one most worth avoiding is oscillations in the solution. Oscillatory solutions from nonlinear conservation laws are as common as they are problematic. In a sense, the proper solution is strong attractor for solutions and solutions will adjust to produce the necessary amount of dissipation in the solution. One vehicle for entropy production is oscillations in the solution field. Such oscillations are unphysical and can result in a host of issues undermining other physical aspects of the solution such as positivity of quantities such as density and pressure. They are to be avoided to whatever degree possible. If explicit action isn’t taken to avoid oscillations, one should expect them to appear.

There ain’t no such thing as a free lunch.

― Pierre Dos Utt

A more proactive approach to dissipation leading to entropy satisfaction is generally desirable. Another path toward entropy satisfaction is offered by numerical methods in control volume form. For second-order numerical methods the analysis of the approximation via the modified equation methodology unveils nonlinear dissipation terms that provide the necessary form for satisfying the entropy inequality via a nonlinearly dissipative term in the truncation error. This truncation error takes the form  C u_x u_{xx} , which integrates to replicate inviscid dissipation as a residual term in the “energy” equation, C\left(u_x\right)^3. This term comes directly from being in conservation form and disappears when the approximation is in non-conservative from. In large part the overly large success of these second-order methods is related to this character.

Other options to add this character to solutions may be achieved by an explicit nonlinear (artificial) viscosity or through a Riemann solver. The nonlinear hyperviscosities discussed before on this blog work well. One of the pathological misnomers in the community is the belief that the specific form of the viscosity matters. This thinking infests direct numerical simulation (DNS) as it perhaps should, but the reality is that the form of dissipation is largely immaterial to establishing physically relevant flows. In other words inertial range physics does not depend upon the actual form or value of viscosity its impact is limited to the small scales of the flow. Each approach has distinct benefits as well as shortcomings. The key thing to recognize is the necessity of taking some sort of conscious action to achieve this end. The benefits and pitfalls of different approaches are discussed and recommended actions are suggested.

Enforcing the proper sort of entropy production through Riemann solvers is another possibility. A Riemann solver is simply a way of upwinding for a system of equations. For linear interaction modes the upwinding is purely a function of the characteristic motion in the flow, and induces a simple linear dissipative effect. This shows up as a linear even-order truncation error in modified equation analysis where the dissipation coefficient is proportional to the absolute value of the characteristic speed. For nonlinear modesupersonic-bullet_660s in the flow, the characteristic speed is a function of the solution, which induces a set of entropy considerations. The simplest and most elegant condition is due to Lax, which says that the characteristics dictate that information flows into a shock. In a Lagrangian frame of reference for a right running shock this would look like, c_{\mbox{left}} > c_{\mbox{shock}} > c_{\mbox{right}} with c being the sound speed. It has a less clear, but equivalent form through a nonlinear sound speed, c(\rho) = c(\rho_0) + \frac{\Delta \rho}{\rho} \frac{\partial \rho c}{\partial \rho}. The differential term describes the fundamental derivative, which describes the nonlinear response of the sound speed to the solution itself. This same condition can be seen in a differential form and dictates some essential sign conventions in flows. The key is that these conditions have a degree of equivalence. The beauty is that the differential form lacks the simplicity of Lax’s condition, but establishes a clear connection to artificial viscosity.

The key to this entire discussion is realizing that dissipation is a fact of reality. Avoiding it is simply a demonstration of an inability to confront the non-ideal nature of the universe. This is simply contrary to progress and a sign of immaturity. Let’s just deal with reality.

The law that entropy always increases, holds, I think, the supreme position among the laws of Nature. If someone points out to you that your pet theory of the universe is in disagreement with Maxwell’s equations — then so much the worse for Maxwell’s equations. If it is found to be contradicted by observation — well, these experimentalists do bungle things sometimes. But if your theory is found to be against the second law of thermodynamics I can give you no hope; there is nothing for it but to collapse in deepest humiliation.

– Sir Arthur Stanley Eddington


Tadmor, E. (2016). Entropy stable schemes. Handbook of Numerical Analysis.

Margolin, L. G., & Rider, W. J. (2002). A rationale for implicit turbulence modelling. International Journal for Numerical Methods in Fluids, 39(9), 821-841.

Grinstein, F. F., Margolin, L. G., & Rider, W. J. (Eds.). (2007). Implicit large eddy simulation: computing turbulent fluid dynamics. Cambridge university press.

Lax, Peter D. Hyperbolic systems of conservation laws and the mathematical theory of shock waves. Vol. 11. SIAM, 1973.

Harten, Amiram, James M. Hyman, Peter D. Lax, and Barbara Keyfitz. “On finite‐difference approximations and entropy conditions for shocks.” Communications on pure and applied mathematics 29, no. 3 (1976): 297-322.

Dukowicz, John K. “A general, non-iterative Riemann solver for Godunov’s method.” Journal of Computational Physics 61, no. 1 (1985): 119-137.