You should never be surprised by or feel the need to explain why any physical system is in a high entropy state.
― Brian Greene
For those of you who know me, it’s neither a secret, nor a widely known fact that I’ve gotten some tattoos recently. They aren’t the usual dreck most dudes get (like those tribal ones), but meaningful things to me. Now I have five in total, four of them are science related. One of the things that I wanted was an equation (yeah, I’m a total nerd). The question is what equation do I believe in enough to get permanently inscribed on my skin? A “common” choice for a science tattoo is Maxwell’s equations, and a friend of mine has the Euler equations on this arm from his PhD thesis. This post is about the equation I chose to care enough about to go through with it.
I’ll write the equation in TeX and show all of you a picture, you can make out a little of my other ink too, a lithium-7 atom and a Rayleigh-Taylor instability (I also have my favorite dog’s paw on my right shoulder and the famous Von Karmen vortex street on my forearm). The equation is how I think about the second law of thermodynamics in operation through the application of a vanishing viscosity principle tying the solution of equations to a concept of physical admissibility. In other words I believe in entropy and its power to guide us to solutions that matter in the physical world.
The all-knowing yesterday is obsolete today.
― Jarod Kintz
This is in contrast to much of the mathematical world that often cares about equations that are beautiful, but mean nothing in reality. A lot of the tension is related to following the beauty of Newtonian determinism and its centrality to continuum mathematical physics, and the need to embrace to stochastic nature of the real world. The real world is random and flows along time’s arrow and needs to embrace entropy and uncertainty. Our education and foundational knowledge of the physical world is based on Newton’s simplified view of things (a simplified view that revolutionalized science if not mankind’s understanding and mastery over nature). Newton’s principles can only take us so far, and we are probably reaching to end of its grasp. It is past time to push forward toward incorporating new principles into our model of reality.
Here is the equation in all its glorious mathematical statement, taking the limit as . The equation is the time rate of change of a variable determined by a flux balance and a diffusive term where the limit of diffusion is taken to zero, which implies the satisfaction of the second law of thermodynamics implying that entropy increases. OK, but WTF is it all about? So in words the equation is a hyperbolic conservation law with a diffusive right hand side where the coefficient of viscosity goes to a limit of zero. In this limit we find solutions that are physically admissible, that is ones that could exist in the real World. These solutions lead to satisfaction of the second law of thermodynamics, which implies that entropy or disorder monotonically increases in time. The second law can be viewed as the thing that gives time a direction (time’s arrow!), and without the increase of entropy, time can flow equally well forward or backward, that is being symmetric. We know time flows forward in the real world we all live in, so we want that (or at least I want that, and believe you should too).
lot of effort is spent studying the equations of inviscid flow, flow without dissipative forces most commonly the Euler equations. This form of equation is studied a lot because it is so pure. One can really get some awesomely beautiful mathematics out of it. Commonly the math leads to some great structure by studying these systems through their Hamiltonian and its evolution. Unfortunately, this endeavor while beautiful and hard has no physical merit whatsoever. No physical system really adheres to this Hamiltonian structure (except perhaps isolated systems of very small scale, and I really don’t give a much of a shit about these). They are seductive, pretty and almost without any physical utility. I care about stuff that appears in nature.
The important thing for equations to represent is physical reality (unless you’re doing math for math’s sake). As Wigner pointed out, mathematics has an incredible, almost mystic capacity to model our reality as he says is unreasonably effective. Exploiting this power should be a privilege we exercise whenever possible. In that vein the equations that connect to reality should be favored. In many cases inviscid equations are incredibly useful for modeling, but an important caveat should be exercised, the solutions to the inviscid equations that are favored are those associated with the presence of viscosity. These solutions are found through the application of an asymptotic principle, vanishing viscosity. The application of vanishing viscosity provides a route for these equations to satisfy the second law of thermodynamics, and its demands for increasing disorder.
These principles actually don’t go far enough in distinguishing themselves from inviscid dynamics associated with Hamiltonian systems. Let me explain how they need to go even further. A couple of the most profound observations associated with fluid dynamics are associated with shock waves and turbulence, and share a remarkable similarity (it might be argued that both are inevitable through dimensional similarity arguments!). For shock waves the amount of dissipation occurring via the passage of a shock is proportional to the size of the jump in the variables across the shock cubed (Bethe came up with this in 1942). For turbulence the amount of dissipation is a high Reynolds number flow is proportional to the size of the velocity variation cubed (Kolmogorov came up with this in 1941). Both relations are independent of the specific value of the molecular viscosity.
What people resist is not change per se, but loss.
― Ronald A. Heifetz
These relations are profound in their implications, which are not intuitively obvious upon first seeing it. The dissipation rate being independent of the value of viscosity means that the flow contains something that approaches a singularity. These singularities are called shock waves and have no name at all in turbulence because we don’t know what they are. These singularities are the mystery of turbulence and they are surely ephemeral as they are important. In other words we don’t see the turbulent singularities like we see shocks, but they must be there. Moreover the supposed equations of turbulence, the incompressible Euler equations don’t appear to contain obviously singularity producing features. This whole issue has produced an utterly stagnant scientific endeavor of immense practical importance.
Of course what is usually not covered is the horribly degenerate and unphysical nature of the incompressible Euler or Navier-Stokes equations. The key term is incompressible, which is intrinsically unphysical and removes sound waves from the system making their propagation speed infinite. What if these sound waves, which are always present, contain the essence of what drives dissipation in turbulent system. Incompressibility also removes thermodynamics from the equations and can only be derived from the compressible Navier-Stokes by considering the flow to be adiabatic. Turbulence in its essential character is non-adiabatic and intrinsically dissipative. Anyone see the problem(s)? Perhaps its time to start considering that the lack of progress in turbulence is exposing fundamental flaws in our modeling paradigm.
I would posit that we are trying to solve this monumentally important problem with a set of equations that we have systematically crippled. These equations were posed during an era where the fundamental issues discussed above were not well known. There really isn’t an excuse today. Could our lack of progress with turbulence be completely related to focusing on the wrong set of equations (yes!)?
Let’s dig just a bit deeper on the philosophical implications of the Bethe’s and Kolmogorov’s relations for dissipation of energy. Both of these relations also imply a satisfaction of the second law of thermodynamics by these systems. The limiting value for the satisfaction of the second law is not simply the inequality at zero, but rather an inequality for a finite value of dissipation. This finite value of dissipation is directly related to the large-scale flows structure and quantitatively proportional to the cube of the variations in the inertial range. Thus, the limit of zero dissipation is not physically relevant, the limit is a finite amount of dissipation set by the large scales of the system of interest. This deepens the implications associated with any study of completely dissipation-free dynamics being utterly unphysical. The dissipation-free system is separated from the real world by a finite and non-vanishing distance.
This feature of the physical world should be reflected in how we numerically model things (this is my philosophical point of view). It gets to the core of why I chose the equation to ink on my skin. A lot of numerical work is focused on trying to remove every single bit of dissipation from the method while maintaining stability. This mantra is tied to the belief that dissipation is bad and one is fundamentally interested in the numerical solution to the dissipation-free Euler equations. I believe this is utterly foolhardy and unproductive. The dissipation-free Euler equations are close to useless. The key dissipative relations I’ve introduced above tell us that the dissipation is never zero, but rather non-zero in a very specific way that is irreducible.
Some would argue that this non-zero dissipation should be the target of modeling, and the numerical methods should be pure and not intrude into the modeling. I believe that this perspective is laudable, foolish and unworkable practically. I favor more holistic approaches that combine modeling and numerical methods into a seamless package. This approach works wonderfully well in numerical methods for shocks, and produces a set of methods that revolutionized the field of computational fluid dynamics (CFD). I believe the grasp of these methods is far greater and extends into turbulence through implicit large eddy simulation (ILES). ILES implies strongly that the turbulence modeling is strongly addressed by techniques that practically solve compressible flows in the vanishing viscosity limit.
Generally for turbulence this approach has not been taken and the reason is clear to see. Turbulence modeling remains to this day a dominantly empirical activity. The core reason for this is the comment above about knowing what the dissipative structures are in turbulence. In compressible flows we know that shock waves are the thing to focus on and where the invariant dissipation occurs. Shocks are the hard thing to compute numerically, and we know what to do. For turbulence the same things cannot happen and the result is empirical modeling without targeted numerical methods. What remains is a philosophy that drives numerical methods to be innocuous, and allow the modeling to hold sway. The problem is that the modeling is blind to what the real physics is doing and the precise mechanisms to connect the large-scale flow to the dissipation of energy.
Nothing limits you like not knowing your limitations.
― Tom Hayes
As far as the tattoos are concerned, I haven’t decided yet if I’m getting more ink or not (its probably a yes). Maybe I’ll keep to the theme of science on the left side of my body and personal meaning on the right side of my body. Ideas are hatching, and I need to mind my tendency towards obsessive-compulsive behavior.
The law that entropy always increases holds, I think, the supreme position among the laws of Nature. If someone points out to you that your pet theory of the universe is in disagreement with Maxwell’s equations — then so much the worse for Maxwell’s equations. If it is found to be contradicted by observation — well, these experimentalists do bungle things sometimes. But if your theory is found to be against the second law of thermodynamics I can give you no hope; there is nothing for it but to collapse in deepest humiliation.
— Sir Arthur Stanley Eddington
Wigner, Eugene P. “The unreasonable effectiveness of mathematics in the natural sciences. Richard courant lecture in mathematical sciences delivered at New York University, May 11, 1959.” Communications on pure and applied mathematics 13.1 (1960): 1-14.
Bethe, H. A. “On the theory of shock waves for an arbitrary equation of state.” Classic papers in shock compression science. Springer New York, 1998. 421-495.
Kolmogorov, Andrey Nikolaevich. “Dissipation of energy in locally isotropic turbulence.” Akademiia Nauk SSSR Doklady. Vol. 32. 1941.
Grinstein, Fernando F., Len G. Margolin, and William J. Rider, eds. Implicit large eddy simulation: computing turbulent fluid dynamics. Cambridge university press, 2007.