Insanity is everyone expecting you not to fall apart when you find out everything you believed in was a lie.

When you are doing a single calculation a couple of things can happen; the calculation can run to completion and give you an answer, or the calculation can bomb. In reality we have four distinct possibilities:

1. You will converge to the correct solution
2. You will converge to a solution, but not the correct one
3. You get a solution, but don’t converge
4. No solution, unstable (code bombs).

This is really a continuum of solutions with varying degrees of each of these four categories. For example there might be a big difference between solutions that are wandering in phase space as opposed to one that is bouncing between several possible equilibrium points.

One day everything will be well, that is our hope. Everything’s fine today, that is our illusion

Too often we simply see a single calculation be the end product. Determining what category your calculation is actually in has great bearing on the credibility of the calculation. In other words the credibility of a single isolated calculation is questionable a priori. You really need to be offering ensembles of calculations that together define the credibility rather than a single calculation.

The point is that if your calculation runs to completion it could be in one of three states. You should be quite interested in which one. Verification is the vehicle to help you figure this out. You can really only start to sleuth out what state the calculation is in by doing a sequence of nearby calculations using variations in the models and methods. If you run a family of problems you will generally find that the easier problems converge to the exact solution without problems (assuming you have an exact solution). You can only distinguish between 1 & 2 when you have an exact solution.

As you raise the problem difficulty either by varying the parameters or geometry or materials, you will gradually get solutions that depart of the exact one, but converge to some solution. As the conditions for the problem become more extreme, the code will stop converging to a single answer, and ultimately encounter stability problems. For more complicated problems especially in multiple dimensions one may always be looking at problems that don’t converge to a single solution, nonetheless you should be asking these questions all the time.