Code and calculation verification are an overlooked step for assuring the quality of codes and calculations. Perhaps people don’t have enough good reasons to do this work. It can often be labor intensive, frustrating and disappointing. That is a real awful sales pitch! I really believe in doing this sort of work, but the need to provide better reasoning is clear. I’ll get at the core of the issue right away, and then expound on ancillary benefits.
In the final analysis, not doing code verification is simply asking for trouble. Doing code verification well is another matter altogether, but doing it well is not as necessary as simply doing it. Many developers, students, and researchers simply compare the solutions to benchmarks using visual means (i.e., comparing to the benchmark solution in the infamous eyeball norm, or the “viewgraph” norm if its presented). This is better than nothing, but not by much at all. It is certainly easier and more convenient to simply not verify calculations.
Very few of us actually create codes that are free of bugs (in fact I would posit none of us). To not verify is to commit an act of extreme hubris. Nevertheless, admitting one’s intrinsic fallibility is difficult; dealing with the impact of one’s fallibility is inevitable.
So, without further philosophizing, here is my list:
- Don’t assert that your code is correct; prove it’s correct. This is the scientific method; respect it, and apply it appropriately. Treat a code as you would an experiment, and apply many of the same procedures and measures to ensure quality results.
- Mistakes found later in the code development process are harder and more expensive to fix. There is vast evidence of this and I recommend reading the material on the Capability Maturity Model for Software, or better yet Steve McConnell’s book, “Code Complete,” which is a masterpiece.
- Once completed (and you aren’t ever really done), you will be confident in how your code will preform. You will be confident that you’ve done things correctly. You can project this confidence to others and the results you and your handiwork produce.
- You will find the problems with your code instead of someone else like a code customer or user. As much as we dislike finding problems, some one else finding our problems is more painful.
- Verification will allow you to firmly establish the accuracy properties of your method if you look at error and convergence rates. You might have to confront the theory behind you method and problem, and this might help you learn something. All of this is really good for you.
- Doing the same thing with your calculations will allow you to understand the error associated with solving the equation approximately. Again, confront the available theory, or its lack of availability. It will provide you much needed humility.
- It is embarrassing when a numerical error is influencing or hiding a model deficiency. Worse yet, it is badly conducted science and engineering. Don’t be responsible for more embarrassments.
- When you are calibrating your model (admit it, you do it!), you might just be calibrating the model to the numerical error. You want to model physics, not truncation error, right?
- Verification results will force you to confront really deep questions that will ultimately make you a better scientist or engineer. Science is about asking good questions, and verification is about asking good numerical questions.
- You are a professional, right? Doing verification is part of due diligence, it is being the best you can be. Adopting a personal quality mentality is important in one’s development. If you are in the business of numerical solutions, verification is a key part of the quality arsenal.
- You won’t really understand your code until you look really hard at the results, and verification helps you understand the details you are examining. You will looks at your work more deeply than before and that is a good thing.
- Conducting real error analysis can help you make sure and prove that your mesh is adequate for the problem you are solving. Just because a mesh looks good, or looks like the thing you’re simulating isn’t evidence that it actually allows you to simulate that thing.
- It is 2013, so come on! Please do things in a way that reflects a modern view of how to do computational science.