This week I’m traveling all week (and next week too). Traveling is a mixed bag; I’m seeing new things, and exposed to new ideas; I’m away from home and family. It’s a yin and yang sort of thing. I love it, and I hate it. Today, I’ll focus on the good part, the part that keeps my brain humming along. Without mixing up whom you interact with your ideas may become stale, or fail to consider issues that come from a differing perspective. You also limit your ability to contribute to wider debates, and learn about new ideas developed elsewhere. You need it for intellectual vibrancy.


First, I went to visit the University of Notre Dame where DOE has funded a center to study a really cool material processing idea. The physical idea itself is fascinating involving reacting shock waves and the production of exotic materials. At the university they are combining disciplines to design and predict the experimental results. The process has never been executed before, but they believe it can be if they get everything right. They are using computational physics, multiscale modeling and advanced computing ideas along with the laboratory work. The key is engagement across areas that often don’t mix in an academic setting. This mixing is itself a challenge at a university where the tendency is for work to be done in narrow-deep silos of knowledge. I’m there leading a team of National Lab scientists who are there to help and advise the Center as well as make sure the work stays aligned with things the Lab values. Key among those values is multi-disciplinary work requiring disparate skills for success. This part was “easy” since it was the first two days of the two-week trip, and full of cool mind-expanding ideas. The hard part was having to “on” for the entirety of the meeting as the chairman of the committee.


The last three days have been spent at the third ASME V&V Symposium. The meeting has a real sense of being important to the V&V community. There seems to be a nexus at hand where the field will either grow and flourish, or decay and die. Opportunity lies to the right, and danger lies to the left; or is it the other way around. Decisions are being made that are important such as whether to have a V&V journal; how V&V figures into the regulation of medical devices and other aspects of the biological sciences. The danger with a Journal is that it becomes a cul-de-sac where all V&V work is done, and offers an excuse for not applying V&V in application areas. The distinction is important, and the spectrum of detailed V&V work needs to run from the development of methodology, to its application in purely applied work. For V&V to flourish, it needs both theoretical work and committed application in domain science and engineering. Regulatory work offers similar, but different challenges. The tension exists between strong regulation of action where people do things “right” and too much rigidity to allow for innovations in scientific methods to continue to benefit the quality of work.

For me, there was a big event. I was giving a plenary talk the second morning of the meeting. My talk’s abstract was published on the blog earlier, “What kind of person does V&V?” and in retrospect it drew attention appropriately to the talk. It also raised expectations. I had to deliver. Part of the talk was a desire to work on my own approach to giving a talk. In other words, I was pushing myself. I integrated elements of “Presentation Zen” and TED talks into the presentation. These elements meant using more images and fewer words while focusing on stories and free flowing narratives to give the message. My intent was to provoke thought and self-reflection in the V&V community. The talk was crafted into five narrative arcs some of which have been posted here. The first of these was the analogies between V&V and Human Resources, next I spoke about the danger of technology being too easy or simple (V&V is technology), the third is the use of “you idiot” appended to questions to screen bad questions out, followed by imploring V&V to act more as coaches and less as referees.

The final story revolved around the ongoing revolution in computing and data science coupled with the end of Moore’s law. V&V is needed to help manage the transition in computational science that will occur in the next decade. Without V&V computational science might be lost or go seriously off the rails. Without Moore’s law in effect we no longer have bigger, faster computers to simply crush problems with resolution. Instead, we will have to rely upon being smarter and improving methods, but new methods produce different answers, and V&V is necessary to build trust and confidence in the new methodology. Furthermore, the direction that computing is going offers the possibility to leverage the technology in a myriad of creative ways. V&V is core to success in many of these opportunities.

I felt very good at the end, and the objectives of the talk seemed to be generally achieved. I was imploring V&V to be collaborative, flexible and emotionally intelligent. As I’ve discovered if you give a good talk you’ll get questions and comments. People will want to talk to you. I received both in spades.

I’ll draw to a close with few observations about the meeting. The topic of V&V is quickly maturing. The practice of V&V is improving across the board. More and more talks are getting at subtleties in the practice rather than the basics. This is clearest in the new biological science use of V&V and particular medical device modeling. The quality of the work is much better than the previous year, and in fact the pace of improvement is astounding especially compared to the physical sciences that birthed V&V. A deeper concern is the penetration of V&V into the application sciences. Is V&V being done better and more extensively where it is needed in engineering practice? Or key scientific endeavors such as climate science. Climate science is politically charged and needs V&V, but the sensitivity to criticism acts to effectively poison any V&V despite the magnitude of the need. Pat Roache compared the discussions the climate community has about quality to a couple having a hushed argument behind closed doors fearing their children might overhear. There is a distinct lack of domain science expertise at the meeting, and that is a major concern.

I will also mention the extensive use of commercial codes as a concern. The verification work on many of these codes is not up to standards. I can’t see it doesn’t exist, I just haven’t seen it. As many have noted V&V is evidence based, and the evidence isn’t there. The use of commercial codes for CFD is extensive. Laboratory and government built or company internal software is now dominated by commercially sold CFD codes. The actual quality of these codes is difficult to understand, and they aren’t very open to discussing their “secrets”. In the end we need better transparency so that the solutions they create can be trusted. Like many applied codes the “robustness” of the code takes precedent over accuracy. As such low order numerical methods are highly favored. Another concern is the utilization of antiquated and simple methods for multiphysics coupling.

The good thing about the meeting was seeing lots of old friends, great conversations and a whole lot to think about when I get home. People who manage science federally and have restricted our attendance at meetings fail to understand what is important about conferences. Yes, we present our work to our peers, and our peers give us feedback, but much more happens. First and foremost, we see the work our peers are doing. We engage people socially, and we laugh, argue and eat together. The social aspects of science are critical to a well-functioning activity. Conferences are essential to the conduct of science because they allow people to interact as people. Presenting a paper at a conference is only one small aspect of a much broader engagement.

Next week I’m in France and seeing what the application folks in high energy density physics are up to. I have my hopes and have my fears.