To avoid criticism say nothing, do nothing, be nothing.

Aristotle

Whenever I get a review of a paper or proposal, I feel some deep sense of trepidation upon receiving it. The first time I read one; the sense of dread permeates my thinking. Why? I like most others don’t like being criticized, and I’ve had some really awful reviews over the years. The worst have been the product of a single anonymous reviewer, who I happen to know the identity of.

If we had no faults we should not take so much pleasure in noting those of others.

François de La Rochefoucauld

My very first “real” research article was never published. I had inadvertently stepped meshSensivityAxisinto the middle of an extremely contentious topic (almost a religious war from what I found out later). Three famous scientists reviewed the paper; one of them identified himself to me via letter (the late Ami Harten). His review was quite cursory, simply “accept, and publish immediately.” I was beside myself with joy. That emotion was short-lived.

It’s easy to attack and destroy an act of creation. It’s a lot more difficult to perform one.

Chuck Palahniuk

The second review was the complete opposite. It was technically supremely well accomplished. As professional as the technical content was, the writing was beautiful, but horribly unprofessional. It was venomous and full of ad hominem attacks. Due to an editorial mistake I found the reviewer’s identity (perhaps it was even semi-intentional).

The pleasure of criticizing takes away from us the pleasure of being moved by some very fine things.

Jean de La Bruyère

A few years later I got another review from the same person. It was stunning in exactly the same way as the first review. I figured out who it was because the figures included in the review were identical to those in an article I was reading. Given this evidence and the style of the review, I knew who it was. I’ve met this person and he is brilliant, and quite friendly and kind in person, yet under the cloak of anonymity he is an awful person. His venom tipped writing blunts the extraordinary technical quality of the work he puts into the review.

I should keep this experience in mind because I review papers all the time. I turned in one review last Thursday and have four more in my queue. To be honest I rarely return reviews of the technical quality of the ones I mention above. I hope and pray that I never match the above-stated level of unprofessional and personal venom toward any author. It should be something that I keep in the back of my mind. A couple of events in the last week put this front and center in my thoughts.

Critics are our friends, they show us our faults.

Benjamin Franklin

Yesterday I was doing a different sort of refereeing, soccer games. I had a couple of very competitive youth games. I also work with a gifted an extremely successful young referee who is half my age, but much more highly ranked. At the end of my match as the main, center referee, he gave me a critique of my performance. Massimo_Busacca,_Referee,_Switzerland_(10)I certainly didn’t have my best game, but it wasn’t bad either. His feedback to me felt extraordinarily harsh, and even a bit personal. He has been reviewed by the very top referees in the United States (i.e., assessed). I took his comments with the utmost seriousness, but still his style made me feel awful, and undermined the effectiveness of the comments.

c4140d578dbefc683e095db035e1-largeThis young man has as you might expect a very serious style, which is keenly reflected in his success and personal style. Plus critiquing someone twice your age can’t be easy. So, I will take his feedback to heart, and make efforts to improve my performance (work on positioning, signaling advantage and style in man-management). In addition I need to reflect upon the dynamic of critique in every part of my life.

A coach is someone who can give correction without causing resentment.

John Wooden

Work provided yet another situation that connects with each of these instances. It also reflects deeply on why V&V so often generates such negativity. I reviewed some very good work in analysis by a couple of younger staff. The issue is that the standard practice in analysis is lacking in a rather critical way in one respect. The key to the problem is that the young staff followed, standard accepted practice, yet failed to examine and recognize a critical source of error. In a sense the real core of the problem lies with their seniors, who have mentored the younger staff and set the standard they follow.

Often those that criticize others reveal what he himself lacks.

Shannon L. Alder

I won’t get into the specifics, but it goes directly at uncertainty quantification and setting quantities of interest. Both issues set the stage for the problem that arose. In doing analysis the quantity of interest is often set by the application and defined by worst-case issues. Unfortunately, these worst cases quantities are horribly behaved numerically. They put the numerical methods under enormous pressure, and show them at their very worst. Uncertainty quantification is a big deal these days, and expected from first-class analysis. The combination of the awful quantity of interest and requisite poor numerical behavior, the analysts have shied away from examining numerical convergence and estimating numerical error.

The trouble with most of us is that we’d rather be ruined by praise than saved by criticism.

Norman Vincent Peale

Instead the standard practice is conducting mesh sensitivity, but not verification analysis. Mesh sensitivity usually looks at the change in a quantity of interest across several meshes. If it is small, then the mesh is assumed to be adequate. This is the heart of the problem. If the convergence rate is low, the mesh sensitivity can be very small, and the numerical error can be very large. In the case I examined this was the case. In the final analysis the numerical error that had been assumed to be small was almost as large as all the other uncertainties. The convergence rate was so low that the numerical uncertainty was about 10 times larger than the mesh sensitivity would have indicated.

I communicated these results to the younger staff, who thanked me, but I fear did nothing else. We have to provide highlights of our work each quarter, and this work was written up as my contribution. Some of these reports get kicked up the chain, and this one was chosen. Ultimately a few of these reports get sent to Washington, and this one made the cut.gr2

Getting chosen is a two-edged sword, it is good to be recognized, but given the critical nature of the work, it hurt feelings too. This is an unintended consequence of being critical. This is the heart of why V&V is so often reviled.

A critic must be knowledgeable in several fields, practices, and mediums. Brushing off art that they personally don’t understand, is not a critique.

Justin K. McFarlane Beau

Advertisements