Wednesday, July 21, 2010

Uncertainty of Measurement

Advance notice. This is a rant. I know that some folks don't appreciate rants. That is OK, but this is still a rant.

In science there are two kinds of measurement; hard measurement and soft. Hard measurement is what is needed in astrophysics and nuclear chemistry and engineering where exquisitely small variation has huge impact. Consider the impacts of calculating flights to Mars and being off by 1% or building a bridge and missing a critical weight by a few pounds. Soft measurement is what most of what we do in the medical laboratory. Our values need accuracy and precision and absence of bias, but few of our numbers meet the same demands as in astrophysics. (Maybe blood alcohol levels do because of the legal complications that  surround 0.08 mL/L).

Our values are more about positive and negative, or high and low, or trends. In most situations, if a two-point differentiation is difficult to interpret, then a three-point can be created. Our values are more about contextual interpretation and clinical relevancy. We can tolerate certain inherent insensitivity and non-specificity because we have processes and safe-guards including treatment regimen that can compensate.

In the 1990's international organizations introduced the concepts of measurement uncertainty (see Guide to Expression of Uncertainty of Measurement) which was, and is, of tremendous benefit in the hard measurement disciplines because it creates a method of considering all factors that may influence a measurement and addressing them collectively.

But then discussion started when folks started to think about Uncertainty of Measurement (UM) in the context of the softer measurement areas. Think about a sample in the medical laboratory and all the steps that can have an impact on determining a value. Consider precision, bias, maintenance, timing sequences and sensitivity of equipment. Consider in addition quality, concentration, solubility, and dating of reagents.

How about how the sample was collected, the integrity of the tube and its content (such as EDTA or serum separator gel)?, How about trace or heavy haemolysis?, How about drugs and medications?. How about fasting? How about transport time, temperature, atmosphere? And what about personnel techniques and focus? Yes, yes, and yes.

Can we take all these things into consideration when we do our testing? Of course we should.  Some of them are actually calculateable, and some are not. Some of them are at best, crude guestimates. And what about all the elements that Donald Rumsfeld would call “unknown unknowns”, should one put in a crude guestimate for these as well?

So why the rant? When accreditation bodies decide to make doing these sorts of uncertainty of measurement calculations based on crude guestimates an accreditation requirement it diminishes the notion of quality. It reverts to quality by dictate, rather than quality by principle. It diminishes the credibility of quality laboratorians, and it reinforces that quality is all about (or worse, only about) doing what accreditation bodies say, even when it makes no sense. This does not make laboratories better.

It makes laboratories worse.

PS:  There will be no more rants, at least not today.

For additional reading on UM consider:
An Introduction to Uncertainty in Measurement
Les Kirkup and Bob Frenkel
2006.  University Press, Cambridge.

1 comment:

  1. In 2005 I attended a lecture by Callum G Fraser on biological variation. Being all excited about this new concept I had a discussion with our biochemist about incorporating these “biological variations” into our reference intervals. Unfortunately that did not fly… Five years later the accreditation requirements introduced uncertainty of measurement (UM). Every laboratory in the province of Ontario had to show “knowledge and few examples” of UM for quantitative tests. Strangely enough we use the work of Fraser (Biological Variations: From Principles to Practice 2001 AACC Press) as the basis for our calculation of UM. Unfortunately sometime quality by dictate is the only way to make things happen. We now know that for some biochemistry tests the UM is >100% for high level.


Comments, thoughts...