Risk Benefit analysis in the Medical Laboratory

A few years back I was involved in the writing of an ISO technical specification on the role of Risk Management in the medical laboratory.  The thesis of the TS was that managing Risk interfaced with Quality Management principally through Preventive Action.  By examining for potential events before they occur, adverse events can be prevented or at least contained.  I still think that is true.  Although the revision of the document will point out a much broader view of what constitutes Risk in the medical laboratory, the core of the link will be through preventive action.

One of the major tools that is used in Risk Assessment is an Occurrence – Outcome grid, where a potential event is either qualitatively or semi-quantitatively assessed from never occurring (0)  through to always occurring (100) .  The contrasting concern assesses outcome from never significant (0)   through to potentially fatal (100).  The point of the exercise is to determine the controls and the degree of risk aversion or avoidance or tolerance need to be considered and applied with regard to the event. This sort of tool can be used in a variety of situations that could potentially have an adverse effect on the patient, the clinicians, the staff, the laboratory or the community.

An important characteristic and perspective of that approach is its singular view that outcome falls on a negative gradation, which starts at neutral (no impact) and goes to bad and then to worse.

 

But as I read and think about risk and the medical laboratory I see one variance between the traditional view of medical laboratory risk and risk literature.  In risk theory and in its applications to finance, to innovation, and to discovery, there is an also a positive side to risk, which is on the risk-reward side.

 

Consider in the world of gambling and investment.  One looks at potential outcome as either leading to profit or loss, and weighs the balance of circumstance and evidence and comes to a decision.  Either the potential benefit is worth putting money on the line or it is not, and through to a certain point, the greater the potential benefit, then the more that the person is prepared to risk.  At a certain point either the opportunity for benefit is either so low or the likelihood of major success is so small that the exceeding a level of risk is no longer rational or appropriate.  Consider the common phrase: if a deal seems too good to be true, then it probably is. 

And so I have to ask, is there a place in the medical laboratory where Risk and Quality come together where the potential positive side of Risk and Benefit comes to play?  It seems intuitive that the right answer is clearly “no”.

 

Many would agree that over the last many years hospitals have had policies that say that the only level of acceptable risk is risk aversion, where there is absolutely no tolerance for any level of risk.  An outcome, even with a frequency of 1:10,000,000 is unacceptable.  That clearly has not been a workable or financially viable policy.

 

What makes more sense is to make a measured, objective calculation of tolerable risk where patients are protected, perhaps to an arbitrary risk of perhaps 1:1,000,000. 

But to get back to the question at hand, the health care system, and the laboratory in specific cannot and must not make risk-reward decisions that provide benefit to the hospital or the laboratory or the laboratorians that may put patients at risk of harm which would include the risk of faulty diagnosis.

Similarly in a public pay health care system, one should similarly argue that the hospital should not be making decisions that put public money at risk for loss if the benefits accrue only to the health community.  We see that often enough in the news when people get caught.  The newspaper calls that abuse of public office, and people sometimes can go to jail.  (In Canada we have seen a really good example of this in area of building Electronic Medical Records systems.)

But let me argue that there are situations, such as implementing green energy systems, or error tracking systems, or high efficiency operating rooms to allow for more laparoscopic or other accepted minimal invasive surgical techniques that have the potential of resulting in better or improved service. 

Given an option I prefer this to be part of the private care system when the owners and decision makers carry the liability of poor outcome, but the reality is that at a certain point in the public system, there needs to still be an institution that has to go first. 

I think that the point that I am trying to make is that in my experience, many institutions make arguments more by impression than by calculation of measured risk.  Measuring risk in most situations is an exercise to reduce error and negative outcome.  But sometimes, risk can be a positive concept where the point of the laboratory and the institution can benefit from positive reward.   

A standard on medical laboratory risk needs to recognize and address both sides of the topic, 

Innovation and the Medical Laboratory - part 2

Innovation and the Medical Laboratory – Part 2

In the early 19^th Century (around 1811-12) a group of textile (artisans) workers became concerned that the use of mechanized looms were going to interfere with their artistry and their profession.  Their employers were using fewer people with lesser capability.  They saw this as an affront to their talent and skill.  Their response was to damage the looms.  

This act of protest was interpreted as a public statement about the broad changes associated with the Industrial revolution.  The British government response was to make an equally public statement about the hazards of interfering with industrial change and either shipped the perpetrators off to Australia or had them executed.  The group (henceforth called a movement) was tagged with the label Luddites (named after another loom destroyer on an earlier age).  

Over the years the term Luddite has shifted from one identifying folks involved in job-protection to a more general concept of resistance to change, with particular reference to resistance to technological change.

I am not a Luddite, in the modern sense.  I am by-and-large an “early-adopter” of new technologies.  But I also watched the impact of a lot of new technologies have done to the medical laboratory. 

My first connections with the medical laboratory were in the late 1960’s when most tests were done manually and with next to no standardization or quality monitoring (read the works of  F.W. Sunderman on Proficiency Testing in the 1940’s and 50’s).  In 1957 the Autoanalyzer was invented and let to the introduction of equipment into the medical laboratory shortly after (Technicon, SMA, SMAC) all of which created a platform that ultimately lead to more consistency in testing, but also changed test order practices with more patients having more tests more frequently.  

Changes in microbiology followed suit, about 10 years after chemistry with early equipment as the Sensititre, Radiometric Bactec.   The laboratory business exploded with activity. 

More recently (and by that I mean very recently) Anatomic Pathology is now going through a dimension of change that will have profound effects.  As we move towards increased automated digital pathology there will be higher consistency in performance with broader distribution of knowledge and a striking decrease in the number of AP pathologists required to do what they do now.  A single room in virtually any city will be able to service the interpretive needs for thousands, if not millions of patients around the world.  Maybe not today, but soon.

With each generation of change came increased capability, increased speed, and from a strictly technical perspective better analytic sensitivity, more sophisticated data handling, and some levels of data management. 

Unfortunately a lot of this progress has had its dark side as well: more and more liberal testing patterns with inappropriate testing of patients both in terms of poor patient and test selection.  Increased numbers of laboratories using fewer and fewer staff with decreasing technical training.  [Note: The Luddites of 200 years ago were actually visionaries].  And all that has led to substantial levels of laboratory errors and continuing challenges to laboratory Quality.  Not to be excessive, but one can argue that many of the problems that we have today would be avoided if we had a better control on the frequency of testing and they procedures requested.  [As I hear it, this may not be too far off, as organizations such as the US Congress and HHS look at restructuring the fees for laboratory testing.]

My point is this.  Medical laboratories have a long and distinguished history of innovative technical automation change that has improved the lives of many through better and more available diagnostics and monitoring.  It would probably be a fair and honest statement that innovative change has been the single-most powerful driver in medical laboratory.  

But it has occurred all too frequently within a technology and enterprise silo, and in many regards absent from critical planning such as human resources training and management, quality management, and risk. 

A theoretical calculation:  if introduction of technical change had been more effectively controlled and we were now doing fewer but better selected tests, would we be further ahead, about even, or behind if we consider the impacts of proper diagnosis, along with the impacts of no diagnosis and also the number of people that are harmed or inconvenienced by excessive inappropriate and uninterpretable testing.

As I work through my awakening to the impacts of innovation and quality on a personal level, I have come to two understandings.  First, being a cautious Luddite is  maybe not such a bad thing, and second, that Pogo might have been more right than we would want to acknowledge:  "We have met the enemy and he is us."

I'll have a Grande Latte with that Proficiency Testing, please.

I have mentioned before my views on the close associations between Quality and Leadership and Innovation.  The other day I was watching television in a mindless sort of way, when I got to see a telecast of Howard Schultz, chairman, president and CEO of Starbucks giving a presentation at the Vancouver Board of Trade.

Now I know that Starbucks coffee is not for everyone, but one would be hard pressed to not recognize the company as pretty much a world wide success.  Recently I was in Ankara Turkey and still had my daily double espresso. 

Schultz has an interesting story with Starbucks because he built it up the first time, and left the company when it decided to chase growth over quality, and then was asked to return when the company was going sour in a hurry.  Three years after his return, the company is back on top.

During this presentation he talked about several key issues.  The first was that successful companies are driven first from their core principles.  Second, that human resources have to be at the center of decision making.  Successful companies ensure that their staff are front-of-mind.  Third, that in the “eco-system of enterprise”  companies that give back to customers succeed.  Not giving back in the sense of kick-backs, but in the sense of being a contributor to education, outreach, leadership and support. 

And fourth that innovation that is visible to and beneficial for the customer is critical.  Companies ignore the opportunities of innovation at their peril. 

Customers are no longer interested ONLY in price.  Customers are more comfortable when they perceive that they share your values and see you as part of a “thread of trust”.

Now I understand that most of this is way outside the scope and authorities (but hopefully not the interests) of the public sector institution based medical laboratory.  The drivers of decision making in the public sector institution may not see themselves in the same way as Starbucks does.  But it seems to me that the core principles are pretty central to every successful organization.

And for Quality Partners including Standards Development Organizations, and Accreditation Bodies, and Proficiency Testing Providers, and Reagent/Equipment Suppliers, and to some degree, Quality Consultants, the iteration of principles has to be absolutely “spot-on” advice.

Organizations need to know what they are good at and what their core values are.  If they don’t know, then how can customers know what to expect. 

And trust is critical.  To the extent that your customers have control, they will stick with the folks that they trust.  But watch out when the thread of trust is broken.  When you are perceived as only about growth and money and serving your own interests only, and not about providing support, then problems will blossom.

And then comes innovation that is visible and relevant to the customer.  Are we writing standards that matter?  Are our proficiency testing samples a boon to quality improvement and to today’s needs for continuing education?  Along with reagents and supplies, do we provide the element of real innovation that is meaningful to the laboratory?

Are you a value-add, or just the same old-same old?

CMPT spends a lot of time, effort, energy, and money one these very issues for year.  We like to think we are successful, although not quite to the same level as Starbucks. 

If you visit www.CMPT.ca you can see our latest Annual Report for 2010-2011

SWOT Priority Table for Medical Laboratories.

So I was thinking about SWOT analysis and came up with some ideas about a tool that one could develop to help the process of translating SWOT analysis information into a priority list to get the important repairative  tasks addressed first.
The attached is based on the following assumptions or principles.

1. Laboratories with more than one area of weakness, or threat, or areas with opportunities may seek help prioritizing tasks.
2. While all Quality procedures are equal, some are more equal than others.  (I can this the Animal Farm principle).  For example, if there are improvements to be made both in Management Review and also in updating the organizational chart, it is more important to focus on management review.
3. If a procedure is at an acceptable level, it does not need any work (other than maintenance), but if there are tasks that need to be done, those at a level of severe deficit (threat) they need to be addressed first.  
4. Those where there are weaknesses or opportunities come next.  

So here is how this works.

• In the left hand column I have listed many of the areas and procedures  that a laboratory doing internal review should evaluate.
• Each area should be considered as either being in a healthy condition (strength) or having a weakness.  The weakness may be bad enough to be a potential liability.  Or there may be resources available to address certain areas (opportunities).  It is conceivable that a area could be both a threat and have an opportunity at the same time.  
• For each procedure I have put a "1" in the Strength column.  If after evaluation you want to change this to any value from 0 to 1 to 1 decimal point.  You can also add a value (0 - 1) in any or all of the other columns.  
• The four columns can add to 1.0 or greater. 
• The more you make the line worth, the greater will be its priority.
• Different procedures have different inherent procedure priority levels (PPV).  The PPVs  are the one’s estimated by me.  If you think different values could or should be used, change them.  Again, the greater the value, the greater will be its priority.

http://dl.dropbox.com/u/173944/SWOT%20PriorityQD-0511.xls
 
Note: I have verified that this file works properly.  
Also note I have not  validated it as giving the best priority for addressing tasks.  
 
Feel free to use it, or experiment with it, or disregard.

Interested in your thoughts.

One size does not fit all.

When I am doing project and lecture preparation, I tend to go back to basics; more times than not that means Deming and Crosby, two men with a common goal, but different approaches.  One place where they differ 180 degrees is in the area of error reduction and prevention.  Crosby promoted the principle that setting a goal of zero tolerance for error was the most effective way to prevent costs and consequence.  Do it RIGHT the first time (DIRFT).    Deming thought this was not only nonsense, but indeed dangerous and destructive nonsense, that would be used as a stick to punish and blame workers when things went wrong. 
 Forty years after the two were busy establishing the fundamentals of business Quality, the truth still lies somewhere in between.  

But as I was preparing for a presentation, I came across an now interesting 2007 article entitled ``Trying to Do It Right the First Time Isn't Always Best``.  Hooray for Google (see http://www.chacocanyon.com/pointlookout/070314.shtml) .  The author, a Quality consultant Rick Brenner, is rather dismissive of the concept of DIRFT (``These slogans might make some sense in the operational context, where tasks are very repeatable.``) , and then makes the points that an important aspect of project management and learning in general is to have the opportunity to do it wrong the first time and that risk taking is about giving permission to allow doing things wrong at first, if the consequence is that you might by accident or intent find a  better solution.

Well I can see where he is coming from.  In the research and development setting, we are looking for new insights, new knowledge, and novel approaches.  That is why we call it ``search and re-search``.  But as much as the medical laboratory is about new information, it is about correct information.  The right information from the right patient sent to the right clinician at the right time.  Everytime.  While  Brenner describes this as being merely operational, but it is pretty narrow-minded when describing this as repeatable as a simple repeatable task. 
 Clearly while being a consultant in one very narrow field, he has little interest or experience or expertise in the broader laboratory world.  
And can we agree that might not be the strategy of choice when building nuclear reactors!

From my perspective, in the medical laboratory, establishing DIRFT is not so much an imperative, but more a target and  attainable  goal, and the only way that we can move towards error reduction and reduced clinical consequence.

I recognize that in the busy laboratory there are many distractions that contribute to slips, but if information is delayed or faulty, the clinician and patient are not  interested in our problems.  They want what they want and they want it when they want it.   Period.

In 1986 Briggs Phillips published in an article entitled "Human Factors in Microbiological Laboratory Accidents" in Laboratory Safety: Practices and Principles.  Laboratory accidents, injuries and infections do not occur randomly;  most happen around a small group of people with repeat problems.  By case-control comparison to accident-free peers, accident involved people were less aware of work environment risks, Less aware of work behaviour risks, and were more prone to working quickly, especially before lunch and breaks and days end.  Accident involved people were more likely to take risks.

There is reason to suspect that similar issues are at play when it comes to Quality slips.  Most slips and errors occur in the laboratory in a similar fashion, a small group of workers represent the largest group of repeaters slippers.  I am not aware of any study that has looked at behaviors of this group, but it would not surprise me if we found a pattern similar to one described above.   Wouldn’t it be great if that type of information was available!

When we don’t don’t have an policy or a mechanism to deal with the repeaters, then we are only apt to find ourselves in the same place over and over again.  That doesn’t mean trashing the person.  Maybe  some investigation into causation and responses to distractions is in order, or some  re-training assistance may be  required.  

Regardless, at some point management needs to have a solution, and it seems to me that having accepted a DIRFT policy and principle to aim towareds is not a bad place to start.  

Quality Education: what's it good for?

We are putting together a Master’s in Laboratory Quality Management which we expect will be available both in an on-line option and an on-site option.  The on-line will primarily be for folks who are working and intend to continue to work.  The on-site will be for folks who want to take some time away and spend some time doing primary research in medical laboratory quality.  I will continue to write on the subject as the program becomes ready for prime-time.

In the process of putting the proposal together, we did a survey of the folks who participated in the UBC Certificate Course.  Rather than invite and re-invite folks to respond to the survey we made the decision to take the responders from the first pass.  Both approaches can be equally useful, as long as you take methodology  into consideration as you interpret the results.

So after 5 days we gathered about 25 percent of course attenders.  The distribution of folks from across British Columbia and across Canada, and from outside Canada were essentially the same as the total group.  In addition the responders  equally included participants from the early years, the middle years, and the recent years, and few from the current group.   As we did not put any pressure on this group we can interpret this as a reasonably representative group.

Of interest to us was that about 90% belief there is a market for people with a Master’s degree today, and that the same percentage see that the market will be much stronger over the next 5-10 years.  That is what I believe too.

One finding that interested me, and disappointed me at the same time, was that only about 65% of responders currently are holding positions involved directly  in Quality.  We didn’t ask how many had held quality positions but had changed jobs, in large part because that was not the primary point of the survey  But what was interesting was that folks who had taken the course but were not actively engaged in Quality positions still believed there are opportunities out there, and that the market will continue to get stronger.

Which all got me to thinking; there are all sorts of positions for people interested in and trained in Quality.  The most obvious position is as part of a quality team or a part of laboratory  management.  But there are lots of other places as well.  There are positions with all the quality partners, like the proficiency testing group and the accreditation bodies.  There are positions within public or civil service, especially within the Ministries (or Departments) of Health and the health authorities, where knowledge of laboratory quality is important.  There are positions with service and equipment and reagent suppliers who need to be aware of validation, verification, stability, and measurement uncertainty.  And there are consultant positions, especially as we in Canada move more and more towards official inquiries for laboratory error.  And importantly there are increasing opportunities within education with the schools of technology, as well as the medical schools.

Bottom line appears to be that Quality in health and in particular the medical laboratory is a growth opportunity; today and tomorrow, and from the folks who responded to our survey, for the next 5-10 years plus.

Resident Quality Seminars as Adult Education

A few weeks ago I wrote (See “Communicating Quality and the Principles of Adult Education - January 30th 2011) that adults learn what they want, and  how they want and accept the knowledge if it makes sense and is consistent with what we already know.  The educator needed to be seen to be organized and pragmatic and relevant and interesting.  If adults are motivated they learn better.  

So with this in mind, can I make any objective comments about my own teaching capabilities?

I got back the results from my Resident Seminar Series.  
 
Background 
To summarize, I put on a seminar series for anatomic and general pathology residents.  The series begins the week after New Years and my sessions start at 08:00 AM.  A draw to the academic half day program is that they get a free breakfast when they attend.  
Before I start the seminar series  I create an anonymous on-line survey that includes a series of 10 questions on quality facts, along with some demographic information.  I then present 7 hours of seminars over 4 sessions and give 1 hour to a guest presenter, and after that asked the group to respond to a second on-line survey which asked the same questions as before and added some additional comments on the series based on a 7 point Likert ranked scale with a maximum value of 6.0

Attendance
The pre-seminar survey had 25 responders.  
On average there were 29-31 attenders to the seminar series.  Nearly everyone attended all 4 sessions.
The post-seminar survey had 14 responders.

Findings
So my first point.  Adult learners do what they want and respond to surveys if there is something on the table.  After the series we were able to get 14 responders (despite repeated requests).  Not getting a better response rate in the post-survey was annoying, but it is what is is.  

My second point:  There was improvement in the responses to all the fact based questions.  There was significant improvement (yates corrected p<0.05) in 2 questions.

My third point:  The responders found the information relevant to their career (5.5/6.0)  informative (5.36/6.0), relevant to their residency (5.29/6/0), and relevant to preparation for examinations (5.07/6.0)  The weakest response was when the group found the information interesting (5.0/6.0).  

Interpretation.  
Overall I can this a success.  Not an overwhelming success, but a success none the less.  

What was encouraging was that after the series, one resident  approached the hospital quality manager to get engaged in a project, and one resident approached me on a similar topic.

The increase in knowledge performance was ojectively positive and in part significantly so.  And there is a powerful opinion and recognition that knowing something about quality is important in a laboratory career.

Discussion and Conclusion
The objective measures point to the seminar series as a measurable success with opportunities for improvement.

Teaching my on;line course is the ideal adult education situation.  The participants seek us out because they want the information.  There is money on the table.  They feel free to participate actively.
With residents, they are present because they feel required to be there, even if they don’t care.  At worst they still get a free breakfast.  There is no money on the table.  They feel free to drop out if they are not being entertained.   

Note to myself:  
Look forward to doing the seminar series again, maybe next year, more likely in 2 years.  
Make some changes to beef up interest.  
And importantly don’t take the weak response rate too personally.  If they really thought the series stunk they probably would have taken full advantage to let me know.  

PS:
The information will be presented more completely and formally during the POLQM Quality Weekend Workshop in June.  There will be an education breakout session.