Proficiency Testing Does Improve Quality

A good friend of mine once made the comment that you don’t get rich running proficiency testing programs.  He was correct.  There have to be other reasons to  get programs started and continued.  For most organizations, the prime motivation is directly associated with a belief and commitment that laboratory testing can and must be the best that it can be for the benefit of patients, clinicians, and laboratorians.  It is for a Commitment to Quality.

I suspect that most laboratorians understand why they are required to participate in proficiency testing.  That does not make it any easier to accept.  In over 30 years I don’t recall many technologists or microbiologists telling me that they look forward to receiving and testing our challenge samples.  But the important thing is that they do it anyways.

Recently we sent out a survey to our participants asking questions about the relationship between proficiency testing and quality management.  One of the questions stated “Laboratories Quality Systems recommend using laboratory testing errors as the foundation for investigating for systemic errors that can impact on broad aspects of their testing routine. Is it your experience that investigating CMPT proficiency testing errors has led to the detection and awareness of systemic errors that affect both proficiency testing results and also clinical testing results?”

As an aside, I will say that this is a good but not perfect question.  While it is focused and specific, the preamble is too long.   

We provided a scale of 6 different answers that varied by degree.  (See the enclosed figure).

To increase participation and reliability we follow our rules that all surveys are anonymous and optional, with no required responses.  And generally we accept the responses in the first 2 weeks.  If we get a response rate of 20 percent or greater we don’t bother with additional prompts.  

The responses to the question were both enlightening and encouraging.  More than seventy-five percent of laboratories reported back to us that when they get proficiency testing challenge errors they find that the investigation of that PT error leads to finding and amending system problems.  Usually these were minor issues that were addressed with procedure tweaks, but sometimes the issues were significant.  Only about 13 percent said that they found no reason to follow up a PT with a peak at their system to ensure that it was working properly. 

The balance (who responded with “other and a comment”) were all laboratories that had made some form of system follow-up. 

When “other and comment” were combined with the direct responders, almost 90 percent of laboratories reported that having a PT error lead to some form of larger investigation which lead to some level of system improvement. 

That is a very good finding, and shows that the crafters of laboratory Quality were right when they required laboratories to participate in proficiency testing because error improvement contributes to better patient and customer care.

Many laboratories continue to overwork PT samples, because they view PT as a test rather than as a Quality measure.  But overworking samples can hide or minimize system errors.  Some of those that noted changes that needed small tweaks only may have missed opportunities to discover and correct larger issues.

It appears there are a small number of laboratories that may need some help in seeing the value of error detection and aware through PT.  While errors can be PT specific, it is more likely, if the laboratories used their routine procedures that PT errors may have their foundation in the laboratory’s normal structure and function, and that taking a look may be worthwhile.  These are fully lost opportunities.

Proficiency Testing is too valuable.  Rare is the event in live clinical testing that a laboratory gets near immediate feedback when they make a mistake, except perhaps in the domain of sexually transmitted infections.  The unfortunate reality is that in the vast majority of time the laboratory has no mechanism of capturing external failure errors unless there is a penalty involved.  Think about the yet again recent mess in Nova Scotia [see: http://www.medicallaboratoryquality.com/2013/08/transcription-errors-can-maim-and-kill.html ]

So to me there is a bottom line.  This survey provides supportive evidence that participation in proficiency testing aids Laboratory Quality Management.

That is good for laboratories and their clients and is the best reason for us to continue providing medical laboratory proficiency testing programs.

PS: Hope to see you in Vancouver in October.  http://polqm.ca/conference_2013/conference_2013/conference_home.html

New insights on Quality, People, and Culture

Two new documents came across my desk today; both are worth mentioning.  One, “Organizational Culture, Tenure, and Willingness to Participate in Continuous Improvements in Healthcare” by Marco Lam, York and Dan Robertson is the lead article in the July 2012 edition of the ASQ Quality Management Journal.  You can find it at the ASQ site [www.ASQ.org].  The other is a Final Draft International Standard (FDIS) version of a new ISO document ISO 10018:2012 “Quality management –Guidelines on people and involvement and competence”.    


What ties these two documents together is that both focus in the the importance of people within an organization to make or break an effective Quality environment.  


There is lots of literature that describes what happens when people in the organization are not engaged in developing and embracing a Quality Culture.  They either put barriers in the way, or they interfere with the implementation, or worst of all they ignore the whole process.  All these are tactics that interfere with the Quality process, and they can undermine the work environment and can undermine the company.  


Getting people on board with Quality may be one of the most significant initiatives that an organization can undertake.


FDIS 10018 is ISO Technical Committee 176’s offering of an approach to bring people on board with Quality Management, as it is expressed through ISO9001:2008.  First, to be clear, it is not a “normative:” document.  You see a lot of “can” and “should” but no “shall”.  You can translate this as “we think this is a helpful guide that you may want to consider”. 


It is pretty straightforward document and philosophy; it leans heavily on the notions of open communication, teamwork, leadership, and Quality application, innovation and improvement.  It is well written and provides some helpful thoughts that should resonate in the smaller organization.  


This 10018 document is not so much an implementation guide.  There are better documents to help with Quality implementation; consider for example “Canadian Standards Association ISO9001:2008 Essentials”.  This is more of a fine tuning document that provides some guides to help improve the organization that feels that it is not getting as much out of Quality Management as it could or should.


The article by Lam and Robertson is something very different.  It is an example of survey based research designed to answer some specific questions.  
In this case the questions are ones well worth asking:  

• If individuals perceive that their organization is committed to continuous improvement (QI), are they more likely to volunteer to participate in QI that do not perceive the organization is the same way?  
• Are individuals who feel stable and secure (tenure) more likely to volunteer for QI projects that those that feel less secure?
• How do secure and less secure people engage in quality initiatives if they believe the organization is not really interested in Quality?

Understanding how people in a organization feel and act are very important, and it would be wonderful if we had better tools to work with than opinion surveys, but they did come to a few insights. 


First off, people who feel stable and secure are NOT more likely to be the folks that volunteer for QI projects, but people who thought that being stable and secure with the organization was desirable did volunteer more.  
Second, if people feel the organization is committed to Quality they are more likely to volunteer.  
Third, if people feel the organizations approach to Quality is more a “flavour of the day”, workers may still participate, but NOT  to the same extent.  


On the surface these may appear to be pretty mushy conclusions, but from my perspective they support my own experience.  Don’t look to the “old-timers” to come on board with Quality projects; look rather to the folks that are the “up-and-comers” who are the people who want to be tomorrow’s leaders.  And second, you can’t fool up-and-comers; the more committed you are, they more committed they will be.  Try to fool them with flash, they will see through it.  They will participate, learn the skills they want and need and move on.  


And management is left asking themselves, “what happened to all our aspiring up-and-comers?”  


And that is never a good thing.


PS:  At the moment you can not purchase ISO 10018.  For those interested, contact your National Standards agency (ANSI, SCC, BSI, AFNOR, etc) for more information.

Quality and Research Funding

This will seem like a grouse, and it is, in a way.

Today I got a letter about a grant that I did not receive.  I applied for it over 6 months ago, and indeed had long ago decided that our chances for success were trivial at best.  I was already back on the road looking for other funds.  It was kind of a side-issue decision on my part to even bother applying, since expectations were low.  But for those interested in studies in medicine related Quality, the opportunities for funding are few and far between.  If you are dependent upon grant funding for Quality studies you should probably think about changing your field of interest.  Fortunately I have access to other resources.

This fund had enough money for 8 mini-grants, each a small amount of money, barely enough to support a graduate student for a year.  It was for a grant I planned to use to cover some of the expenses for a new graduate student starting in September.  Unfortunately we were not successful, but then nor were 90 percent of the other applicants.  There were over 80 submissions with enough funding for 8 successful applicants.  That is not a grant submission, it is more like buying a lottery ticket.

In a field where 90 percent plus lose, one can not particularly expect to have any real expectation of success, unless there is an inside track.  Frankly, and this is not intended as a total slur, I doubt that with those kinds of odds, the reality of bias and loss of objectivity among the judges comes into play; perhaps  not overtly, but bias nonetheless.  

The process in many ways is similar to my time on the medical student admissions committee.  Looking at grant applications is like looking at admissions applications.  Basically the best you can do is split the group into 3 subgroups, the definite yeses, the maybes, and the definite nos.  Then the politics would begin with each committee person and subgroup pushing to promote their individual and collective favorites.  By the time the smoked had cleared the winners were defined along with the waiting list group and the rejections.  People would move from subgroup to subgroup until the final decisions were made.  Not uncommonly some of the “best by the numbers-definite yeses” would end up on the waiting list, and sometime would get displaced all the way to the “maybe next year” list.

I’m not saying that is what happened here, but it would be hard to work through a large pile of applications any other way.

To make it clear, I am not particularly upset here; every competition has its winners and losers, and the amount of money at stake here was not very big.  And I will have enough to cover the expenses. even without the grant. 

My thoughts go to a different set of issues.  Medical funding has for the longest period of time fallen along traditional tracks of basic research usually within certain subsets based on traditional subjects including sub-specialties.  There is little room within this funding model to address new advances in innovation in Quality.  While there are smaller pools of privately endowed moneys, this too goes primarily to heart disease or cancer or childhood diseases in recognition of donors’ family members.  For Quality research to grow, the pool of research funding dollars dedicated to Innovation and Quality will have to grow, beyond what exists today.

On the positive side, this current pool of money was chased by a veritable herd of interested folks wanting to be involved in Quality Management focused research.  It is encouraging to know that our little group is getting larger and more active.      

With increasing numbers comes influence and perhaps the ability to develop more financial support.  And if not support, then perhaps the opportunity for a  little inside support.

So congratulations to the winners.  Maybe it will be my turn next.

A dismaying visit to yesteryear

I understand the challenges in Quality research, in every definition of the phrase.  In the context of MMLQR I mean that it is difficult to create new knowledge that is relevant to the study of implementation of Quality measures to the medical laboratory.  
Medical laboratories are at a new junction.  To move forward we need knowledge about what is working and what is not.  Are national or international standards creating an environment of reduced laboratory error?   Are Quality initiatives making medical care safer?  Can we draw new insights from past experiences to learn new and better approaches to communicate our message in a more constructive manner.  
We need novel ideas to answer basic questions that affect our laboratories and our role in care.

I commend all those those that try and I congratulate those that create useful contributory information.   I would like to say that I congratulate the authors of an article in the March 2011 edition of ASCP LabMedicine.  But that would not be very accurate.

I was interested in reading the article entitled “Measuring the Application of Quality System Essentials in Vermont Clinical Laboratories” especially in a journal by ASCP.  It was published under the section called “Science”.  And then I saw that it was an evaluation of a survey that was sent out in 2005, near six years ago.  Basically, a survey was sent out to approximately 500 laboratories asking (A) if the respondent was aware if their laboratory had some aspects of Quality in place and (B) if they believed that the measure being assessed was effective and (C ) if they believed the measure contributed to Quality.  Their finding was that supervisors with more than 10 years of experience were more likely say they were aware that there were some Quality Measures in place, and that supervisors were more likely than the non-supervisors to say they believed that having an orientation program was effective and contributed to Quality.  
About half of the supervisors didn’t know if their laboratories developed specifications for selecting purchase or vendors.  

In the academic environment we talk about research being the process of developing “new knowledge” as the path to new concepts, methodologies and understandings.  I know this sounds harsh, but I find it hard to characterize the publication of 6 year old “I think” or “I believe” information that is not verified, or linked to a tangible fact as meeting any of those criteria.  Even if the study was published now so that it could be used as the basis of a repeat study done now to see if there were changes in knowledge and believe, it would be highly suspect, at best.  

The Quality movement in medical laboratories is a rapidly evolving picture.  In the US laboratories are choosing to voluntarily adopt international standards (ISO15189:2008) and are looking at Baldridge as the measure of excellence.  

Information about soft perceptions from 6 years ago does not meet any of the criteria for new knowledge.  For Quality in the medical laboratory to progress we do need new information that contributes to new concepts and better understanding.

Some folks might think that I am being unfair or unduly critical.  But they would be wrong.  To paraphrase a comment that I heard on television a few weeks back, if I agreed with those folks then I would wrong too.