Showing posts with label Laboratory Performance. Show all posts
Showing posts with label Laboratory Performance. Show all posts

Thursday, September 19, 2013

Laboratory Proficiency: Closing the Gap?



Laboratory Proficiency: Closing the Gap?

I have written on the subject of proficiency testing as a measure of laboratory quality before, especially on the consistent reality that large laboratories have consistently and progressively out performed small laboratories for the last decade.  [see:  http://www.medicallaboratoryquality.com/2012/10/laboratory-size-complexity-and.html ]. 
In our program this was not always the case, and indeed in the early years, based upon our program structure, all the laboratories performed equally on their category appropriate challenges.
Note:  In our program, we have an approach that works on the premise that large complex university based or similar laboratories often receive complex samples from complex patients, while smaller outpost laboratories do not receive as many complex patients and as a result tend to not see as many complex samples.  Thus we send complex university based laboratories more complex proficiency testing challenges, and send more basic samples to the outpost laboratories. 
Note within the note:  We know this is not always true.  These days people on dialysis or on cancer chemotherapy drugs can live almost everywhere.

Over the last 10 years. In direct correlation with a national drive to reduce laboratory access through closures, we have seen a consistent drop in the quality of laboratory proficiency in all microbiology laboratories except the largest and most complex.  The large laboratories primarily found in large urban centres, and often associated with university teaching hospitals have consistently maintained a proficiency well above 90 percent.  On the other hand, the smaller laboratories have not fared as well, As the size and complexity lowers, so does their proficiency.  This is magnified because we do send them the less complex challenges, and some have considerable difficulty, with a mean proficiency much closer to 70 percent.  We argue that this is not about finding fault, other than of the ministries of health who under-funded and under-supported these laboratories with fewer (or none) opportunities for continual improvement or refresher programs, and reduced their supervision by microbiologists to a near non-existent level.  Given the choice between supporting these laboratories or killing them off, the ministries have strongly moved to the latter. 
This may be effective use of resources, but the physicians and patients who live in these areas have not been getting much of a fair shake. 
But this year, it appears that something has changed; for reasons unclear there has been a substantial improvement in the proficiency testing scores of the smallest laboratories. I am not sure why.
I know it is not an arithmetic error.  I know it is not a knowing shift in the degree of difficulty in the challenges. 
It may be a geographic shift in the sense that the number of C1 laboratories has increased and that most of the laboratories are in one province, and many of them are laboratories recently downgraded by their province.  Maybe more of these laboratories have a higher level of local support 


At the moment I cannot say if the improvement is a transient bleep or a heralding of better laboratory performance yet to come.
Only time will tell.
Fingers crossed.

M

Wednesday, July 24, 2013

Laboratory Quality: words or rhetoric?

Laboratory Quality: words or rhetoric?

This month the Infectious Diseases Society of America published a new article (A Guide to Utilization of the Microbiology Laboratory for Diagnosis of Infectious Diseases: 2013 Recommendations by the Infectious Diseases Society of America (IDSA) and the American Society for Microbiology (ASM)) in the journal Clinical Infectious Diseases. It is an impressive document and for those interested in laboratory medicine, and clinical microbiology and laboratory diagnosis to assist infectious diseases, it is an impressive document.  It is about 100 pages long and includes 250 references, so it would be fair to call it comprehensive. 
It was an impressive task to compile all this information and put it together in a single comprehensive manuscript.  The authors should be congratulated. 
Unfortunately, despite all these positives and kudos, it not only misses the bull’s eye, in my opinion it by-and-large misses the whole target. 
The shame of the matter, is that the authors are all very well know and well respected microbiologists in the clinical laboratory community; and they know what the target is, or at least should be.  They say right up front “Physicians need confidence that the results provided by the microbiology laboratory are accurate, significant, and clinically relevant. Anything less is below the community standard of care.” And they are bang-on.  I agree entirely. 
So what is their plan to ensure that laboratory results meet those criteria; their answer is that “all microbiology specimens be properly selected, collected, and transported to optimize analysis and interpretation.”  The problem is that this is just far enough off to be misleading and misguided. 
Don’t get me wrong, I think that collection and transport of clinical samples is important, indeed very important.  And if you want a document to help with clinical sample collection and transport, you can consider this manuscript, or better, look at the Canadian Standards Association document “Z316.7-12 - Primary sample collection facilities and medical laboratories - Patient safety and quality of care - Requirements for collecting, transporting, and storing samples”, which I think is eminently readable and applicable. 
Where we diverge from the value of the IDSA article, is that even if the sample is collected and transported properly, that does not ensure that it will be processed properly or accurately, or that the results reported in the clinical report will be significant or  clinically relevant.
While it is true that many, indeed most REPORTED laboratory errors are derived from the pre-examination phase and it is also true that testing accuracy and significance are affected by specimen quality, it is also true that testing, interpretation are influenced largely by laboratorian practice and competency as is the ability to generate a clinically relevant report.  We know that from data generated from our proficiency testing program over the last 10 years. 
In the table below, you can see performance of advanced laboratories (Blue), intermediate (Red), small (Silver) and very small (Green).  


Creating a manuscript that says all the problems are caused by improper collection and transport is an easy out.  “It’s not my problem, it’s not my fault.  Those guys who collected the sample did it wrong.”  “And if they don’t understand my report structure or my report content or my report jargon, that’s their fault too. “  “The problem isn’t me.  I know what I’m doing.  It’s all those other guys (i.e. you!) who are the problem.” “Trust me, trust me.” 
It speaks to the mindset of the authors when the word “Accreditation” is found in a 100 page (about 10,000 word) document only once, and this is in a footnote.  The word “proficiency” is similarly found only once, and not in the context of proficiency testing.  And the word “competency” is not found at all, nor is the phrase “quality management”. 
If clinicians are going to have the confidence that laboratory results are accurate, clinically relevant, and significant we need to be able to document in a public fashion that we laboratorians take these issues seriously and that we undertake the necessary steps to ensure through qualified external assessment that our leadership and sample management and competence and knowledge and proficiency are active and intact.   

PS: The October in Vancouver Conference for Medical Laboratory Quality is coming along nicely.  Hotel space is filling up well.  I suspect we will have more registrations than rooms at the convention rate.



Sunday, October 14, 2012

Laboratory Size, Complexity and Performance



Size Matters.

Tomorrow is our Annual General Meeting for CMPT, our proficiency testing program.  I have updated my meeting presentation to include all our review materials for the year that ended May 2012 (2011-2012). 

One subject that gets addressed in laboratory performance, not in the sense of finger pointing, but as a measure of our performance as a Quality Assessment program.

I have talked about this subject before, but obviously without particular effect.  To me it is a big deal.  Others may not see it that way, but I think they would be wrong.

We run CMPT which does proficiency testing in clinical and water bacteriology across Canada.  Proficiency testing programs structure themselves in one of two ways, either they give all laboratories the same challenges all the time, or they do not.  We do not.

When we started we stratified the laboratories into two groups.  The largest was category A are large complex laboratories that are expected to handle samples from all sorts of situation.  Often in addition to being large facilities with their own sample load, they also receive referral samples from smaller laboratories. The second group, category B, was large laboratories that tended to work at a lower complexity level.  They were less likely to receive cerebral spinal fluid samples or samples from severely immune compromised patients.  [Note: with the increase in homecare, and political will to ensure more services outside the core, the lines between categories A and B have blurred over time].  

As our program expanded we developed two additional categories; category C which are generally small rural laboratories, and category C1 are facilities whose primary function is to do sample set-up and then pack and ship.  The may be asked to so some basic tests which are defined to include preparing and examining Gram stained slides.  In my mind I would not call microscopic examination a basic procedure.

With 4 categories we decided from the get-go that large complex [A labs] would get a large volume of PT samples that include basic samples, complex samples, and very complex samples.  category B would get many of the same set, but would not get the very complex, largely because they would likely not see those samples often.  Similarly category C got fewer samples, and even less complexity, and category C1 received samples one-notch less.  If laboratories wanted to compare themselves [I hate that!], at least they could see themselves in a pool of comparable organizations.  The point was that we expect to see all laboratories performing well, if not perfect, as compared to others in their group.  

Unfortunately it doesn't work out that way.


Every year, the smaller the laboratory, the less well it performs, and over the decade the trend has never corrected and if anything continues to deteriorate.  This is not a good thing.  

Our best guess is that what this reflects is the relationship between size and complexity, and money, and opportunities for Quality.  The larger the laboratory, the more younger staff they can hire, the more continuing education they can support.  Money buys success.  The smaller laboratories have fewer staff, fewer fresh staff, less money for continuing education and fewer opportunities for Quality.  And the smaller the laboratory, the worse the situation.  

Proficiency testing should not be about inter-laboratory comparison in the sense of a competition.  That is an obsolete old notion retained mainly in the minds of bureaucrats.  But in this sense group to group comparison has value and merit.  

People living in small towns deserve to know that if they get a test done at a local medical laboratory they should have the assurance that the test should be received, set-up, performed and read, and interpreted accurately.  At least at a level higher than 70 percent.  I think that is part of the social contract.  It certainly is part of Patient Safety.

Maybe what is happening is they process all their regular sample work just fine; they only make their errors on the PT samples.  

Maybe. 

But here is a scary thought.  Imagine what the graph would look like if we gave all the laboratories regardless of size or complexity the same challenge samples.

Follow us
@Qualitology

Thursday, July 26, 2012

There are no MRSA


Several years ago I gave a presentation in which I made the point that “there are no Methicillin Resistant Staphylococcus aureus (MRSA), nor Vancomycin Resistant Enterococci (VRE) nor indeed any of the so called “superbugs” (better known as antibiotic resistant bacteria … UNTIL … a sample has been collected and transported to a laboratory capable to accurately testing for antibiotic susceptibility or resistance.  

I can and do argue that susceptibility testing is one of the most clinically significant procedures done by the microbiology laboratory.  In the absence of a laboratory a clinician can diagnose urine infection, or pneumonia, or abscess formation, or sexually transmitted infection, and can make a reasonably informed decision about treatment.  But definitive decision making about treatment of infection requires accurate information about likelihood of response.  And that means having access to accurate susceptibility testing results.

In most people, antibiotic susceptibility only needs to be “within the ball-park” because even if the bacteria are only partially damaged by a less-than-optimal agent, most healthy person’s neutrophils and antibodies will finish off the rest, and the infection goes away.  But in people with damaged immune response, susceptibility testing has to be far more accurate because successful therapy is far more dependent on drug activity.  

That is a long preamble to get to a simple point; antibiotic susceptibility testing must be monitored by regular and ongoing proficiency testing to ensure that systemic error has not been introduced in a way that could affect testing outcome and people’s lives.  

Fortunately we do that.  


Of all the tests that are performed by North American microbiology laboratories, antibiotic susceptibility testing is the one with the greatest degree of homogeneity.  Laboratories as a group usually do consistently well, and as such the percent achieved graph is much tighter.  Despite this there is a long tail with some laboratories with considerable under performance.  

I think I can understand and interpret this pattern.  Many, perhaps most microbiology laboratories use automated equipment that performs bacterial identification and susceptibility testing.  The susceptibility testing is very simple to perform and interpreted by software algorithms (so called "expert routines").  The amount done by the traditional manual agar diffusion assay is limited.  So the number of opportunities for error are reduced, unless the equipment is not working properly, or a slow-growing or special nutrient requiring bacteria is being tested.  

Consistent with most of our examination of percent achieved scores the tail of reduced performance is with the smaller laboratories (data not shown).  My bias is that the reduced performance is tied to  older equipment or to people not recognizing that unusual organisms may have questionable results and need some form of verification.  I suspect that both of these are directly linked to the problems of reduced supervision, and of reduced opportunities for continuing education and reduced budgets.  This is not a worker problem, but a management and supra-management problem. 

So here is the bottom line.  Susceptibility testing is done very well, but some laboratories continue to have problems.  Accurate information on the amount of antibiotic-resistance activity in a community is directly dependent on the Quality of the laboratory generating the information.  It is inappropriate to tolerate any laboratory getting their proficiency testing right only 70 or 80 percent of the time, or less.  It is a dis-service for clinicians and it puts patients at risk.


PS:
I mentioned that susceptibility is “one of the most clinically significant procedures” performed by microbiology laboratories.  The most significant procedures is the detection and documentation of cluster outbreaks either in the hospital or community setting.  

More on this later.