Sunday, April 29, 2012

Planning for Planning.


It has become an absolute truth that if a person, organization, business or government starts a new initiative without spending some effective planning time, based largely on measureable evidence, the initiative will likely fail.  The more time that is put into developing the plan, the greater the likelihood of success.  “Shooting from the hip” is rarely (if ever) a good idea.
The concept has its origins in the 20th century literature tied to the thinking and writings of Walter Shewhart, either in the original (1939) form of Specification – Production – Inspection or the ultimate adapted form by Deming:  Plan – Do – Study – Act.  Reality is that the concept has its true origins much earlier, perhaps to the early introductions of scientific method, but for purposes of modern systems, if you haven’t learn from the lessons of the past 70 years, then adding more historical perspective will contribute nothing.    

The larger question is not so much whether planning is a good idea, but rather are there tools that can help the Quality Team and the organization get to the point of a cohesive designed plan.    And the answer is yes.  Depending on the size of the task, the tools will vary, although the principles remain the same.

For simple measures like planning for an internal audit the process can be pretty straight forward. What is our goal, what instruments do we need to develop.  Who will be involved, who needs to be notified, and can we set a timeline to complete the task; an all internal operation.  

If the goal is to create a new product or service then having discussions with the “customer” is a good place to start.  You likely will want to include the design people to make sure that the organization has the structured and skilled where-with-all to achieve what needs to be achieve.  If the project is about introducing innovation, then sort out that it is an innovation that forks are interested  in having.  

If everyone is clear on what exactly the end-point is supposed to look like, then the process to be travelled has a chance of being accomplished on-time and on-design.
With organizational renewal or restructure, the introspective approach of a SWOT analysis can be helpful.  By collectively putting a critical and objective eye to an organization’s Strengths and Weaknesses, and Opportunities and Threats, an approach to maximize the positives and minimize the negatives can be organized, prioritized, constructed and put into operation.  The process may or may not involve the thoughts, opinions and inputs from folks from outside the operation, but does need a cold look at objective evidence and the ability to acknowledge weakness.  Looking inward will reduce the risk of starting along a path that does not take appropriate advantage of what exists or stumbles over what is missing.

Recently I was introduced to a new planning-for-a-plan tool called a Policy Lens.  While new to me, there is a considerable literature on the concept going back over 40 years.  It may well be the instrument of choice for policy advocates by which I mean folks and organizations that live for the opportunity to establish policy in large structures such as government, international mega-corporations, academia, and industry sectors.  If you do an internet scan on “policy lens” or “conceptual scan” you will understand where this tool gets its most common usage, especially over the last 5 years.
The term “lens” is and example of organizational jargon, and is probably better replaced by terms such as point-of-view or perspective or vision. 

For establishing as trivial example policy to address the creating of a webjournal (blog) on laboratory quality management one could look at the topic from the perspective of (a) writers (b) blog-space providers (c) quality teams, (d) academics, and (e ) laboratorians.  Each group could be asked to consider impacts on (1) TEEM units (2) credibility (3) risks and liabilities and (4) quality improvement, plus issues of (i) continual professional development, and regional autonomy and (ii) international applicability or (iii) short-term or (iv) long-term application.  To the extent possible, options should be supported by objective measure and consensus.  Each step would needs tools such as questionnaire designs to be developed.  And all this would need to go through the steps of priority and consensus.

This is obviously not a short term exercise and one would be strongly cautioned against developing a “lens” infrastructure for a trivial topic such as blog writing.  But it would certainly be a useful approach for establishing health policy or delivery systems.   Done properly with appropriate validations and confirmations and broad based objective recordable and measurable inputs, thia is a major exercise. 

Can you go through this whole exercise and still end up with flawed policy?  Of course you can.  That is why Planning leads to Doing and then Studying and then Acting. 
But if the policy is big enough or important enough, the risks associated with under-Planning or non-Planning are guaranteed to be profound.   

Tuesday, April 24, 2012

Assessing and improving quality in molecular genetic testing laboratories in Europe

Over the last few weeks I have referred to a person about to complete their PhD in medical laboratory Quality.  To my knowledge Sarah is the first person to have done this.  Congratulations, and here's to her being the start to a new trend.  


I have invited Sarah to prepare an entry for Making Medical Lab Quality Relevant.  I am pleased that she has contributed the following.  With that I am pleased to introduce Dr. Sarah Berwouts.


Only recently I defended my PhD dissertation and it was a pleasure to have Dr Michael Noble in the jury. I also feel honoured to get the opportunity to write a guest post on his entertaining blog. My research project mainly focussed on quality in Europe, so having input from Dr Noble on the Canadian situation was interesting and provided some food for thought. 


An intriguing discussion we had was about the difference between accreditation and certification. In Europe we believe that laboratory accreditation according standards such as ISO 17025 and ISO 15189 is the most efficient way to assure the quality in your laboratory. Certification, according to the ISO 9001 standard, is believed to imply a significant lower level of quality assurance, and not to assess, in general, the technical competence of the laboratory. Although I did not directly measure the difference in quality between accreditation and certification, I gathered information on the degree of implementation of certain quality aspects in both accredited and certified laboratories. These aspects involved, among others, having standard operating procedures, having a quality manual, and performing internal audits. I requested European molecular genetic testing laboratories to indicate the degree of implementation (fully implemented=score 3, advanced=score 2, just started=score 1 or not implemented=score 0) for these quality aspects and this degree of implementation was recalculated as a percentage. In the end, I got an overview of the implementation of the surveyed aspects in about 300 different laboratories. 


As I requested also the accreditation or certification status of the laboratories, I was able to compare the degree of implementation of quality aspects in both groups. Standard operating procedures were implemented very well in both groups (100% vs 99%). However, the availability of a quality manual in the laboratory, was implemented significantly less in the certified laboratories (96% vs 78%). A similar observation was true for performing internal audits (97% vs 68%).


These results supported the European view of certification being less stringent than accreditation. Nevertheless, it is surprising that a relatively high number of certified laboratories do not have a quality manual as this is a certification requirement. 


One could argue that there is a bias in the data, as the accreditation/certification status was collected through self-reporting and laboratories might have overestimated their accreditation/certification status. However, I validated the answers to the survey question ‘Is your laboratory accredited?’ and ‘Is your laboratory certified?’ by checking databases of accreditation and certification bodies. It is however possible that the person who filled out the survey was not aware of what is implemented in the laboratory and that this was more often the case in certified laboratories than in accredited laboratories. Or might it be possible that certification bodies certify laboratories even if they do not conform fully to the ISO 9001 standard?


I understood from Dr Noble that in Canada the situation is slightly different: certification and accreditation are closer to each other, and certification has been appreciated as a valuable tool to assure good quality in the laboratory. All depends of course also on the competence of the accreditation and certification bodies themselves. In Europe, recent efforts by the European co-operation for Accreditation (EA) lead to improved harmonisation among accreditation bodies: there can only be one accreditation body per country now, accredited according ISO 17011. Other European organisations and projects such as the European Society of Human Genetics and EuroGentest actively encourage and promote laboratory accreditation, more than certification. 
Although there might be slight differences in organisation of quality assurance in both Canada and Europe, we all aim for the release of correct and accurate test results and continued efforts are needed to assess and improve the quality in laboratories all over the world.


Sarah Berwouts (sarah.berwouts@med.kuleuven.be)

Sunday, April 22, 2012

When Standards Bodies Get it Wrong.


Today I saw an interesting article on the front page about a new requirement for the quality of drinking water in British Columbia.  As measured in nephalometric turbidity units (NTUs) our current requirement for water shall be improved from a clear level of 5 NTUs down to 1 NTU.  Sounds good and healthy and easy.


The story reminded of a few personal experiences.  A few years ago two countries looked at developing a new standard for protecting health care workers by requiring the thickness and density of the plastic walls to be increased so they would be able to resist a force of 20 N on a standardized Penetration Test.  The existing requirement required a thickness or density enough to resist 15N.  Making containers tougher has got to be better.


Again in the creation of our medical laboratory quality standard, a group of metrologists required the implementation of a expectation that laboratorians would analyze their testing studies for “uncertainty of measurement”.  This would provide a statistical tool that would provide clinical doctors with information to better interpret laboratory tests.  


All of them sound good.  A definitive and forceful statement was made.  Lives have been saved.  The only problem is that in all three situations, there was another point of view that was either not taken into consideration or was rejected. 


According to the newspaper article, it is internationally accepted that that 5 NTU provides safe clear water and the technology to get down to the 1 NTU level will cost millions of dollars to open new filtration units.  According to the public water purveyors, the water value of the new level in terms of safety and drinkability is trivial at best.  This has more to do with politics than quality.


With the sharps containers, the sharps container manufacturers had information that demonstrated the increase in plastic would not materially make the containers safer, but would result in a major equipment retooling and cost millions of dollars.  The consequence would be containers that are more expensive, but not better.  


And with the uncertainty issue, most laboratorians had no understanding or experience with uncertainty but they were pushed into adding it as a requirement anyways.  Even today, 10 years later, the only reason laboratories spend any time on this measurement estimate is to satisfy accreditation teams who don’t know or understand it any more than the laboratorians.  I was talking earlier this week to a group of accrediters who ignore the clause on “uncertainty of measurement” in ISO17025 because they recognize that it makes no sense in the laboratories they assess.  


Three stories all with a common theme, that the developers of these documents broke the first rule of what constitutes a standard; consensus.  Without wanting to flog the point, (see Standards Development and Social Media Tuesday, April 17, 2012), the problem with all these examples was can go wrong when there is clearly a fissure in standards development committee process.  


Any time one group strongly asserts that they are “right” and the other guys are “wrong” the chances of ending up with a consensus is near impossible.  And this impacts on the credibility and applicability of the document.  These sorts of problems should not be come up if the standards committee does its job properly.  
There are a few reasons for ending up with flawed or controversial clauses.  There is poor group selection, where bias is introduced even before the document got written.  The second is the power of a persuasive participant with a singular point of view.  And the third is when documents don’t get a sufficient public review process.  These may make for faster documents, but not better ones.


My advice is to read the documents thoroughly and sceptically.  If you are knowledgeable in your field and find something that seems like nonsense, you almost assuredly are not the only one who feels that way.  Write a letter to your professional body and to the organization that wrote the standard and point out the problem.  And find out which organization is responsible for standards in your country and write to them.  In Canada it is Standards Council of Canada (SCC).  In the US it is the American National Standards Institute (ANSI).  In UK it is the British Standards Institute (BSI).


Make sure your point of view gets heard, because without that you can expect no action. 

Tuesday, April 17, 2012

Standards Development and Social Media.

The International Organization for Standardization (ISO) and the collective body of world standards development bodies play an essential role for Quality.  Without them the documents that provide rules and guidelines for consistent practice and activities would not likely exist.   Standards based on the combined results of science, technology and experience and bound together by consensus are the foundation of trade, common societal goals which today includes consistent medical laboratory practices including quality management. 


I would like to think that there is some truth to the notion. 


As mentioned, the reason that standards can work is because they are developed using a consensus model in which there is a requirement for solid agreement that substantial objections are resolved.  This doesn’t mean that everyone has to be happy.  It only means that everyone has to be able to accept or at least live with the document.  While to some it may sound like an invitation to the “tyranny of the minority” consensus requires that every person who has legitimate concern needs to be heard and every attempt to resolve their major conflicts be resolved.


It sounds so good.

But all too often the process gets hijacked because everyone in the room is of like mind.  Get a group of scientists or engineers or industrial design folks in a room and it is very easy to come to common understanding.  It is not so much a nefarious plot, but rather you have people from a common background, common education, and generally a common point of view.  Even if you would want to extend the group, while you may introduce differences in politics or single issue differences, the pool continues to have consistency in the understanding  that matters the most.  Folks from disparate regions all too often can’t afford to participate, even if they get to know what is happening.  And often consumers and public representatives (patients, students, less educated, less funded, and people who don’t belong to the specific professional organization or folks with a longer range vision) do not get their point of view included.  In fact since most of the folks on the inside wouldn’t even know where to find these alternate opinions, they don’t even get asked.

So it should not be too surprising that sometimes standards or sections within standards  that look great on paper fall apart in practice.

And this is where new technology comes to play.  Organizations like the Canadian Standards Association have introduced public access points where the general public can visit and view documents and comment on them during the period of public consultation.  Truly different points of view can get raised and need to be considered and examined.  They can not be ignored.  In Canada it means that interested parties living in Iqaluit or Yellowknife or Regina (a joke; it’s just a joke!) can in the comfort of their home of office review documents and make informed comments before the document is actually published.  This may give a committee pause to consider or reconsider.  The committee may even consider contacting the person by phone for further discussion.  Or they may, upon reflection, decide that the issue is sufficiently off-topic or does not meet the level of substantial objection that it can be noted by rejected. 

Well you can say that this is not really a democratic or balanced extension to the consensus process because when push comes to shove all that they have done is figured how to include more people who (a) know where to look (b) have a computer (c) have an internet connection (d) can read and communicate in the common language and (e) can understand the language of standards development committees.  And that argument has some merit.  

But the public access sites can be promoted on Twitter© or Facebook© or blogs and forums.  Distribution of knowledge will get broader and deeper, as will the diversity of opinion.

But until the day comes that all the world governments widely dispense free  Google© glasses to all the world citizens at birth, I am satisfied that giving more consumers and other folks who potentially can be directly impacted new requirements a voice that they did not have before is a pretty big deal. 

So “good on ya” to the CSA and other standards development bodies taking a bold step forward.  And to the others, what’s taking you so long?


Saturday, April 14, 2012

Progress in Quality Research


I have been immersed in the arena of European laboratory quality over the last few days.  It has been a very interesting and positive experience.  I have had the opportunity to participate as a jury member for a candidate’s dissertation and defence of her PhD thesis on laboratory Quality.  To my knowledge this was the first (maybe second) ever PhD based on research on medical laboratory Quality and Proficiency Testing (External Quality Assessment –EQA).  Incredibly interesting and exciting.  What I saw was a level of creative and innovative enquiry and a growing awareness in laboratory Quality as a valid subject of study.  To my mind this is directly attributable to the development and dissemination of ISO 15189.  

As we continue to see progress with new laboratory technologies and techniques that will expand the technical information we will also see growth in the tools that ensure that this new technology has a Quality foundation.  Having gone through an era of too many tests that were fast, slickly packaged but complicated with poor sensitivity and even poorer specificity, the progress in studies on better quality monitoring will shine light on assay inadequacies and do much for continual diagnostic improvement.  
I have invited the candidate to appear in this web-journal site as a guest and I look forward to her presentation and insights soon.
 
During the PhD dissertation and defense some key observations were presented and recommendations for change were offered.  One recommendation suggested a role for a more uniform approach to quality for European laboratories.  Over the last 5-7 years European countries have been changing their approach to accreditation by requiring that the number of accreditation bodies reduce to only one official one per country.  That will be similar to the Canadian situation where there is only one provincial accreditor.  For some countries this may be seen as a restriction because in the past some countries have had multiple accreditation bodies, but this will result in increased intra-country consistency and uniformity in quality assessment.  There was also thoughts on the development of a over-arching role for EU policies for a more unified inter-country approach to Quality - something akin to the US CLIA.

Before that happens, I see an opportunity for some fundamental comparative study that could shed light on the modern value for supervening policies including mandatory accreditation.

In the United States there is a quality system that is driven centrally through the CLIA requirements under the authority of Health and Human Services (HHS) and the Center for Medicare and Medicaid Services (CMS).  While states are allowed to have their own accreditation program, they must as a minimum meet the requirements of CLIA.  In each state there is a 100 percent requirement for accreditation.  
There is another model in Canada where we have a  decentralized approach.   Each province has the authority to decide on the role and necessity of  laboratory accreditation.  Most provinces have their own single accreditation program or use the accreditation program from another Canadian province.  (Some provinces allow the intrusion of a foreign accreditation program, although the benefits or reasoning for this escapes me.)  In almost every province where there is accreditation, the requirement is mandatory and there is 100 percent participation. 
In Europe they have a system similar to Canada but different.  If one considers each of the EU countries as an independent but integrated unit, then each country has a status similar to a US state or a Canadian province.  In the EU, like Canada, there is no overarching “CLIA-like” structure.  But in the EU countries, where there is accreditation, it is voluntary, so that the participation in most countries is only around 20 percent.

So we have 3 jurisdictions US, Canada, Europe; each with comparable development of medical laboratories, each with over 1000 medical laboratories, but with three different stable structures for Quality and Quality Assessment.  I can’t imagine a model better suited for study and analysis.     

Sunday, April 8, 2012

Slogans, Exhortations, and Targets

Slogans, Exhortations, and Targets

It is relevant to remember that W. Edwards Deming passed away nearly 20 years ago, and that while brilliant and a visionary in Quality, he was not the only person to develop a Quality philosophy.  That being said it is really important to take note of the Deming method and to consider it as a key measure of context. 

For those that are not studiers of the Deming Management Method, I will tell you that it was largely built around a series of 14 points and 7 deadly diseases.  If you are not aware, or were aware but have forgotten, I will tell you that there continues to be a lot of wisdom here that is worth reading and considering as you implement Quality program of your own.  There are a bunch of places to find this stuff, on the web and in books.  One oldie, but goodie, that we still use in our course is “The Deming Management Method by Mary Walton.  It was written in 1986, when Deming was 86, and he wrote the forward for the book himself. 

One of his 14 points was “Eliminate slogans, exhortations, and targets for the workforce”.  His point was that slogans don’t make things easier for the worker, they make things harder.  They make for unreasonable expectations, and create more anxiety and frustration than they provide encouragement.   One slogan in particular that he hated was “DIRFT – Do it right the first time”.  There are a number of writings that make it pretty clear that Deming and Crosby did not see eye-to-eye in a number of areas.  I suspect that at a very gut level, Deming was not a fan of Phillip Crosby.

But before I go off too far on a tangent, I will say that what Deming was a fan of was making things measurable.  Continual improvement is not a state of mind, it is or is not a measurable mathematical reality.  Define the conditions for improvement and develop an indicator that you can count or otherwise objectively measure.  You either meet the measure or you don’t.

I mention this because I was flying back from a meeting, flipping through the airplane magazine and I saw an advertisement for a car.  There was a big and bold punch line:  “Excellence wasn’t our goal.  It was our starting point”.  As I read the line, my first reaction was “well I am sure some ad exec thought that was pretty cool, but from a Quality perspective it’s pretty dumb”.  And then, knowing that Deming had worked within the automobile industry, I wonder what Deming would have said.  Well I know what he would have said. 

Deming in the early 1980’s was a consultant for the Ford Motor Company and was instrumental in helping them make the transition from the flash and excess and poor construction of the American automobile that characterized the fins and weight and gas guzzling of the fifties and sixties and seventies.  He taught them that in the long run meeting positive expectations year over year was the road to success.  Rather than making flash as job 1, or making speed as job 1, Ford would succeed making Quality as Job 1.  And thus a very successful, and measureable slogan was born.  
And in 2008 when Chrysler and General Motors were dying, Ford was able to say “no thanks” to the bale out. 

I continue to see sloganeering in advertisements but more in Quality mission statements.  I argue that meaningless phrases that are impossible to measure or impossible to address have no role in Quality mission statements.

We provide “world-class” service.
We are the leading edge service provider.
We are the bleeding-edge of our industry.
We are the best-in-class world stage performer.
We meet or exceed all world class standards.  (Even if this sounds measurable, what does “exceeding” standards mean?)

I guess it all sounds sexy and flash, but it is just nonsense. 

Deming would not have been impressed.

Sunday, April 1, 2012

Invention and Innovation and “new knowledge”

Invention and Innovation and “new knowledge”

Our recent Canadian budget shone light on an issue critical to Quality.  Apparently Canadians are very good at invention and very poor at innovation. 

For purposes of this discussion it is important to see these two words as different.  Invention is the formulation of new ideas for products or processes, while Innovation is about developing practical application of new (or old) inventions into better products or services.  In the business entrepreneurial context “better” mean “marketable”.
The more that I explore the concept, the more I appreciate the continuum: Invention is at the top end and innovation happens downstream.  Moving from idea to product creates one form of innovation (innovation leading to new products), tweaking the process further downstream creates another (innovation leading to improved products).    According to our federal government, Canadians have a good track record on invention, but we stink at innovation, and that results in a poor record of falling productivity.

I have been thinking about this concept for a while, not only because of our federal governments concerns, but because it pertains to some things that I am very involved with right now. 

At my university the handbook for Graduate Supervision says to earn a master’s degree the student has to produce a significant scholarly work with some original contribution whenever possible, and to earn a doctoral degree the student shall develop a substantial and original contribution of new knowledge to the field of study.  To me that sounds a lot like my institution is expecting to produce doctors of invention and masters of innovation.  But when push comes to shove, I suspect that that represents more rhetoric than reality. 

For the longest time I have been both intrigued and confused by the term “new knowledge”.  The root seems to come from the Artificial Intelligence community who were talking about the synthesis of “new knowledge” from previously stored “information and knowledge” in the nineties, but I can find references going back thirty years previous.  Today it has become a new hot phraseology like “new knowledge economics” and “new knowledge society” and “new knowledge culture”.  So whatever it meant before, it means something different today.

I trust that what academia is trying to express when they require our PhD candidates to produce “new knowledge” is that after working on a project for several years that we should be able to expect something that represents some level of progress and innovation.  
Patentable new ideas and invention would be nice but innovative ideas that lead to progress in understanding are powerful in their own way. 

At a more immediate level, my proficiency testing program defines our mission as “innovation, education, continual quality improvement and quality management”.  Innovation comes first. 

So can we justify our bold statement.  I think we can. 

This year we have created a new tool for providing proficiency testing for detection of shiga toxin (aka verotoxin) in enterobacteria.  It works regardless of which method the laboratory uses for detection.  We have not put a patent on the material, but I think that this qualifies as both “invention” and product innovation.  At the same time we have redeveloped our gram stain slides so that we can now ensure that they will retain their quality for examination for up to a year so that we can be more productive in producing our materials for our various programs, and I would read this as clear evidence for process improvement.  

(Question to self: would this be a reasonable foundation that could lead to a PhD in Laboratory Quality?)

So here we are; reasonable evidence for invention, solid evidence for innovations and a pathway to better productivity.  I should let our finance minister (Jim Flaherty) know that we are innovative and productive Canadians.

And perhaps it has become time for our universities to work through some continual improvement process to ensure that it uses terms that we can all understand.

Note: minor revisions to spelling and grammar made June 16, 2012.