Quality and Medical Research

I was reading the Globe and Mail newspaper the other day and there, on the front page of the section on Life and Arts was the headline “A Study in Sensationalism”.  It was a brief article about the hazards of reading journalist’s articles about medical research.  (I suspect the placement of the article and the headline was intentionally sensational in its own little way).
 

Essentially the author’s view was that many medical researchers hyper-promote their studies when they send press releases and journalists do a poor job of screening the materials that they receive.  Without using these words the article basically described the chain of the inept commenting on the works of the incompetent with the inevitable outcome being an ill-informed and poorly-guided public.  

Call me a cynic, but unfortunately I agree with her. (News journalism is like a stopped clock.  Even if it is almost always wrong, it will be correct for one second twice a day).

 

Is there any wonder why the community at large is suspicious and skeptical and harbours little confidence in institutes of higher learning, and demonstrates no regard for journalists [a recent 2012 Gallup Poll indicates that about 30 percent of independent readers have great or fair confidence in the media; a 2011 poll by Ipsos has a higher level of confidence in scientists, although that confidence has dropped over the last 10 years].

John Ioannidis is a studier of medical research and in 2005 published an essay entitled Why Most Published Research Findings are False.  He accounted for his position, after studied examination that indicated that many studies are too small, too hurried, too imprecise in definition and design, and too influenced by outside factors to have a positive predictive value, meaning that for these studies there is poor or no correlation between the study prediction and reality.  That is why one day you read that coffee is bad and the next that coffee is good or low fat diets are important followed by high fat diets help you lose weight.

The public reacts by turning away from both science information in the media, probably with good cause. 

From my perspective, I see this largely as a failure of Quality awareness and an avoidance of Quality principles. 

I have mentioned before that one does not need to be planning to be accredited or certified to have an interest in Quality principles.  Consider the following:

Principle 1: Customer focus

The research system as it exists today dictates that the reason that one does research has nothing to do with customers or clients or patients.  The principle drivers are mainly self-interest.  For many laboratory heads the goal is to attract money, to generate data to take to meetings and to get published in order to keep the laboratory afloat.  For the graduate student it is to keep the laboratory director and graduate supervisor happy so that they will get their PhD, and in the meantime get enough papers out to create a CV so that they can get a job.  These are not necessarily bad things, but they are enough of an influence that they lead to all the concerns that Ioannidis talks about that lead to low probability of value.

Principle 2,3,4,5:  Leadership, Involvement of People, System and Process Approach. 

The reality is that graduate students and residents get little direct training or engagement.  The system is almost always that trainees learn by getting your hands dirty.  Not exactly what Quality would call an effective systems or process approach.

Principles 6 and 7:  Learning from facts and continual improvement. 

Few research laboratories have any more than a basic level of quality control, and because most research is solitary or competitive, few (none) have programs of proficiency testing, or sample sharing, or internal auditing or external auditing. There is virtually no system in place to become aware when mistakes are being made or error is being introduced, except at the grossest of levels, like major contamination or an explosion.  If you don’t know and don’t look to see if error is happening, then how can the laboratory stop and learn. 

You can’t learn from what you don’t know.

Principle 8:  Mutual Relationships

In many regards, the system is stacked against research laboratories developing a collaborative and mutual relationship with any of its suppliers of equipment, or reagents, or finances.  When they keep an arms-length distance the opportunities for considerations on optimising equipment or considerations on different purchasing arrangements can become very difficult to create.  If they get too close, then the hazards of influence and bias and interference can become a problem.   

So here’s my point.

While I concede that research laboratories work differently from diagnostic laboratories, their self-exclusion from nearly all the Principles of Quality Management make the situation worse than it has to be.  Healthcare in general excluded itself in the same way until the Institute of Medicine published its famed To Err is Human and pointed out the damage of separating healthcare from accepted methods for err detection and continual improvement.  

Sooner or later, the skepticism and cynicism of the general public as a direct result of questionable research and outlandish claims will come to haunt.  It always does.  At that point the outcome will be out of the hands of researchers and in the hands of the public and politicians.  And best of luck when that happens.

And as for journalists, I suspect most neither care nor worry that most people consider them at the near bottom of trustworthiness.  Protections provided by Freedom of the Press exempts them from any requirements or considerations of Quality or Competence.

Exchanging Quality Information.

Being a part of university faculty means that I can expect to be involved in what some people call “information exchange”.  

Personally I prefer that term “information exchange leaders” (IELs)  to “teaching” or being a "teacher" because frankly those are terms badly out of date, a product of a world 60 or more years ago.    “Teaching” and its partner term “lecturing” are both uni-directional concepts; I talk and you listen, or I expound and you learn.  

Even if it can work for short term indoctrinating of children, most kids turn out after a short time.  Adult learners barely tolerate that approach at all, except in the narrowest of situations, such as entertainment (think attending a guest lecture), or under compulsion (think military discipline).

Effective information exchange involves sharing knowledge and experience to the benefit of all concerned.  Adult learners always bring something “to the table”.  In the arena of Quality, an IEL does not have to scratch the surface very deep get participant to share their experiences, both of success and failure; it doesn’t matter which.  What the IEL can do is shape discussion so that either or both can be used in the process of stating the principle, then illustrating through specific stories in order to expand and reinforce information and lead to knowledge.  

In Quality we have a good example of a good story teller: W. Edwards Deming and his Red Bead Experiment [see a re-enactment  at http://www.youtube.com/watch?v=HBW1_GhRKTA ].  

I know it sounds like so much pedagogic gobbledygook,  but actually makes sense.  Think about who were your most effective teachers when you were in school; they were usually story tellers who engaged you in your learning.  And think about the classes in which you learned the most; they were usually ones in which you got to participate through sharing your own stories.  

In the UBC Certificate Course for Laboratory Quality Management we have created a virtual classroom in an on-line environment.  The course is discussion based, and highly dependent on the stories and experiences of the participants.  The faculty are very effective of guiding this discussion through to a shared experience of information exchange and acquisition of new knowledge.  

This is what makes our course so different from other on-line courses where there is only opportunity to learn from the content put on-line for observation and instruction.  There is no input, there is only one right answer and that is the information from the course; all in all, an ineffective technique at education.  As much as it appears modern because it is taught on-line with video-clips and multicoloured and flashing text and on-line multiple-choice examinations, it is fundamentally no different from grade school in the 1950s.  The teacher stands and spouts information and the children dutifully learn, no questions asked.

Our course has now been going for 10 years, and is built much along the lines of many of the Quality Management Principles that it covers.  The course is:

Customer-Focused
Designed to meet the needs of the medical laboratory Quality specialist.  It is limited to a small group to ensure that active participation is enhanced.  Participants are screened to ensure that they know and understand the medical laboratory environment and have some understanding of the Quality process.

Leadership
Motivated and guided through an active facutly

Involvement of People
Requires and demands active engagement of participants.

Process Approach
Each module is separate but the process of engagement is consistent throughout.

System Approach to Teaching
Based on pedagogic principles of adult learning.

Continual Improvement
Regularly updated and refreshed.  This year we are adding in a new module, and a prelude to a new supplemental course on use of on-line tools to enhance medical laboratory Culture of Quality.

Factual Approach to Decision Making
Chages are introduced based on participant survey responses.

Mutually Beneficial Relationships
If both the faculty and participants don’t learn from the course, then we have to investigate why not.

So we are very proud of the course.  Indeed it does exactly what we think a course focused on Quality Management should do.  It is committed to making the learning experience for participants a positive program that leads to enhanced Quality and Learning Culture.  

To learn more about the course, it's content, registration fees, continuing education credits, its faculty please go to www.POLQM.ca

The course is filling quickly for our January 2013 start.  If you think you might be interested, I suggest you visit www.POLQM.ca soon.  Registration closes by December 15th.

ISO9000 and the Quality Management Principles

I was reviewing “The ISO9000 Essentials – 4th Edition” as I prepare to update my notes for our on-line course and the new Resident Training program, and I came across an interesting table that I had overlooked before, entitled “Quality Management Principles”.  I am not sure of the history of the table, but the information is readily available through a brochure that is available on the ISO website (www.iso.org).

As I reflected upon the list, I came to the conclusion that regardless if you seek 3rd party recognition through an accreditation program or a certification program, or if you plan to implement Quality without benefit of an external assessment, these principles are excellent and should be a part of every organization’s mission and vision.

Here is the list:
1. Customer focus
2. Leadership
3. Involvement of people
4. Process approach
5. System approach to management
6. Continual improvement
7. Factual approach to decision making
8. Mutually beneficial supplier relationships.

Most of these are self explanatory, even in the context of the medical laboratory.  If the laboratory does not keep patients and clinicians front of mind, then they do not deserve to stay in operation.  If leaders don’t take care of the present, and at the same time keep their eye on progress and future trends, then their organization will soon become obsolete and vulnerable.   If leaders don’t create opportunities for staff to become engaged, their staff can, and will and should leave.  Management should not need a certification body or an accreditation body to remind them of these realities. Incoming complaints and discontented and leaving staff are loud signals that the public sector hospital leadership or the private sector CEO can not and will not ignore.  

Continual improvement, in its traditional sense, should be a given.  If you don’t learn from errors and accidents then you are likely to repeat them over and over.  It should be inherently obvious that when the bad thing occurs, someone should think “well let’s make sure that doesn’t happen again”!  Any management group that does not understand that has lost before it starts.  This doesn’t have to be a formal process, although a formal process helps because it will likely pick up the conditions for error earlier and with less potential damage.  

With respect to making decisions based on actual information, I am reminded of the recent Dilbert cartoon from the other day where the manager says “From now on, I’m going to rely on my intuition rather than on market research”, to which Dilbert replies “If guessing can bring your success rate up to 50 percent, I am all for it.”  Not every decision needs a full scale investigation and report, but on the other hand, who needs a management team when flipping a coin works just as well.  Again this should be inherent management knowledge, and should not need an assessor’s view.

Systems approach and process improvement are another matter.  Maintaining a structured process in laboratory testing makes sense because it increases consistency, reduces the risk of error, and in both the long and short run saves time, effort, energy and money.  But by and large, most of the downside can be hidden from view for a long time.  Budgets can appear to be in control, and hospital executives tend to become aware of injuries and accidents and excessive sick time only when they become union issues or when the reports from the watchful eye of external assessment comes to the fore.  Mission and vision statements and Quality policies and adherence to programs of internal audits can all too often slip by the wayside unless management is diligent.  The watchful eye of a 3rd party assessor increases the likelihood of attention to detail.

Of the eight principles, probably the one on supplier relations is the weakest.  This is not because of less regard for suppliers (we include suppliers within the group of Quality Partners [ see http://www.medicallaboratoryquality.com/2011/06/more-musings-on-quality-partners.html]
) but rather because it is too specific to a principle.  There are all sorts of groups with whom the laboratory should be seeking out mutually beneficial relationships, including government, administration, clinicians, patients and education providers.  This principle probably needs restating using broader terminology and concepts.

My bottom line: The list of Quality Management Principles is essential information at every level.  I recommend that you visit www.ISO.org and look under ISO9000 Quality Management and find the Principles brochure.  You will be glad you did.

PS: ISO Technical Committee 176, the group responsible for ISO9000 has started the process of revision for the principles.  We will follow (and participate) with interest.