Quality and Medical Research

I was reading the Globe and Mail newspaper the other day and there, on the front page of the section on Life and Arts was the headline “A Study in Sensationalism”.  It was a brief article about the hazards of reading journalist’s articles about medical research.  (I suspect the placement of the article and the headline was intentionally sensational in its own little way).
 

Essentially the author’s view was that many medical researchers hyper-promote their studies when they send press releases and journalists do a poor job of screening the materials that they receive.  Without using these words the article basically described the chain of the inept commenting on the works of the incompetent with the inevitable outcome being an ill-informed and poorly-guided public.  

Call me a cynic, but unfortunately I agree with her. (News journalism is like a stopped clock.  Even if it is almost always wrong, it will be correct for one second twice a day).

 

Is there any wonder why the community at large is suspicious and skeptical and harbours little confidence in institutes of higher learning, and demonstrates no regard for journalists [a recent 2012 Gallup Poll indicates that about 30 percent of independent readers have great or fair confidence in the media; a 2011 poll by Ipsos has a higher level of confidence in scientists, although that confidence has dropped over the last 10 years].

John Ioannidis is a studier of medical research and in 2005 published an essay entitled Why Most Published Research Findings are False.  He accounted for his position, after studied examination that indicated that many studies are too small, too hurried, too imprecise in definition and design, and too influenced by outside factors to have a positive predictive value, meaning that for these studies there is poor or no correlation between the study prediction and reality.  That is why one day you read that coffee is bad and the next that coffee is good or low fat diets are important followed by high fat diets help you lose weight.

The public reacts by turning away from both science information in the media, probably with good cause. 

From my perspective, I see this largely as a failure of Quality awareness and an avoidance of Quality principles. 

I have mentioned before that one does not need to be planning to be accredited or certified to have an interest in Quality principles.  Consider the following:

Principle 1: Customer focus

The research system as it exists today dictates that the reason that one does research has nothing to do with customers or clients or patients.  The principle drivers are mainly self-interest.  For many laboratory heads the goal is to attract money, to generate data to take to meetings and to get published in order to keep the laboratory afloat.  For the graduate student it is to keep the laboratory director and graduate supervisor happy so that they will get their PhD, and in the meantime get enough papers out to create a CV so that they can get a job.  These are not necessarily bad things, but they are enough of an influence that they lead to all the concerns that Ioannidis talks about that lead to low probability of value.

Principle 2,3,4,5:  Leadership, Involvement of People, System and Process Approach. 

The reality is that graduate students and residents get little direct training or engagement.  The system is almost always that trainees learn by getting your hands dirty.  Not exactly what Quality would call an effective systems or process approach.

Principles 6 and 7:  Learning from facts and continual improvement. 

Few research laboratories have any more than a basic level of quality control, and because most research is solitary or competitive, few (none) have programs of proficiency testing, or sample sharing, or internal auditing or external auditing. There is virtually no system in place to become aware when mistakes are being made or error is being introduced, except at the grossest of levels, like major contamination or an explosion.  If you don’t know and don’t look to see if error is happening, then how can the laboratory stop and learn. 

You can’t learn from what you don’t know.

Principle 8:  Mutual Relationships

In many regards, the system is stacked against research laboratories developing a collaborative and mutual relationship with any of its suppliers of equipment, or reagents, or finances.  When they keep an arms-length distance the opportunities for considerations on optimising equipment or considerations on different purchasing arrangements can become very difficult to create.  If they get too close, then the hazards of influence and bias and interference can become a problem.   

So here’s my point.

While I concede that research laboratories work differently from diagnostic laboratories, their self-exclusion from nearly all the Principles of Quality Management make the situation worse than it has to be.  Healthcare in general excluded itself in the same way until the Institute of Medicine published its famed To Err is Human and pointed out the damage of separating healthcare from accepted methods for err detection and continual improvement.  

Sooner or later, the skepticism and cynicism of the general public as a direct result of questionable research and outlandish claims will come to haunt.  It always does.  At that point the outcome will be out of the hands of researchers and in the hands of the public and politicians.  And best of luck when that happens.

And as for journalists, I suspect most neither care nor worry that most people consider them at the near bottom of trustworthiness.  Protections provided by Freedom of the Press exempts them from any requirements or considerations of Quality or Competence.