Meta-analyses: Do not rely on them

I got your attention there, did I not? Well – that is NOT my statement. I am quoting a senior professional from one of the super specialties in surgery. I heard this person mentioning this in a conference of the said super-specialty a few years ago. He was from one of the top medical institutions in India. I figured that he was an expert in his discipline, going by the talk he delivered and the several questions he was asked.

This is what he said in his talk:
“And I come to meta-analyses. Please do not pay too much attention to them. Authors could pick up some of the articles they like and do the meta-analysis of just those, to arrive at results they want” –

What made this specialist say this?

My head was first filled with thoughts – Did he not know that meta-analyses and systematic reviews were sources of evidence? How could a senior practitioner – who could influence so many minds – say this?

But then in a few minutes I realized – He was not totally wrong. This could be true in many cases, at least in the Indian scenario and maybe in many developing countries too.

His statement “- Authors could pick up articles they like and do the meta-analyses to arrive at results they want” – could be right because there is practically no structured training in the special search techniques required to find all available studies to do a meta-analysis and to author a systematic review. Lack of access to several databases is a factor known to many, but rarely is the lack of training discussed! 

Let us wake up. And ensure that if we wish to increase the awareness and use of evidence based practice, we must teach the right techniques to find evidence and also to find studies that go into creation of evidence.

Note:

In the same conference, when I gave a keynote, I specially mentioned that to write an article with meta-analyses, the authors need to find all the available studies on the topic. To find all studies, one needed special training to do a systematized search process. And that we lack such training in our country. And of course that  well done meta-analyses and systematic reviews that are recent, are the best sources of evidence. Someone had to tell them this truth. And I was happy that I could do this without making out that the senior specialist was wrong.