Before doing anything, you should ask yourself some key questions:
- How much do I care about this paper?
- How much are others likely to care about this paper?
- How likely are others to rely on the data and/or conclusions of this paper for external reasons such as the reputation of the journal or authors?
- How likely is it that I’ve missed something in the paper or otherwise misinterpreted the authors' work?
Every year sees the publication of many scientific papers, and the vast majority of them are destined for obscurity
. A lot of it is, frankly, crap. Doing anything about the situation you describe will take huge amounts of time, courage, determination, and energy. I wouldn’t touch this paper unless there were something serious at stake.
Let’s say you believe strongly that the paper is important enough to spend valuable time and energy on. You should treat this just like any other piece of science you would conduct. Formulate a hypothesis and develop a plan that takes your own weaknesses into account in testing it.
Here’s one plan you might follow:
Your first goal should be to gather as much feedback from your peers as possible. The reason is simple: we’re all fallible, and you could be missing something obvious or your own biases could be preventing you from seeing the paper in the proper light. If possible, you want someone else to be able to point out your own errors quickly.
Choose three people whose scientific opinions you value. Show them the paper and briefly outline your concerns. How does what they see compare to what you see? If they don’t see something that needs fixing, you’re likely to have a very tough time getting anyone else to give a hoot. Keep iterating on Step 1 until you strike a nerve or run out of time/energy.
Write a short blog post about the problem as you see it. Be clear about the problem and its significance. Provide the right context so your audience can see why your concerns matter to them.
Publicize your blog post. Write to the corresponding author, giving him/her a link to your post. Use social media to spread the word to your followers, friends, etc. Media that might be useful include: Mendeley; FriendFeed; LinkedIn; Twitter; Reddit; and Google+. Email a few key scientists in your field, giving them a link to your post. Contact other bloggers or reporters in your field about your post. Write to 3 or more scientists in your field who you respect but may not know.
You’re likely to receive much more private correspondence than public. Make sure to keep track of all of it. Get back to people quickly if they have questions or concerns.
What kind of response did your post generate? If few cared or bothered to write back to you, go back to Step 2 and keep iterating until you run out of time/energy. If few agree with you but response was good, go back to Step 2 and iterate, or call it quits.
By now, you’ve received a number of responses that share your views on the paper. You now have leverage with the editor of the journal in question. Write a letter summarizing the problem with the paper and the responses you’ve received to your criticism. Provide a link to all relevant blog posts, tweets, articles, online discussions, or other references on the web. Address any criticisms your post may have drawn. If you’ve received permission from those who have privately corresponded, quote them in your letter.
Make sure to be clear about the action you’re proposing. Do you seek a retraction? Do you want your letter published in the journal? Something else? Whatever it is, be clear and up-front.
As you can see, this approach is iterative and is designed to build consensus about the position you’ve taken on the paper. Many other approaches could be effective, but this is one that can be done by any scientist, regardless of rank, title, or standing in the scientific community.
Also notice the prerequisites (e.g.: a blog; a network of peers, readers, and followers; persuasive writing skills). Now is the time to get started on them if you haven’t already, if not for this case, then for one that you’ll come across in the future.