Over the last few weeks, the dinosaur blogs and vertpaleo listserv have been in an uproar over a rather atrocious paper that came out on sauropod dinosaur relationships. I’m not going to talk about that paper, in part because I’m not a sauropod expert and can’t really give any expert commentary that the folks who actually work on sauropods couldn’t do better (although even to a mostly mammal worker like me many of the flaws in the paper are obvious). However, in addition to apparent severe scientific errors, the paper is questionable on some other grounds, including the use of privately held specimens and the use of a non-peer-reviewed “publication”. Mike Taylor and Matt Wedel led a detailed discussion about the flaws in this paper at SV-POW. In the comments on those entries it was obvious that many folks have misconceptions about how scientific publications actually work. Mike addressed some of these in a third post at SV-POW.
In interacting with the public through museum activities I’ve heard many of those misconceptions, as well as some that weren’t mentioned there, so I wanted to address some of those as well.
Misconception 1: Only “professional scientists” can publish scientific papers.
Reality: Anyone can submit a paper to a scientific journal for publication, and (at least in theory) it will receive the same consideration as a submission from a “professional scientist.” As one example, students write papers all the time; in the paper shown above, Anna was an undergraduate student when the paper was being written.
By the way, I put “professional scientist” in quotes because I’m not sure what that even is. Very few people are actually paid to be paleontologists. Most are paid to teach geology, biology, or anatomy, and doing research is either a small part (or no part) of their job responsibilities. I am actually paid to do research (although in my job description that’s listed at less than 50% of my time responsibility), but positions like mine are relatively rare.
Perhaps by “professional scientist” people mean someone who has a graduate degree in a relevant science. This is actually not a requirement for publication, either. However, getting a graduate degree is not simply a means of getting more prestige and a higher salary. Graduate school is where you get training in how to do science. You learn about anatomy, geology, evolutionary theory, ecology, and a whole host of related fields, but you also learn techniques, like how to collect field data, how to take measurements, how to analyze data, how to write scientific papers. In principle, it’s no different than learning any other trade. When I get on an airplane I certainly like to know that the pilot has been through flight school! That does not mean that you MUST go to graduate school to learn to do science, any more than you must go to a trade school to learn a trade. But in general if you’ve been trained in a field you will be able to do the job more effectively.
Misconception 2: Scientists are paid royalties for publishing scientific papers.
Reality: For the vast majority of scientific papers, the scientist does not receive any compensation (other that keeping his/her job is publication is part of their job description). On the contrary, most journals request (and sometimes require) the author to pay a fee to get the paper published. These fees often run over $100/page. Moreover, many journals ask or demand that authors turn over the copyright to the publisher, so that the author does not even own the right to use their own paper without paying a fee (this is an increasing contentious issue was also recently discussed on SV-POW). The main exception is in certain semi-popular books, and textbooks, but this is a relatively minor component of the scientific publication sphere.
Misconception 3: Scientists can publish whatever they want.
Reality: In reputable journals, there is some type of rigorous review process that a paper goes through before it’s accepted for publication. In fact, having this review process is what makes a journal reputable. For highly visible journals like Science or Nature, the rejection rate may be 90% or more. (A common activity for paleontologists is to sit around the bar discussing who got the fastest rejection from Science or Nature.) Papers might be rejected because of some serious and obvious scientific flaw, or because it’s outside the scope of the journal, or because the editor just doesn’t think it’s a good fit for the journal.
Increasingly (almost universally, these days) part of the review process is peer review. In this case, the journal will send the manuscript to one or more reviewers who are experts in that field. The reviewers work as volunteers, and as such are not paid to do the reviews. During 2009 I did five reviews for four different journals. Peer review can have its own problems. Sometimes the reviewers are not really qualified to do the review (although hopefully an unqualified reviewer will decline a review request). Scientists are people too, and sometimes personal grudges can result in improper reviews (although, again, there are safeguards; an author can request that certain people not review their paper). But for all its flaws, peer review is a pretty good system, and when done properly can greatly improve the quality of a manuscript.
As it happens, I have a manuscript currently going through the publication process with Jeffersoniana, VMNH’s online publication. I thought it would be useful to describe in detail what happens with a manuscript submitted to Jeffersoniana. The details of the publication process differ with different journals, and because Jeffersoniana is small it’s a little simpler than, say, the Journal of Vertebrate Paleontology. Even so, it serves to demonstrate what happens when a manuscript is submitted for publication.
Typically Jeffersoniana submissions originate with one of the VMNH curators. Submissions from non-VMNH staff are received by one of the curators (it’s a small operation, so submitters often know someone on the VMNH staff). The submission then goes to the chair of the Publications Committee (a curator), who circulates it to the members of the Editorial Board, which is made up of the VMNH curators and curators emeriti. The Editorial Board decides if the submission is appropriate for inclusion in Jeffersoniana (or any of our other publication series). They check to make sure the submission really is science, that there are no ethical issues, and that it’s within the scope of the journal. If a board member is qualified on the subject, they may do a more detailed critique of the paper. Note that if the author is an editorial board member they are excluded from this step. Therefore, I don’t get to vote on whether or not my submission is suitable for the journal (in fact, I have to go through the submission process as if I don’t work here).
If the editorial board approves the submission, next it goes out for peer review, typically to two reviewers. The author can recommend reviewers (or request exclusion of reviewers), but the chair of the Publications Committee makes the decision as to what reviewers are selected. She will send the title and abstract to potential reviewers, and, if they agree to do the review, she sends them the entire manuscript. As Jeffersoniana submissions are short, reviewers are usually asked to return their review within two weeks.
After reviewers are finished, they send their comments to the chair of the Publications Committee. She then forwards those comments to the author. (The reviewer can request that their identity not be revealed to the author.) If the comments indicate that the paper is severely flawed, she may choose to reject the submission, but for Jeffersoniana more typically she would instruct the author that the paper must be revised to address the reviewers comments.
After making the revisions, the author sends the revised manuscript back to the chair. If the revision is satisfactory, it is then reformatted for Jeffersoniana (usually by either the publications editor or by me). At this point it goes back to the author and to the chair, to do a final check for typographical and other errors. The chair may choose to have additional people proofread the manuscript as well (my current submission is at this stage). Any errors are corrected, and the author has one additional chance to check the manuscript and approve it. It then goes to the museum’s webmaster to be put on the website, and when it goes live it is considered published.
This may seem like a long and convoluted process, but it’s necessary to ensure quality control for publications. (Also, referring back to the misconceptions above, at no point do we ask the author to present their credentials, and at no point do we pay them anything.) By the time Jeffersoniana issues appear on our website, a minimum of eight scientists, including at least two experts in the particular field, have had a chance to read and comment on the paper. That still isn’t a guarantee that everything that gets into print is perfect, but it’s certainly better than no review at all, and it allows the reader to have some measure of confidence in the validity of the paper. Every reputable journal has a similar process that will differ mainly in the details. That’s why, for example, things that I write on this blog would not (and should not) be considered scientific publications worthy of citation in someone’s manuscript, and it’s one reason (among several) why the sauropod paper being discussed at SV-POW should not be considered a proper publication.
(Edited to correct the title.)