Journal of College Science Teaching — January/February 2014
Teaching About Ethics and the Process of Science Using Retracted Publications
The use of primary literature in the undergraduate classroom has great potential for improving students ’ understanding o f the methods o f science as well as specific topics required for a given course. However, it is important to teach students that the scientific literature is subject to change as new information accumulates. In this article, we discuss our experience using articles that have been “abandoned” (where their results are no longer accepted due to new evidence) and/or retracted as methods for teaching students about scientific literature in general and specifically about scientific ethics. Being presented with a more accurate picture o f primary literature can help students develop an improved understanding o f how science is actually practiced and how scientific ideas change over time. By examining retracted articles in which ethical lapses have been uncovered, students are able to develop a more clear understanding of the types of ethical problems that can occur and improve their ability to recognize them. The use of primary literature (PL) in the classroom has a long history (e.g., Dye, 1986) and is suggested to increase students’ understanding of the process of science (Brill & Yarden, 2003; Mulnix, 2003; Wenk & Tronsky, 2011); increase their success and retention in science, technology, engineering, and math (STEM) majors (Hoskins, Stevens, & Nehm, 2007; National Research Council, 2003); and prepare them for graduate study (Janick-Buckner, 1997; Kozeracki, Carey, Colicelli, & Levis-Fitzgerald, 2006). PL can be used in the classroom in a variety of ways, including case studies (Camill, 2000), class projects (DebBurman, 2002), and journal clubs (Glazer, 2000). Although specifics vary, PL in the classroom typically involves the use of recent publications that relate to the course content, sometimes taught in conjunction with database/ library searching skills (Leckie & Fullerton, 1999; Mulnix & Penhale, 1997; Priore & Giannini, 2007). The focus on current literature provides students with up-to-date information and some background on the development of the topic through the introduction and references cited in the paper. Reading and analyzing the PL is considered an important aspect of scientific training (National Research Council, 2003; Peters, Baysinger, & Craig, 2011). Undergraduate students may have a difficult time deciding that the results of a peer-reviewed publication are questionable or outdated (Gillen, 2006) because students at this level are often lacking the necessary background understanding (Herman, 1999; Jacques-Fricke, Hubert, & Miller, 2009; Janick-Buckner, 1997; Rauschert, Dauer, Momsen, & Sutton-Grier, 2011; Smith, 2001). However, they are capable of addressing at least some aspects of PL (Janick-Buckner, 1997; Rauschert et al., 2011). They also benefit from exposure to topics that they will see in future upper level courses or graduate school (Janick-Buckner, 1997; Rauschert et al., 2011).
Even if students are able to understand the material in a given publication, there is still the problem that most usage of PL in the classroom provides students with a picture of scientific knowledge at a single point in time but misses the fact that later research may have rendered the current publication outdated. Because of these limitations, students can mistakenly develop the impression that any PL they read is correct, needing only minor critiques, thus preventing the development of the critical-thinking skills that are crucial for young scientists. Students need to understand that as scientific knowledge progresses, information is not simply added; ideas that are no longer useful are also discarded, sometimes creating major paradigm shifts that can dramatically change the way that results are interpreted (e.g., Kuhn, 1970).
There are two types of situations that can produce significant modifications in our understanding of a topic, classified on the basis of whether the original publication is abandoned or retracted. An abandoned article represents one that contains conclusions that are refuted by later studies while still remaining in the record of scientific publications. In contrast, a retracted study is one that is withdrawn from the scientific literature, often due to serious problems in the study. These types of publications allow us to teach students about the process of science and can help improve their ability to critically analyze materials they are reading.
When publications are abandoned, it is common for the original publication to remain in the literature with little or no indication that it no longer represents accurate science (e.g., Miller, 2010a). Abandoning incorrect ideas is a goal for science. but it can be difficult to understand this process when examining articles at a single point in time. In addition, some scientific controversies that we consider settled today took significant time to be resolved (Boantza & Gal, 2011; Gillen, 2006). By addressing abandoned ideas more explicitly, students gain an understanding of science as it is practiced. There are also clear benefits to having students learn how scientists address controversies and how they are eventually resolved (National Research Council, 2003). Ideally, as students develop an understanding that all studies suffer some flaws, they will learn to distinguish between major problems that might render conclusions suspect and those that are less serious.
In contrast, teaching with retracted publications represents a situation where it is possible to “take sides” because publications that have been retracted contain issues that are so significant that there is less need to worry about the merits of the ideas presented by the authors. This allows an instructor to focus on the identified problems with the research without requiring a broad understanding of a particular topic from students who may not have the appropriate background. Major cases of research misconduct and the resulting retractions have received coverage in scientific journals (e.g., Marshall, 1996), and high-profile retractions of research have made headlines in both the scientific literature and mainstream media (Harris, 2010; Nature Medicine, 2010, 2011; Steen 2011b), so using such examples can address issues of both scientific merit and public policy. There are many reasons that may lead to a publication being retracted; we have provided some examples in Table 1. Many of these papers represent different types of unethical behavior that we hope to help our students avoid. In retractions that are not due to misconduct, it is still beneficial to examine them with students to help them understand how scientists are supposed to address errors in their work. These types of studies also allow illustration of the different ways that scientists can violate the ethical norms of the profession and the consequences of such behavior. However, with retracted articles, there is the problem of the delay between the original publication and its retraction, which can occur for a variety of reasons (Couzin & Normile, 2008; Normile, 2009a, 2009b; Oransky, 2012; Roberts, Sivaguru, & Yong, 2007; Trikalinosa, Evangeloua, & Ioannidis, 2008). Thus it is vitally important for students to develop the ability to recognize that publication is no guarantee of accuracy because peer review is a highly subjective process in which even fraudulent papers can be published more easily than scientists like to admit (Marcus & Oransky, 2012; Smith, 2006). In fact, even articles that have been retracted fully may still be available in their original form, with no clear indication of their retraction (Fang, Steen, & Casadevalid, 2012; Long, Errami, George, Sun, & Garner, 2009; Steen, 2011b) and often continue to receive citations in other peer-reviewed publications even after being retracted (Budd, Sievert, & Schultz, 1998; Budd, Sievert, Schultz, & Scoville, 1999; Fang et al., 2012; Howard, 2011). If journals and practicing scientists are not aware of the retracted status of papers, it seems likely that undergraduates are going to have difficulty as well.
At first, one might question if using scientific research that has proven to be flawed is the best approach to teaching ethics (Epstein, 1998). Although this unconventional approach may not seem obvious, it is really quite similar to using case studies to start a dialogue (Fisher & Levinger, 2008; Niece, 2005; Shachter, 2003; Vincent, 1999). In addition, given the dramatic impacts that flawed research can have on the process of science and public trust in science (Committee on Science. Engineering, and Public Policy, 2009), it is imperative that students are aware of this problem. By using retracted articles, students gain experience reading PL at the same time as they develop an understanding of scientific ethics. Because retraction is generally going to occur after a significant problem has been discovered, it allows students to gain a better understanding of the dynamic nature of scientific literature.
In the end, the use of PL has significant potential benefits for undergraduates, and evidence suggests that its inclusion in the undergraduate curriculum is a good idea. Our goal was to use PL to increase the ability of our students to critically examine the literature they were reading for course assignments as well as any publications that they might read in the future.
We have addressed abandoned and retracted literature in several sophomore- to senior-level courses for undergraduates majoring in biology and chemistry. We have used such literature in a variety of ways, including as background for lectures or in-class discussions. Here we focus on a seminar for sophomore chemistry majors (CHEM 2500) in which we expanded the use of retracted literature to be a major component of the course. This course focused on scientific ethics (Singiser, Clower, & Burnett, 2012), so the retraction of papers for ethical violations served the purpose of the course in many ways. CHEM 2500 was taught in the spring of 2011 and 2012 with a total enrollment of 19 students, each of whom examined a previously retracted journal article. Students were not limited to papers in chemistry as the purpose of this was to focus on the ethical issues (Singiser et al., 2012), so many students chose papers from the biomedical sciences. Each student was required to read the paper, interpret the reasons for the retraction in terms of the ethical concerns that were at issue, and present that information to the class and other faculty in both oral and poster presentations. These presentations covered the topics from the article, the ethical problems, and a discussion of why those problems existed. To perform this project, the first requirement was that the students have access to a set of retracted articles that they can read and understand.
There are a variety of sources in which students can search for retracted articles, which also provides the opportunity for the students to gain skills at searching for scientific literature. Some useful sources are SciFinder (Chemical Abstracts Service, http://scifinder.cas.org), PubMed (National Center for Biotechnology Information [NCBI], http://www.ncbi.nlm.nih. gov/), and even Google. In addition, there are now specialized websites that are tracking retractions (Marcus & Oransky, 2012), and some journals publish information on major retractions (e.g., Nature Medicine, 2011). In terms of the searching strategy, the major difference between retracted papers and regular papers is that there are fewer retracted papers to find, so more perseverance may be required. The actual number of retracted papers is difficult to ascertain and will depend on the methods used. Estimates over different time periods range from several hundred to more than 2,000 (Budd et al., 1999; Fang et al., 2012; Saunders, 2010; Steen, 2011a). Most studies on retraction focus on articles indexed in PubMed, which currently indexes over 5,600 journals (National Library of Medicine, 2012), but clearly this does not represent the entire range of possible retractions. In addition, for PubMed to list an article as a retraction, certain conditions must be met (Long et al., 2009, National Library of Medicine, 2011), meaning that some articles that are in this database may be retracted but not marked as such. There is also evidence of bias concerning which journals are more likely to contain retracted articles. Journals with higher impact factors typically have higher retraction rates relative to the number of articles published (Cokol, Iossifov, Rodriguez-Esteban, & Rzhetsk, 2007; Steen, 2011a; Trikalinosa et al., 2008). All of this provided fertile topics for discussions with students during the process of finding and interpreting the retractions. For example, do high-impact journals publish more bad science or do they simply detect it and take action more often (Cokol et al., 2007; Steen 2011a; Trikalinosa et al., 2008)? This allowed the process to serve as a tool for increasing the students’ critical thinking at the same time as it improved their ability to work with primary literature.
Once a suitable set of retracted articles was identified, students worked with instructors both to understand the science and to identify ethical issues that were problems for the study. After each student had selected an article, he or she needed to find the reason for the retraction. This usually required searching the specific journal’s website, where this information may be published as a comment or may be included directly with the electronic copy of the original article. One problem with retracted literature is that the cause of retraction is not always clear (Steen, 2011a) or is not accurately reported by those involved (Fang et al., 2012), so this serves as an additional way to help students develop their critical-thinking skills. Numerous examples of retracted papers could be selected, but one that was used by students in the course is Chemouri & Mekelleche (2009), which was retracted in 2010 due to errors in the text and supplementary materials. These errors included an incorrect procedure for calculating some of the values reported in the publication. The information on this retraction is provided on the website for the journal (Chemouri & Mekelleche, 2010). Another example is Deb, Sivaguru, Yong, and Roberts (2006), which was retracted in 2007 due to research misconduct by the lead author that included fabrication of multiple figures in the publication (Roberts et al., 2007). In this case, the information was provided as a retraction notice in the journal, which was also appended to the electronic copy of the original article.
We have used the issues of abandoned and retracted papers to teach about the process of science, primary literature, and scientific ethics. The use of such articles introduces some challenges (e.g., finding relevant articles for a particular course), but the potential benefits to students’ understanding of the scientific process are significant. When restricting such assignments to only retracted articles, the instructor has a more limited pool of articles to use, but even within that limited number, it is possible to illustrate almost any type of ethical lapse that one might wish to examine. Because of the small number of students who have taken this course, we are not able to address how significantly this process has helped them understand scientific literature and ethical problems. Comments given on end-of-semester evaluations do suggest that students found the focus on ethics interesting and felt that it was an important part of their development as scientists. Even with only anecdotal evidence, it seems reasonable to propose that the only way students will be able to develop an understanding of scientific ethics is through exposure to ethical issues that actually arise in the process of doing science. This course may represent a small step in the process of developing their understanding, but by presenting examples of problems in science, students may gain an improved understanding of the progress of science (Epstein, 1998).
It has also been noted that the number of retracted articles is dramatically increasing, and although there are numerous ways to interpret this increase in retractions (Steen, 2011b), this situation presents serious cause for concern that should be shared with students. Students should understand that high-profile cases of research fraud can cause serious harm to the public (e.g., Harris, 2010) and/or seriously undermine the public trust in science (e.g., Normile, 2009a). Additionally, ethical lapses in publication can do serious harm to the scientific process, diverting effort and resources to paths that will not provide useful results (Miller, 2006, 2010a, 2010b; National Research Council, 2003). By focusing on ethical issues, we were able to raise our students’ awareness of these problems, which will help them avoid these problems in their own careers (Moody & Freeman, 1999). We hope that they will recognize such problems when they encounter them and therefore be less susceptible to them. In some cases it may not be completely obvious to a student why a particular situation represents an ethical concern. Such cases serve as excellent points of discussion to give students a better understanding of what is and is not ethical conduct. In other cases in which the violation is obvious, such as in situations where the authors falsified data, it is still important to note that some authors may refuse retraction (e.g., Oransky, 2012). This could serve as another discussion point for the class by asking: What are the possible reasons why not every author would agree that the paper needed to be retracted? Frequently, it is even possible to find the rationale of those authors and delve further into their thought processes, giving students more chances to discuss the specifics of a particular situation (e.g., Oransky, 2012). From retracted articles it is easy to stimulate discussion on a variety of issues, even if the specific methods might be too advanced. We have found that the best way to help define ethics is to show examples of ethical failures. Students can clearly see what is wrong in most of the retracted papers, and when they cannot see why a paper has been retracted, it becomes an important teaching tool to help them understand ethical conduct.
Stephen Burnett (firstname.lastname@example.org) is a professor of biology, Richard H. Singiser is an associate professor of chemistry, and Caroline Clower is an associate professor of chemistry, all in the Department of Natural Sciences at Clayton State University in Morrow, Georgia.