Journal of College Science Teaching September/October 2011 : Page 54
What Do They Expect? A Comparison of Student Expectations and Outcomes of Undergraduate Research Experiences By Lara Brongo Pacifici and Norman Thomson 7KHEHQH¿FLDORXWFRPHVRI XQGHUJUDGXDWHUHVHDUFK�b;85�c;LQ VFLHQFHKDYHEHHQFRQWLQXRXVO\ VXSSRUWHGLQVFKRODUO\UHVHDUFK 7KHUHSRUWHGRXWFRPHVDUH RIWHQGLVFXVVHGLQWHUPVRI LQVWLWXWLRQDOJRDOVIRUHGXFDWLRQ 7KHSXUSRVHRIWKLVPDQXVFULSW LVWRH[DPLQHWKHRXWFRPHVRI 85LQVFLHQFHLQUHODWLRQWR WKHLQGLYLGXDOH[SHFWDWLRQVRI VWXGHQWVSDUWLFLSDWLQJLQ85 $TXHVWLRQQDLUHRQVWXGHQWV¶ H[SHFWDWLRQVRIWKHLUUHVHDUFK H[SHULHQFHZDVFRPSOHWHGE\ XQGHUJUDGXDWHVFLHQFHPDMRUVDW WKHEHJLQQLQJRIWKHVHPHVWHULQ ZKLFKWKH\ZHUHGRLQJUHVHDUFK 6WXGHQWVFRPSOHWHGDVHFRQG TXHVWLRQQDLUHDWWKHHQGRIWKHLU VHPHVWHURIUHVHDUFKRQWKHLU SHUFHLYHGRXWFRPHVRIWKHLU H[SHULHQFH)ROORZ XSLQWHUYLHZV ZHUHFRQGXFWHGZLWKRIWKH SDUWLFLSDQWVWRJDLQDGHHSHU XQGHUVWDQGLQJRIWKHUHODWLRQVKLS EHWZHHQVWXGHQWV¶H[SHFWDWLRQVDQG RXWFRPHVRI85,QFUHDVHG*3$ DQGKHOSJHWWLQJLQWRJUDGXDWH RUSURIHVVLRQDOVFKRROZHUHWKH WZRRXWFRPHVIRUZKLFKVWXGHQWV UHSRUWHGVLJQL¿FDQWO\KLJKHU RXWFRPHVWKDQH[SHFWDWLRQV 0DNLQJIDFXOW\FRQQHFWLRQVDQG SXEOLVKLQJUHVHDUFKZHUHWKH WZRRXWFRPHVIRUZKLFKUHSRUWHG H[SHFWDWLRQVZHUHVLJQL¿FDQWO\ KLJKHUWKDQRXWFRPHV7KHVHUHVXOWV FDQEHXVHGWRKHOSLQVWLWXWLRQV ¿QH WXQHWKHLU85SURJUDPVWR DGGUHVVVWXGHQWH[SHFWDWLRQV 54 Journal of College Science Teaching ndergraduate research (UR) has consistently pro-duced positive outcomes for participating under-graduate students. Several research studies (Kremmer and Bringle 1990; Ryder, Leach, and Driver 1999; Kardash 2000; Bauer and Bennett 2003; Seymour et al. 2004; Kinkel and Henke 2006; Russell, Hancock, and McCullough 2007) have exam-LQHGWKHEHQH¿WVJDLQHGE\VWXGHQWV through experiences in undergradu-ate science research and have made a strong case for the importance of undergraduate research. In addi-tion to understanding the outcomes of UR experiences in science, we should also concern ourselves with the expectations of students enter-ing UR experiences and the degree to which their experiences and the outcomes of their experiences meet their expectations. Recruiting pro-grams could then be accurately tai-lored to represent research experi-ences, which students could enter with realistic expectations and a clear understanding of the possible outcomes that they may gain through the research experience. Previous studies have determined that the benefits of undergraduate UHVHDUFK LQFOXGH LQFUHDVHV LQ FRQ¿ -dence (Seymour et al. 2004; Russell, Hancock, and McCullough 2007), GPA (Kinkel and Henke 2006), understanding of what it means to be a scientist (Bauer and Bennett 2003; Seymour et al. 2004; Hunter, Laursen, and Seymour 2007; Rus-U sell, Hancock, and McCullough 2007), understanding of aspects of the nature of science (Ryder, Leach, and Driver 1999), critical thinking and problem solving (Bauer 2001; Hunter, Laursen, and Seymour 2008), research and skills (Kremmer and Bringle 1990; Kardash 2000; Bauer and Bennett 2003; Seymour et al. 2004; Hunter, Laursen, and Seymour 2007), communication skills (Bauer and Bennett 2003; Seymour et al. 2004; Hunter, Laursen, and Seymour 2007), and clarity about future educa-tion and careers in science (Kremmer and Bringle 1990; Bauer and Bennett 2003; Seymour et al. 2004; Hunter, Laursen, and Seymour 2007; Russell, Hancock, and McCullough 2007). As Cartwright (2000) explained, UR provides gains for both students and the university, so we should be concerned, therefore, with the ex-pectations and outcomes of both the students and the university. This research sought to illustrate the degree to which expectations of research experiences are met for those doing UR in science at a large research university with the goal of better understanding how the gains that undergraduates expect to ex-perience through research compare and contrast with their perceptions of realized gains after their research experiences. With the understanding gained in answering these questions, undergraduate research experiences could be tailored, promoted, and implemented to best serve all students interested in pursuing UR in science.
What Do They Expect? A Comparison of Student Expectations and Outcomes of Undergraduate Research Experiences
Lara Brongo Pacifici and Norman Thomson
<br /> The beneficial outcomes of undergraduate research (UR) in science have been continuously supported in scholarly research. The reported outcomes are often discussed in terms of institutional goals for education. The purpose of this manuscript is to examine the outcomes of UR in science in relation to the individual expectations of students participating in UR. A questionnaire on students' expectations of their research experience was completed by 26 undergraduate science majors at the beginning of the semester in which they were doing research. Students completed a second questionnaire at the end of their semester of research on their perceived outcomes of their experience. Follow-up interviews were conducted with 11 of the participants to gain a deeper understanding of the relationship between students' expectations and outcomes of UR. Increased GPA and help getting into graduate or professional school were the two outcomes for which students reported significantly higher outcomes than expectations. Making faculty connections and publishing research were the two outcomes for which reported expectations were significantly higher than outcomes. These results can be used to help institutions fine-tune their UR programs to address student expectations.<br /> <br /> Undergraduate research (UR) has consistently produced positive outcomes for participating undergraduate students. Several research studies (Kremmer and Bringle 1990; Ryder, Leach, and Driver 1999; Kardash 2000; Bauer and Bennett 2003; Seymour et al. 2004; Kinkel and Henke 2006; Russell, Hancock, and McCullough 2007) have examined the benefits gained by students through experiences in undergraduate science research and have made a strong case for the importance of undergraduate research. In addition to understanding the outcomes of UR experiences in science, we should also concern ourselves with the expectations of students entering UR experiences and the degree to which their experiences and the outcomes of their experiences meet their expectations. Recruiting programs could then be accurately tailored to represent research experiences, which students could enter with realistic expectations and a clear understanding of the possible outcomes that they may gain through the research experience.<br /> <br /> Previous studies have determined that the benefits of undergraduate research include increases in confidence (Seymour et al. 2004; Russell, Hancock, and McCullough 2007), GPA (Kinkel and Henke 2006), understanding of what it means to be a scientist (Bauer and Bennett 2003; Seymour et al. 2004; Hunter, Laursen, and Seymour 2007; Russell, Hancock, and McCullough 2007), understanding of aspects of the nature of science (Ryder, Leach, and Driver 1999), critical thinking and problem solving (Bauer 2001; Hunter, Laursen, and Seymour 2008), research and skills (Kremmer and Bringle 1990; Kardash 2000; Bauer and Bennett 2003; Seymour et al. 2004; Hunter, Laursen, and Seymour 2007), communication skills (Bauer and Bennett 2003; Seymour et al. 2004; Hunter, Laursen, and Seymour 2007), and clarity about future education and careers in science (Kremmer and Bringle 1990; Bauer and Bennett 2003; Seymour et al. 2004; Hunter, Laursen, and Seymour 2007; Russell, Hancock, and McCullough 2007). As Cartwright (2000) explained, UR provides gains for both students and the university, so we should be concerned, therefore, with the expectations and outcomes of both the students and the university.<br /> <br /> This research sought to illustrate the degree to which expectations of research experiences are met for those doing UR in science at a large research university with the goal of better understanding how the gains that undergraduates expect to experience through research compare and contrast with their perceptions of realized gains after their research experiences. With the understanding gained in answering these questions, undergraduate research experiences could be tailored, promoted, and implemented to best serve all students interested in pursuing UR in science.<br /> <br /> Methods<br /> A mixed-methods approach was used to examine undergraduates’ expectations and outcomes of their science research experiences. Quantitative analysis of questionnaire data measured and compared students’ reported expectations and perceived gains, whereas qualitative analysis of interview data provided a deeper explanation of the relationship between students’ expectations and outcomes. This research was approved by the University of Georgia Institutional Review Board (project number: 2009-10867-1).<br /> <br /> Data collection<br /> An open-ended pilot questionnaire (Appendix A) was administered to 20 undergraduate science majors, and responses from the pilot questionnaire were used to construct 10 Likert scale questions (Appendix B) related to research expectations for the preresearch questionnaire. For example, several students responded in the pilot questionnaire that gaining skills and getting into graduate or professional school are among the advantages of UR, which led to the Likert questions about knowledge, skills, and getting into graduate/ professional school. Ten corresponding items relating to the outcomes of the research experience (Appendix B) were constructed for the postresearch questionnaire. In addition to those 10 items, 4 more items were included to address outcomes identified as important in the literature: critical thinking, problem solving, self-efficacy, and understanding what it means to be a scientist. Twenty-six upper-level science majors who participated in science research during the fall 2009 semester completed both the preresearch and postresearch questionnaires.<br /> <br /> Follow-up interviews were conducted with 11 of the 26 students who completed the questionnaire. The aim was to interview at least one-third of the questionnaire respondents. All 26 questionnaire respondents were asked to participate in follow-up interviews, and 11 of them agreed. Interviews were semistructured with questions derived from the questionnaire responses. Interviews were about 30 minutes in length. In the interviews, students were asked to clarify their questionnaire answers and discuss their expectations of research and how their perceived gains corresponded to those expectations.<br /> <br /> Data analysis<br /> SPSS 17.0 (IBM SPSS 2008) was used to calculate means and standard deviations for the responses to each questionnaire item. Paired, two-tailed, student’s f-tests were used to determine the significance of differences between mean expectations and outcomes.<br /> <br /> Interviews were transcribed and interview data were analyzed using the constant comparative method (Strauss and Corbin 1990), in which comparisons are made during each stage of analysis. Interview transcripts were coded, line by line, and codes were grouped into categories on the basis of their relatedness and the categories identified in the model. Transcript analysis was progressive, meaning that each previously analyzed interview informed the analysis of subsequent interviews. During coding and organization of data, memos were written to elaborate the categories and brainstorm ideas for possible themes tying categories together. Themes were constructed and used to provide deeper insight to the interpretation of the quantitative data provided by the questionnaires.<br /> <br /> Results<br /> Mean expectations and outcomes for the categories with standard error bars are presented in Figure 1. Expectations were greater than outcomes on four items: knowledge gain, peer connections, faculty connections, and publishing, but significant differences only existed between the means for faculty connections and publishing. Outcomes were greater than expectations on four items: skills gain, recommendation letter, GPA increase, and help with graduate or professional school, but a significant difference only existed between the expectations and outcomes for GPA increase and help with graduate or professional school. Means were almost equal for two items: enjoyable and clarify career goals. The mean responses for the four additional outcome items are illustrated in Figure 2. The mean values from these four additional outcomes range from 4.3 to 4.9, which corresponds to agreement and strong agreement that the noted outcome was achieved. When compared with the mean values of outcomes paired with expectations, the mean values for the additional outcomes are comparable to the five highest values of outcomes from the original list paired with expectations. The quantitative data provides a baseline understanding of the correspondence between students’ expectations and outcomes of their UR experiences in science. The following section draws from the qualitative data to elaborate on the categories with data provided by students in their own words during interviews. These qualitative data complement the quantitative data already presented.<br /> <br /> Skills and knowledge<br /> Gaining skills and knowledge were the greatest expectations that students had for their research experiences. The gains in skills and knowledge that students percieved closely aligned with their expectations. Students recognized the knowledge and skill transfer between their research and their classes. Students also had greater confidence in their classes when they had experienced the coursework in their research. One student, Theresa, said, “I’m in microbiology lab now, so I know a lot of the techniques already from the research.” Research provided context for what students learned in their science classes, which made it easier to comprehend.<br /> <br /> The knowledge transfer between my research and classes is amazing. That’s one of the things that I really liked about doing research. I’m taking advanced genetics now and it’s basically all primary literature so when I’m going through it and reading, I actually did a lot of these techniques so I can like see it in my head. In cell bio, we got to the cell cycle and we were learning all these things and I was like—I do that! (Ann)<br /> <br /> Ann, like several other students, was enthusiastic about the correspondence between her research and her classes. Research experience provided students with an advantage going into science courses because of the knowledge and skills gained through research.<br /> <br /> When I started doing research, it put me ahead in my classes—like when I started taking microbiology, I actually understood okay so this is why we do plate counts and this is how we do a plasmid transfer. It was like the research that I did really helped me in those courses. (Kelly)<br /> <br /> The applied, hands-on nature of research helped students grasp the deeper meaning of what they were doing and learning in science.<br /> <br /> I think I learn a lot more in research because in class and in labs you’re kind of just following instructions blindly and not really knowing exactly what you’re doing. You don’t really have an end goal, but with research it’s more practical. So I think I’ve learned a lot more this way. (Michael)<br /> <br /> Students valued the skills and knowledge gained in their research experiences for how it helped them in their science courses. Research experience provided a context to apply class material, which benefited students in both their research and their classes.<br /> <br /> GPA<br /> Participants were focused on the need to maintain a high GPA. They saw research as a lower-stakes academic activity than classes. If a grade was attached to their research experience, the grade was based on their performance in the lab and usually on a final research paper explaining what they did through the course of the semester. Research did not involve traditional tests, such as those that account for the majority of grades in science classes. UR students appreciated this distinction between grades in classes and in research.<br /> <br /> I am really self-concious about my GPA. I think I’m a better researcher than I am in classes, and being able to show myself that I can do research and people can be impressed with it and understand it and I can present it and I have all these abilities that aren’t necessarily reflected in my grades. (Holly)<br /> <br /> Holly illustrated the feeling that research provides an alternative to a high GPA to prove one’s ability and competence.<br /> <br /> Faculty connections and recommendations<br /> After gains in knowledge and skills, students’ highest expectations for their research experiences were for the connections that they would make with faculty. This, however, was also the item with the greatest discrepancy between expectations and outcomes. Students were hopeful going into the UR experience that working side by side with faculty would provide them with a strong faculty contact. Most students, however, interacted far more with graduate students or postdoctoral researchers. Students were thankful for the interactions and help that they received from graduate students and postdocs, but it may not be the same as having a connection with a faculty.<br /> <br /> [The postdoc] understood my inexperience being just a student, so he gave me plenty of background information to read just to get me acquainted with everything before we started and even as we were going through he would explain everything so well because a lot of the stuff I had never worked with before like an autoclave. Dr. B would just come in once in a while and check on me and see how my research article was going because that’s how I was graded. (Tyler)<br /> <br /> Undergraduate researchers were impressed with and respectful of their faculty mentors but rarely spent any time actually interacting with them.<br /> <br /> My research mentor is really smart. It’s scary to talk to him— he’s really brilliant. I was working with a postdoc for about a year and a half and then he moved to Australia, so now I’m working with a grad student. I don’t see my faculty mentor super often. Probably like once every two weeks. He’ll come into the lab and ask questions sometimes. (Julia)<br /> <br /> Julia did not express any real disappointment about the lack of contact with her mentor, but her expectations of the connections she would make with her mentor were far different from what actually happened.<br /> <br /> Publications<br /> Going into research, many students had high hopes of publishing their work. For two of the interviewed students, publications were a possibility, at least in the near future.<br /> <br /> We have like three or four publications in the works. So that gives me a motivation to get in there and do a lot of work. Getting a publication would be pretty awesome. (Matthew)<br /> <br /> For the other students, however, it became evident that the amount of research necessary to write a publication would take them several semesters to accomplish. Logistical constraints, such as time, credit hours, and other responsibilities and interests prevented many of them from continuing in research to the extent that would result in a publication.<br /> <br /> Getting into graduate or professional school<br /> Help getting into graduate or professional school was not a high expectation of undergraduates for their research experience, but it was the second highest outcome reported. Some students went into their research experience for the express purpose of padding their resume to help get into graduate or professional school.<br /> <br /> Getting into medical school is what made me know that I had to do research. My [academic] advisor told me that research helps with medical school a lot, and on the [medical school] website, some of their stuff they look at talks about research.” (Chris)<br /> <br /> Others pursued the experience for alternate reasons and found the strengthening of their resume to be an added benefit. Several participants initially entered research to help them decide whether research was something that they wanted to pursue in the future. They found that not only did research help them decide whether science was part of their future plans, but it also provided more opportunity for them to pursue science and research as they desired.<br /> <br /> Confidence in doing research<br /> Before participation in science research, many of the students interviewed recalled being intimidated and nervous about their abilities to do research. After completion of a semester of research, however, most students expressed gains in confidence in their ability to do science research.<br /> <br /> Another reason for doing research is to increase your confidence and your knowledge of yourself. Research really challenges you because no one has ever done this before so you aren’t working toward some known solution. It’s about seeing how far you can explore this problem, and it’s really limited by your imagination and your capability. (Holly)<br /> <br /> Increase in confidence to do research was the greatest gain reported by participants.<br /> <br /> Being a scientist<br /> After a semester of working side by side with graduate students and research scientists, students gain an understanding of what it is like to be a research scientist in “real life.” This understanding includes an appreciation for the process, commitment, and time involved in being a successful scientist.<br /> <br /> I’ve gained better insight to what scientists go through. A piece of information that’s in my textbook— how many researchers and how many experiments it took to be able to put that statement in a book and be accepted just all the huge range that science has on everyday life. My research has helped me appreciate that a lot. (Tyler)<br /> <br /> Beyond an understanding of what it is like to be a research scientist, some students gained an even broader understanding and appreciation for what it means to have a career and be a part of a community of practice.<br /> <br /> Before this, I’ve never really been put in the position where I didn’t know exactly what I was doing. This is the most real world that I’ve ever been in. Twenty years of my life, everything that I’ve been perfect at or good at only affected me or certain people. Research is something much bigger, so it’s made me more realistic about my expectations that I keep having to tell myself that I made a mistake but it’s not that big of a deal. (Ann)<br /> <br /> The growth that Ann experienced through doing research helped her grasp what it means not only to be a scientist but, more important, what it means to be a responsible, contributing member of society.<br /> <br /> Discussion<br /> Funding sources and accreditation agencies are mainly focused on the specific measurable gains that are experienced through participation in UR. Institutions invest great amounts of time, energy, and money into funding and sustaining programs for undergraduate research, so it is important for them to receive positive feedback and outcomes consistent with their goals in order to continue support of the programs. Focusing solely on outcomes, however, downplays the importance of certain other aspects of students’ experiences with research. Some of students’ expectations of research are not met, whereas others are exceeded by their actual research experience. When expectations are different from outcomes, it may have an effect on which students pursue research and whether they decide to pursue research in the future. Understanding both the outcomes of research experiences and the expectations that students have going into research experiences are, therefore, necessary to wholly understand how to initiate and sustain quality UR programs.<br /> <br /> Significant disparities existed where expectations exceeded outcomes in the areas of faculty connections and publications. After participating in research, students realized that demands on faculty members’ time is great and that graduate students or postdoctoral fellows are often more helpful and more accessible. Hunter, Laursen, and Seymour (2008) explained the importance of the development of collegial relationships with faculty as an outcome of UR. Sixteen percent of students in their study indicated that establishing collegial relationships with faculty was an important outcome of their research experience. We do not know, however, what percentage of students in their study expected establishing collegial relationships going into the UR experience. Although some students do benefit from the relationships they develop with their mentors, others’ expectations are not met. This discrepancy should be more of a concern than a point of pride for program administrators.<br /> <br /> On entering the research experience, many students had hopes of publishing their research, but by the end of the semester realized that publishing is rarely possible in the span of a single semester. This may indicate that undergraduates have unrealistic expectations of both their relationship with the faculty mentor and the possibility of publishing their research. Another way to look at it is as a learning experience. Through these unmet expectations, undergraduates learned more about what it is like to be a research scientist.<br /> <br /> Significant disparities existed in which outcomes exceeded expectations in the areas of the effect of research experiences on GPA and in helping with graduate or professional school. With the data collected in this study, it is impossible to know whether the increased GPAs were due to the grades students received for their research credits or due to the increased knowledge that they brought into their other classes. Future research may seek to elucidate the root of increased GPAs linked to UR experiences in science. Regardless, institutions and undergraduate research programs could capitalize on these lesser-known benefits to recruit more students into research experiences. The effect of future trajectory as far as going to graduate or professional school is often a main focus of research on the benefits of research. The results of this study suggest, however, that students are not fully aware of this benefit when they enter their research experiences. Although it is understood by researchers and probably many faculty mentors to be one of the most important benefits of doing UR in science, students may need to be informed more explicitly of this potential gain.<br /> <br /> The purpose of the multitude of research studies on the benefits of UR in science is to work toward common agreement as to what constitutes success in UR research programs (Hunter, Laursen, and Seymour 2008). By looking only at outcomes, the “success” measured is only taking into account the institutional definition of success. Blanton (2008) cited sustainability as the greatest challenge for undergraduate research programs. Although institutional support is imperative to sustainability, so is student interest and involvement.<br /> <br /> Landrum and Nelson (2002) suggested that evaluations of benefits of UR should be used to provide potential undergraduate research students with examples of the benefits they are likely to gain as the result of participating in UR. Perhaps if this was done more effectively, students would have more realistic expectations of the programs they are entering.<br /> <br /> The results of this research are based on a relatively small number of students at a single institution. The results are certainly not generalizable, without caution or caveats, to all students doing UR in science at all institutions. The results do support, however, the examination of not only the outcomes, but also the expectations of UR in science to assess the overall success of UR programs.<br />