The Distributed Mentor Project (DMP) Evaluation Report #1 is the first in a series of three reports as part of a three year evaluation project (1995 - 1998) conducted by the UW-Madison's
A second purpose is to inform and prepare prospective mentors and student participants about the program.
Did the mentee and/or mentor believe that the DMP program helped effect changes in the student's:
Did other factors in the DMP students' experience have an effect on their decision to attend graduate school, including:
The reader should note that it is not necessary to read this report in its entirety nor is it necessary to read the document in a specific order; each piece in the document may be read independently without a loss of context or meaning.
This evaluation report contains three sections. The two sections are based on a qualitative analysis: The Students' Point of View: Issues Involved in Participating in the DMP and The Mentors' Point of View: Issues Involved in Mentoring in the DMP. The purpose of the qualitative section is to present the student and mentor perspectives on the Distributed Mentor Project. The third section of this document consists of the presentation of the results of a survey that we conducted via email with the student and mentor participants in 1994 and 1995. The purpose of the survey was to allow us to triangulate across a range of different data sources during the analysis process.
The qualitative analysis is intended for a varied audience that includes: former and prospective student participants of the DMP, faculty who participated as mentors in the program, faculty who are considering participating in the program, and other interested individuals.
The reader should note that qualitative and quantitative research methods differ not only with respect to data collection but with respect to analysis. Individual interviews allow the researchers to "get inside of" the experiences of these diverse participants. Data collection methods are as open-ended and subject-responsive as feasible to ensure that the experiences of the study participants, not the researchers, are reported. Likewise, analysis processes are fundamentally inductive to ensure that the participants' experiences shape the findings. In practice, this means that the researchers make every effort to at least temporarily suspend the ideas that structured their interview protocols. The analysis of interview transcripts is focused on determining what is most important to the participants. The primary analytical categories that emerge as the researchers process the transcripts are apparent in the table of contents. In contrast to survey methods, these methods do not yield precise, quantitative assessments of the proportion of participants holding pre-specified opinions. However, these methods provide extraordinarily rich information expressing the complexity of the lived experiences of the study participants.
We interviewed ten out of the twenty-eight total 1995 student participants in the summer of 1995. Each student participated in two interviews: one at the beginning of her program and another upon the completion of her program. Interviewing the students at the beginning and end of the program allowed us to observe if and how the students' experiences and attitudes towards graduate school and research in CS&E changed throughout the program. These students will be interviewed a third time in the summer of 1996 in order to determine the long-term effects of the program on their career choices.
We conducted one interview with ten out of the twenty-five total 1994 student participants. (One of these students participated in 1994 and 1995.) The purpose of these interviews was to develop an understanding of their experience in the DMP and also to assess the impact of the program on their career decisions. Only three of the student participants had graduated from their undergraduate institution at the time of the interview and as a result, we were not able to conduct a definitive analysis of the program effect on career decisions. However, our interviews strongly indicated that these students' experiences were similar to those of the 1995 participants and thus served to strengthen our analysis. In addition, many of these students emphasized the effect of their experience in the DMP on their understanding of graduate school and academia reflecting a long-term effect of the program.
We conducted a single interview with nine out of the twenty-five 1995 mentors in the fall of 1995. Four of these mentors participated in the DMP in both 1994 and 1995. We interviewed ten out of the twenty-four 1994 mentors once in the fall of 1995. Six of these mentors participated in the DMP in 1994 and 1995. The purpose of these interviews was to understand the faculty's experiences and attitudes toward mentoring in the DMP.
The survey response rate is as follows:
Twenty-two of the twenty-eight 1995 student participants and eleven of the twenty-five student participants responded to the survey. Two of the students who responded to the survey participated in 1994 and 1995 and we included their response in the 1995 survey results and not in the 1994 survey results. The low response rate among the 1994 results from the difficulty of contacting students who had graduated and did not have email address. Due to the low response rate among the 1994 student participants (44%), we will not present those results.
Twenty-one of the thirty-eight total mentors responded to the survey. Of the thirteen mentors who participated in 1994 only, four responded. Of the eleven mentors who participated in 1994 and 1994, seven responded. Of the fourteen mentors who participated in 1995 only, ten responded.
We will present in the survey results the number of 1994 students who are currently in graduate school or have plans to pursue graduate study.
The verbal quantifiers used in this report are:
Ellipses (...) in quoted material indicate deleted dialogue occurring within the reproduced material. Deletions are made so that the readers can appreciate speakers' views on a particular topic without having to sort through the divergent twists and turns of the raw dialogue. Explanatory words added to quotes appear inside brackets [ ]. The quoted material is presented as faithfully as possible to the speakers' intent. Interview dialogue is marked "I:" to indicate an interviewer's speech and is marked "R:" to indicate the speech of the respondent. In interview passages in which more than one respondent is quoted, "R1:," "R2:," etc. is used.
1Established in August of 1994, the LEAD Center supports individuals engaged in educational reform activities at both the baccalaureate and graduate levels. The LEAD Center focuses on student learning experiences and faculty adaptation and dissemination processes. It provides clients with both summative and "formative" evaluation (defined above).
Research Questions
The following are a set of research questions developed with Professor Anne Condon, Principal Investigator of the CRA-DMP, that inform the evaluation research design:
Guide to the Reader
Intended Audience
A guide to "The Students' Point of View: Issues Involved in Participating in the DMP"
Section I presents students' motivations for participating in the DMP. In section II we discuss the impact of the immersion of students in an academic environment on defining their interests in graduate school and a career in academia. In section III we focus on the importance of students collaborating with and being included as a part of the research team of the mentor. In section IV, we list and discuss the students' perceptions of the roles of their mentor in the DMP. In section V, we present some logistical difficulties that arose for the students. A guide to "The Mentors' Point of View: Issues Involved in Mentoring in the DMP:
Section I presents the mentors' view of the goal of the DMP and how the structure achieves this goal. In section II we focus on the mentors' view of the student participants and how student background and motivation affect the success of the program. Section III presents the two mentor perspectives on the importance of receiving assistance in their research program from the student. In section IV we present three main roles that the mentors assumed during their participation in the DMP. Section V presents strategies that the mentors viewed as essential to creating a successful DMP experience for both the student and mentor. In section VI we present the mentors' suggestions of the ways in which they would like to receive recognition for their participation in the program.Methods
We have thus far pursued these research questions through the use of structured, open-ended interviews and through conducting a survey. In addition, we used a diversely trained research team consisting of an anthropologist and two math graduate students. This enhanced the quality of the research because individuals trained in different disciplines bring together different perspectives in both the areas of data collection as well as in analysis.Open-ended Interviews
We interviewed both mentor and student participants from the 1994 and 1995 program years. The structured open-ended interviews were conducted individually and lasted approximately one hour. The interview protocols for the students and mentors appear in Appendix A. All interviews were recorded and transcribed; an average transcription was twenty single-spaced pages.Survey
A survey was distributed via email to the 1994 and 1995 student and mentor participants in the fall of 1995. Using the survey we were able to collect baseline demographic data on the student participants. The survey was also utilized to determine if the findings from the
qualitative interviews were representative of the experience of the mentor and student participants as a whole. The surveys appear in Appendix B.Notes on the Use of Verbal Quantifiers
Specific verbal quantifiers are used to denote the relative size of a group of participants who presented particular perspectives or described particular experiences in interviews. It is important to note that due to the nature of qualitative interviews, the size of a group who discussed a particular type of experience does not indicate the size of the group who had this type of experience. Although the same interview protocol was used in each interview, respondents' answers often prompted discussion in a particular area that may not have emerged in other interviews.
Notes on Quoted Material
A row of asterisks separating two or more quotes indicates that different interviewees are represented in the quotes.
| Table of Contents | Next Section | Previous Section | Back to the CRA-DMP Home Page |