CHAPTER THREE: RESEARCH DESIGN (CONT.)

Methodology

Student Profiles: triangulation and validity

Participant-observation, student journals, questionnaires and interviews were triangulated to produce student profiles which described the students' feelings and attitudes toward the project and the motivational effect that publishing had on the students' work ethic. The students' projects were also evaluated for thoroughness. Triangulation is the process of comparing the data obtained from two or more contrasting methods. It is a “multimethod” approach to conducting research: if the outcomes of two or more different methods produce consistent results, the researcher will be more confident that his/her findings are valid (Cohen & Manion, 1994, p. 234). The student profiles consisted of answers to the guiding questions from the first questionnaire, positive as well as negative attributes concerning attitude from the field notes, a critique of each students' science fair project from the field notes, anecdotes from the interviews and predominating themes from the students' journals. At the onset, the researcher assumed that a high percentage of student profiles would yield the same results; however, there was no guarantee that the data in the student profiles would be consistent nor that the student profiles would accurately reflect the feelings of the subjects. Therefore it was necessary for each student to read his/her own student profile and then answer a questionnaire in order to ascertain its accuracy.

Questionnaires

Two questionnaires were given to the students while in class. The first consisted of unstructured questions and was conducted in the publishing phase of the project. The guiding questions used to construct the first questionnaire were “Does publishing your science fair project motivate you to do better work?” and “Does publishing your science fair project change your attitude toward doing a science fair project?” The guiding questions were then “operationalized” in order to formulate more specific subsidiary questions used to answer the guiding questions; subsidiary questions are important, can be measured through observation and match observable phenomenon that is described in the literature review (Cox, 1996, pp. 3-7). Once the students learned the software and were working on their own, the first questionnaire was administered. The data from each questionnaire were compiled and included in the student profiles.

Questionnaire 1:

  1. What use of the computer lab did you make outside of class? How often?
  2. Compare and contrast your behavior between working in the computer lab (class time) and doing regular physics work in the classroom.
  3. In what ways has using the computer lab affected your project?
  4. How did you help others on their projects?
  5. What help did you receive on your project from your classmates?
  6. What are some of the science fair activities you enjoy doing?
  7. What are some of the science fair activities you do not enjoy doing?
  8. How did you feel about including your email address on your Web pages and why?
  9. Do you feel that you have become an expert in the topic you chose? Why or why not?
  10. Do you think the projects in your science class have helped you to understand science better? Please explain.

The second questionnaire was given at the very end of the project and was used to judge the validity of the student profiles. The subjects had the opportunity to review their student profiles from the study to get their reaction or “respondent validation” (Cohen & Manion, 1994, p. 241). The questionnaire consisted of two Likert scale questions and six open-ended questions.

Questionnaire 2:

  1. The information contained in your student profile accurately reflects the feelings you had during the project. Circle one: strongly agree; somewhat agree; neither agree nor disagree; somewhat disagree; and strongly disagree
  2. The information contained in your student profile accurately reflects the attitude you had during the project. Circle one: strongly agree; somewhat agree; neither agree nor disagree; somewhat disagree; and strongly disagree
  3. How have your feelings changed? (open-ended)
  4. How has your attitude changed? (open-ended)
  5. Write how you feel about the things that are accurate. (open-ended)
  6. Write how you feel about the things that are inaccurate. (open-ended)
  7. Other than what appears in your student profile, are there other ways in which you were motivated to work harder because your project was published? (open-ended)
  8. Do you think your project has helped you to understand science better? Please explain. (open-ended)

Observations

Researcher's Field Notes and Journal

Field notes were taken on a regular basis during class activities in the computer labs for the purpose of evaluating the students' behavior as well as the documentation of the publishing process to serve as an instruction manual for future virtual science fairs. Observations of the students' behavior did not take place until students were comfortable with the software and began to work on their own. Students learned the publishing software during the research, or literature review, phase of their science fair projects. This enabled students to construct the Web pages while they worked on their science fair projects, simultaneously, thus ensuring that the publishing aspect of the project was ongoing. Specific behaviors were noted as recalled by the observer/researcher in a journal at the end of the day rather than taking notes during the activities--there was not adequate time to make such entries while facilitating the class.

Student work was evaluated for its thoroughness which was rated according to the following criteria on a one to five Likert scale and was entered into a database which was sorted by student and criteria. A score of 5 was “best” based on the researcher's ten years of experience with judging science fair projects.

Attendance

Completion of assignments on or before due dates

Directions are followed

Literature review represents a complete study

Experiment is unique and original

Student has done his/her own work

Student pays careful attention to spelling and grammar

Student cites sources in the literature review

Discoveries are made that are not readily available to the student

Good use of charts and graphs

Data are interpreted correctly

Student constructs a clever experimental apparatus

Repetitions are made to verify experimental results

Student makes predictions based on analytical techniques

Experiment is applicable to the “Real world”

Report is written clearly

Data from the database concerning the evaluation of student work were sorted by student and incorporated into the student profiles.

Video Tape of the Classroom

Each period spent in the lab during the publishing phase of the project was video taped for the purpose of noting the students' behavior in detail. Notes from the video were anecdotal in form and were incorporated into the student profiles.

Interviews

The purpose of the interviews was to solicit anecdotes from the students. The interviews were informal, conversational in style, unscheduled and were conducted near the end of the project. Anecdotes from the interviews were included in the student profile.

Questions:

  1. Greeting
  2. Ethnographic explanation (project and interview explanation): I want to learn what it is like to do a science fair project from your point of view, how you feel about publishing your work on the World Wide Web and how you feel publishing your work has or has not motivated you to work harder. This interview is part of a research project that I am conducting concerning the motivational effects of publishing student work on the World Wide Web. I may take a few notes which I will be sharing with you when the project is completed. I will not share the information gained in this interview without your permission.
  3. Descriptive question: I am particularly interested in your experiences with the computers in the lab. Please tell me about your science fair project and how the use of the computer lab has facilitated your work.
  4. Descriptive question: How has publishing your science fair project on the World Wide Web influenced your work?
  5. Descriptive question: Tell me about an emotional experience you had while you were working on your virtual science fair project and how it affected your work.
  6. Structural and contrasting questions will be asked for clarification and to keep the subject focused on the description (Spradley, 1979).

Student Journals

Student journals were kept to document the progress of each student's project. Entries were made at the conclusion of classroom activities when specific work had been completed on the projects. Students were directed to make entries and to focus on what had been accomplished and how the student felt emotionally:

Prompts:

  1. Describe the work you accomplished today.
  2. Describe how you felt while you were working on this task--was anything particularly rewarding or frustrating?
  3. What will you be (or not be) looking forward to doing the next time we work on the projects in class?
  4. In what way will your work today in class influence the work you will do tonight (or this week)?

Each of the student's journals was collected at the end of the project and analyzed for themes. Selected journal entries were incorporated into the student profiles.

Ethics

Students had access to only their own student profiles. Once the study was complete, the student profiles became anonymous and were published along with this study--only with the student's permission.

next